Researching Research: what academics want from the Web James A. J. - PowerPoint PPT Presentation

Researching Research: what academics want from the Web James A. J. Wilson James.wilson@oucs.ox.ac.uk

From the RDN to 'Intute' • The Resource Discovery Network, 1999-2006. R.I.P • Intute, 2006- – From 8 hubs to 4 'Subject Groups' – Humbul to merge with Artifact • A big 'thank you' to 'The Big Idea' for coming up with the name 'Intute' – We did a quick user survey on that, too

The Research Support Theme • Each Subject Group expected to manage one or two cross-group 'themes' • July 2005: Mike Fraser writes draft proposal for the RDN 'Research Support' theme – Recommend priorities and emphases which should be given to collection and service development – Help implement operational and communication strategies to encourage participation of researchers – Make recommendations to the RDN training theme – Raise awareness of RDN within research councils, JCSR, and other policy makers for research support

Research Survey • On March 12th, 2006, the RDN Research Survey is launched on the WWW – http://www.humbul.ac.uk/rdn/survey/ • Instant response from excited researchers – 350 responses in little over a month • Possible conclusions: – Researchers have too much time on their hands – Researchers are underpaid and motivated principally by Amazon vouchers

Recognition • Recognition of the RDN or any of its eight Hubs was not especially high Ye s, I use a t le a st one of the se se rvice s re gula rly Ye s, I use a t le a st one of the se se rvice s occa siona lly I've trie d a t le a st one of the se se rvice s be fore , but don't use it now I've he a rd of a t le a st one of the se se rvice s, but ha ve ne ve r trie d it No. I ca n't re ca ll ha ving he a rd of a ny of the se se rvice s

Recognition • Why is recognition not as high as we'd like it to be? – Lack of targeted publicity – Lack of word-of-mouth recommendation • Encouragingly, only 22% of respondents who have used the RDN services have since abandoned us • But there's clearly room for improvement

Resource Discovery • How researchers find online resources for their research 100% 80% 60% 40% 20% 0% e o s r h s e l e l o l t g e g a b a h u h t n o u a i t r o r a i O o p o g h e m Y t G p n s s a e f t k b m i o n e h f i o w y d c l t r r b i e o a s c e n ' e W i y w l o s t b i l r r s u a o o r P r e l e N v a n D i t r n e R g o u P e r r e h u h o T t Y o n A

What do researchers look for in a website? • Here are some common responses: – “Is the author genuine - or a nutcase?” – “How comprehensive is the coverage?” – “Authoritativeness, currency, quality of searching” – “Who runs it, who contributes to content, theoretical orientation” – “Relevance, reliability, trustworthiness” – “Accessibility of information, design of site” – “Is the material peer-reviewed” • What do these have in common?

Strength of Intute • Google cannot guarantee any of them! – Well OK, the cream is likely to rise to the top in some subjects, but not so much in those that are of general interest or taught in schools • So why is everyone using Google? – Because it's really good • But we can do things that Google can't – We're an 'authoritative mentor'

Review preferences • Not every Hub reviews online resources in the same manner – Some are more evaluative than other – Some prefer shorter or longer descriptions • Researchers vary in their preferences: 140 120 Esse ntia lly 100 de scriptive 80 60 S om e e va lua tive 40 re m a rks 20 0 S trongly Not us e ful Not M ode rate ly Us e ful V e ry us e ful! e va lua tive particularly us e ful us e ful De ta ile d Re vie w Brie f Re vie w

Search result preferences • Researchers were also divided as to their preferred way of ordering search results: Alpha be tica l By num be r of tim e s a ke yw ord occurs By popula rity (num be r of 'click-throughs') By use r ra ting of re source By spe cia list ra ting of re source Othe r / Don't know • The devious ones recommended a choice of methods – seems sensible given these results

Value-added services • Respondents were asked to rate 24 potential 'value-added' services on a scale of 1 (not important) to 5 (very important) – These services were suggested by the researchers who were interviewed in the preliminary stages of the study • Almost all the suggested services were received enthusiastically, with most respondents rating them towards the 'important' end of the spectrum

The top 10 most wanted Rank Service Points 1 A directory of bibliographic databases in each subject 1271 A directory of journals, with indicators of subscription numbers and 2 prestige 1213 3 A database of sources of funding and scholarships 1205 4 A Directory of regular conferences, with indicators of size and prestige 1175 Searching and browsing filtered by the time period which the resource 5 concerns 1166 5 Ability to filter out resources not directly applicable to research 1166 Online events calendars - for conferences and other events in each 7 subject area 1157 8 National directory of researchers and research 1155 Data that can be exported for use on other Web sites, or for other uses elsewhere 9 1121 List of resources for graduate skills training - publishing, networking, thesis writing, etc. 10 1104

User contribution • Some of the suggested value-added services would depend on user contributions – Online events calendar – Directory of researchers – User rating or review feature • Turn Intute from a provider into a community? – Amazon is probably the most significant model – Means to establish market dominance – So much for 'authoritative mentors'

Willing and able • Surprisingly, a majority of respondents said they would be willing to contribute Yes Would you be willing to contribute your details to a national directory of researchers? Maybe N o Yes Yes, if large Would you be willing to contribute details of events you were organising to an online events calendar? Maybe N o Yes If notable Would you contribute website ratings if we implemented a user-review feature? N o

Able and willing? • Of course, they may have been lying – 27% of respondents said that they didn't put their research on the Web because they didn't have time, 26% 'Just never got around to it' – And people filling in online surveys are likely to be the keenies in any case

Next steps • Split out Hub by Hub findings • Persuade Intute to actually respond to some of the findings – Some (such as the directory of bibliographic databases) are easy hits • Because we've already got this data for some Hubs – Others (such as evaluating journals) are going to prove extremely tough – Yet others might require a leap of faith and an initial kick start

Lessons learned • On the whole, the online survey has been a success – Initial interviews were a good idea • Needed better coordination of publicity – Some Hubs drew many responses, some very few • Would have benefited from tracking referrals – Measuring proportion of respondents arriving via Hubs would have given more accurate picture of recognition