Understanding Web Search Behavior for Analytical Tasks Joann Keyton - PowerPoint PPT Presentation

Understanding Web Search Behavior for Analytical Tasks Joann Keyton and Grant Harned Department of Communication April 20, 2016

Across Analyst Domains  Online searching is key to managing uncertainty in fulfilling task demands  Seeded discovery problems  instructions given to find content  Unknown-unknown problems  instructions given to identify factors, activities, events that may present as anomalies, abnormalities and future threats

Uncertainty Management  Uncertainty reduction theory  (Berger & Calabrese, 1975)  Anxiety-uncertainty management theory  (Gudykunst, 1985)  Uncertainty management theory  (Brashers, 2001; Brashers et al., 2000)  Uncertainty = inability to predict one’s attitudes, behavior, and feelings

Why tasks are uncertain  Information is often fragmented and distributed among sources and locations  The best information may be among the many (and nonobvious) sources of online information  Analysts work on complex problems for which inferences about short- and long-term future events are made LAS Update

Anticipatory Intelligence Tradecraft  Anticipatory Intelligence = approach to assessing the future  Covers the three broad tasks  Exploring the distant future for weak signals of change in major conditions  Assessing how dynamic issues may evolve into trends and impact US interests  Analyzing the probability that specific events will occur

How this translates to online searching  Users have some control or potential to manage the breadth and depth of the information-acquisition process  Information seekers have the opportunity to choose  search terms  websites and pages viewed  when a search begins and concludes Parterning with LAS

Searching behavior is influenced by biases and heuristics  least amount of effort  confirmation bias  strategies for finding and managing the volume  technical and authoritative evaluations = assessments of authority and accuracy Parterning with LAS

Research Questions 1. Given an unknown-unknown task as motivation, how do individuals vary in their online searching activity? 2. What is the distribution of dwell time? 3. What is the relationship between the number of sites visited and the number of sentences participants produced in their written reports? 4. What is the relationship between online search behavior and confidence in decision choice? Parterning with LAS

Dwell Time  Length of time a visitor spends on a page  time per page is indicator of greater levels of systematic processing  most significant indicator of document relevance  Web users read quickly, the distribution of dwell time is based on a model of system failure Parterning with LAS

Weibull distributions  Positive and negative aging Weibull distribution  the longer the component has been in service, the more likely it is to fail/the less likely it is to fail  99% of Web pages have a negative aging effect  People use a 10 second view to make a decision if the page is useful/relevant  At 30 second mark, users tend to stay Parterning with LAS

Method in a  51 experimental sessions  70 min  Pretest  Training search  Task 1: search and write draft  Discussion: talk about process or content  Task 2: search and finalize report Parterning with LAS

Participant search behavior in T1  M keywords per search during = 5.28  SD = 3.04; min/max = 1/19  Total of 2,586 searches  M = 19.98 searches per user  Used 13,654 keywords  M = 104.49 keywords per user  99.92% participants used Google as search engine Parterning with LAS

RQ 1: Where did they go? Total M Time Web Pages Domain Type per User Visited Search Engine 185.30 2557 Presidential Candidate-General 260.19 1233 General Information 174.51 1071 Candidate 49.14 400 Political News: Nonpartisan 28.57 173 News 29.19 169 Political News: Conservative 42.76 135 Social Media 23.87 113 Political News: Liberal 23.67 89 Government 17.11 95 Political Party 3.88 56 Images 2.68 49 Political News: Libertarian 6.85 48 Parterning with LAS

RQ 2: Dwell time distribution  Weibull analysis on the six most frequently used domain types  political candidate sites β = 0.73  social media β = 0.75  candidate portals β = 0.77  general information β = 0.85  search engine β = 0.85  news β = 0.91 Parterning with LAS

Parterning with LAS

Interpreting dwell statistic  Shape parameter in the range 0 < β < 1  user will leave a particular web page monotonically decreases with time  negative aging phenomenon  pages relating to political information across a wide range of domain types  Rate of web page abandonment decreases more slowly for news and information sites than for social media and political candidate sites Parterning with LAS

RQ 3: Number of sites visited and number of sentences?  Participants wrote a total of 1302 sentences in the draft reports  M = 10.25, SD = 4.18, min/max = 3/27 per participant  number of URLs visited = proxy for information gained  more sentences written = proxy for cognitive load.  r = -.22, p < .05; search and writing trade-off Parterning with LAS

RQ 4: Online search behavior and confidence?  No relationship between number of URLs visited and confidence in the decision as reported in the report  r = .04, ns Parterning with LAS

Discussion  Participants could manage uncertainty by how  they structured their online searching  created and reported their inferences in writing  or both  Not one participant found the Federal Election Commission Form 2 website  Cognitive misers  only 18% users view 2nd page of search results, 3% viewed 3rd, and 1.5% viewed beyond Parterning with LAS

Takeaways 1  Rate of web page abandonment decreases more slowly for news and information sites than for social media and political candidate sites  According to Liu et al. (2010), participants were more harshly screening candidate portals, social media, and political candidate  Overwhelming presence of Google as a search tool  Google as an authority?  The lack of reliance on the FEC website is ( insert your favorite word ) Parterning with LAS

Takeaways 2  Search strategy/query is contingent to some degree on the type of task a participant is asked to perform  Participants rarely visit beyond the first page of search results  18% viewed 1 st page  3% viewed 3 rd page  1.5% viewed 4 th or beyond.  Confidence seems somewhat arbitrary in non-expert users Parterning with LAS

And there’s much more to do  Linguistic analysis  Draft and final reports  Discussions  Interaction analysis  Discussions  Sequential analyses of types of talk Parterning with LAS