Introduction Asking Good Questions So youve welcomed the client and - PowerPoint PPT Presentation

Introduction Asking Good Questions So you’ve welcomed the client and made them feel comfortable. Now on to the heart of the consultation. Aaron Rendahl What information do we need? original slides by Gary W. Oehlert How do we ask questions to get this information? with revisions by S. Weisberg School of Statistics University of Minnesota This lecture follows Chapter 5 of Derr (2000) and section 2.1 of Cabrera February 10, 2010 and McDougall. STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 1 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 2 / 30 Why is it important to ask good questions? Oximeter An oximeter is a noninvasive device that measures pulse rate and oxygen saturation in blood. With humans, measurement is done using a finger clamp. Better a poor answer to the right question than a good answer to the wrong question. John Tukey Type III error: Providing the right answer to the wrong question. A. W. Kimball Type III errors are a constant risk — we must know the right question! Source: http://en.wikipedia.org/wiki/Pulse oximeter STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 3 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 4 / 30

Oximeter usage with dogs What are the units of the oximeter? Veterinarian wishes to know if the oximeter reading depends on the But ... oximeter returns percent saturation, which is typically in the range position of measurement on a dog. of 70% to nearly 100%. Uses 40 dogs, each measured in 4 fixed locations nearly simultaneously. Need to account for using percentages: probit, logistic, or at least sin − 1 ( √ Analysis is like a randomized complete block design fraction). Statistician suggests RCB Anova, with multiple comparisons, which the client can do on her own. It is very easy to answer the wrong question. Everyone is “happy.” We need to work hard with clients to avoid it. But nothing seems to work right. STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 5 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 6 / 30 What information do we need? Solution? Slow down! Ask questions! What information do we need? Listen carefully! Watch the good Dr. Derr. What information does she ask for? Clarify all terms! Take notes, if needed. Keep records of what you do. STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 7 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 8 / 30

Background Area of application Background helps avoid type III error. What field? Learn about your client’s field. New situation or old? Learn about statistical methods standard in the field. Relevant methods and/or literature? Background reading? Learn the common problems in the field. Ask if you don’t understand. STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 9 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 10 / 30 Client Background Our Contribution What can we bring to the table? How well does the client understand the project? Experimental/survey design and planning How much statistical knowledge does the client have? Data management and quality control General data analysis Will we need to educate the client? Statistical inference What level of statistical sophistication is appropriate for this client? Statistical graphics Report writing Do we need to help the client formalize the problem? STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 11 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 12 / 30

Learn about the project Planning stage Project status, from Chatfield (1995) 1 Project formulation. Wow, you’re lucky! 2 Data collection. Goals and objectives 3 Data coding, entry, quality control. Resource constraints 4 Data exploration. Best way to collect data 5 Model building and inference. Sample size 6 Comparison with prior work. Plan for analysis 7 Reporting results. More generally, planning stage or analysis stage. STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 13 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 14 / 30 What kind of project? What kind of project? Sample survey — help with Designed experiment — help with Target populations Units Variables Treatments Sampling design Responses Sample size Randomization Sources of bias Design Sampling instrument Power Training Coding STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 15 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 16 / 30

What kind of project? Analysis stage More common. You’ll need to determine Goals and objectives Observational study — help with How data were collected Data sources Limitations of data Linkages Was design followed Data quality Can goals be addressed Sources of bias Plan for analysis Triage/damage control Postmortem STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 17 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 18 / 30 Ask ... ... politely ... the statistician, if [s]he is really to assist the scientist, must The statistician will, then if [s]he is wise in the ways of human perform a necessary, but irritatingly annoying task: he must ask beings as well as learned in statistics, ask these questions the scientist impertinent questions. Indeed, the questions, if diplomatically, or even not ask them as questions at all. He may bluntly asked, may appear to be not only impertinent but almost well guide the discussion with the scientist in such a way that the indecently prying — because they deal with the foundations of answers to the questions will be forthcoming without the the scientist’s thinking. By these questions, unsuspected questions having been even explicitly asked. weaknesses in the foundations may be brought to light, and the W. Lurie (1958) exposure of weaknesses in one’s thinking is a rather unpleasant occurance. It is a consultation, not an interrogation. W. Lurie (1958) STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 19 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 20 / 30

How to ask questions? Leading Questions The “desired” answer is implied in the question. Watch “evil” Dr. Derr. I assume that you got informed consent? What kinds of questions did she ask? Why couldn’t she get the information she needed? Were you able to get a random sample from this population? What did the good Dr. Derr do instead, that worked? You often get the desired answer, regardless of the correct answer, so these should be avoided. STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 21 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 22 / 30 Closed Probe Questions Closed Probe Questions Closed probe questions ask for specific information, usually with a Yes or No answer. Use them for obtaining very specific information, but They must be clearly worded. Often begin with ‘Can,’ ‘Did,’ ‘Will,’ ‘Are,’ ‘Do,’ etc. They must be free from jargon. They can seem very intimidating. Did you interview every person on the list? Their answers have a narrow scope. Did you randomize treatments to units? Are there any missing data? STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 23 / 30 STAT8801 (Univ. of Minnesota) Asking Good Questions February 10, 2010 24 / 30