Survey Design Principles Luke Kachersky, Ph.D.
Why • Understand beyond customer behavior survey? • Perceptions, motives, emotions • Efficient data collection • Scalable deployment • Conclusive
Your ultimate goal Make it easy for people to give you precise, valuable information.
Best Survey Practices Define the issue questions Decide on the right scale responses Design for cooperation + clarity bring it all together Declutter final refinement Deploy to the right people sampling Deduce insights from data analysis
Define the issue
Can you imagine two people interpreting this question differently? What brand of toothpaste do you use? Define: Who? What? Where? When? Define the issue
Can you imagine two people interpreting this question differently? What brand of toothpaste do you personally use? Define: Who? What? Where? When? Define the issue
Can you imagine two people interpreting this question differently? What brand (s) of toothpaste do you personally use? Define: Who? What? Where? When? Define the issue
Can you imagine two people interpreting this question differently? What brand(s) of toothpaste do you personally use at home ? Define: Who? What? Where? When? Define the issue
Can you imagine two people interpreting this question differently? What brand(s) of toothpaste do have you personally used at home in the last month ? Define: Who? What? Where? When? Define the issue
Can you imagine two people interpreting this question differently? What brand(s) of toothpaste have you personally used at home in the last month? Define the issue
Decide on the right scale
Decide on the right scale What brand(s) of toothpaste have you personally used at home in the last month? Structured (fixed choice) or unstructured (open-ended) response?
Decide on the right scale What brand(s) of toothpaste have you personally used at home in the last month? Aim Pepsodent check Aquafresh Rembrandt Structured is easier for all Arm & Hammer Sensodyne that respondents and apply Colgate Tom’s of Maine standardizes data. Crest Other:________
Decide on the right scale Scale Type: What brand(s) of toothpaste have you Nominal personally used at home in the last month? Aim Pepsodent Purpose: check Aquafresh Rembrandt To name/classify all Arm & Hammer Sensodyne that apply Colgate Tom’s of Maine Considerations: Crest Other:________ Mutually exclusive and collectively exhaustive categories
Decide on the right scale Scale Type: In an average week during the last three Ordinal months, how often did you run at least one mile? Purpose: Never To indicate 1-3 times per week relative position 4 or more times per week Consideration: Avoid ambiguous words - sometimes, regularly, etc.
Decide on the right scale Scale Type: Please rank the following sneaker brands in Ordinal - rank terms of comfort, with 1 being the brand you find most comfortable and 3 being the brand Purpose: you find least comfortable. To indicate relative position Adidas New Balance Nike
Decide on the right scale Scale Type: Please rank the following sneaker brands in Ordinal - rank terms of comfort, with 1 being the brand you find most comfortable and 3 being the brand Purpose: you find least comfortable. To indicate relative position Adidas Respondent A 2 New Balance 3 Nike 1 Wears Nike on race days, but mixes in Adidas and NB in training. Likes them all.
Decide on the right scale Scale Type: Please rank the following sneaker brands in Ordinal - rank terms of comfort, with 1 being the brand you find most comfortable and 3 being the brand Purpose: you find least comfortable. To indicate relative position Adidas Respondent B 2 New Balance 3 Nike 1 Runs only in Asics. Suffered injury while wearing NB, found Adidas uncomfortable, and had OK experience with Nike
Decide on the right scale Scale Type: Please rate the following sneaker brands in Interval terms of comfort: Purpose: Neither To indicate uncomfortable Very Somewhat nor Somewhat Very absolute position; adds uncomfortable uncomfortable comfortable comfortable Comfortable Adidas 1 2 3 4 5 distance information New Balance 1 2 3 4 5 Nike 1 2 3 4 5
Decide on the right scale Scale Type: Please rate the following sneaker brands in Interval terms of comfort: Purpose: Neither To indicate uncomfortable Very Somewhat nor Somewhat Very absolute position; adds uncomfortable uncomfortable comfortable comfortable Comfortable A Adidas 1 2 3 4 5 distance information B A New Balance 1 2 3 4 5 B A Nike 1 2 3 4 5 B
Decide on the right scale Ordinal Choose ordinal when... • Absolute value is known or is not of interest vs. • You need people to break ties (reflective of Interval marketplace choices. Choose interval when... • Absolute value is unknown • You want to allow for ties (e.g., feature importance, assessing “must haves”)
Decide on the right scale Scale Type: Please circle the number from 1 to 5 that Interval reflects how you feel about Nike overall: Considerations: Strongly Strongly Number of scale points 1 2 3 4 5 Dislike Like Precision vs. effort LIKING Y Z
Decide on the right scale Scale Type: Please circle the number from 1 to 5 that Interval reflects how you feel about Nike overall: Number of scale points: Strongly Strongly Choose smallest 1 2 3 4 5 Dislike Like number for required precision LIKING Y Z Strongly Strongly 1 2 3 4 5 6 7 8 9 Dislike Like
Decide on the right scale Scale Type: Please circle the number from 1 to 5 that Interval reflects how you feel about Nike overall: Endpoint wording: Choose words that Completely Hopelessly in 1 2 3 4 5 Hate Love invite use of full scale Too strong range Somewhat Somewhat 1 2 3 4 5 Dislike Like Too weak
Design for cooperation and clarity
CONTENT ORGANIZATION Hello! As marketing researchers from Fordham University, we are interested in hearing about your usage and opinions of social media. This research will help advance our knowledge of consumer behavior, which will ultimately be passed on in the classroom and applied in the field. Your participation should take less than 5 minutes. If you have any questions, please contact Luke Introduction/Transitions Kachersky at kachersky@fordham.edu. Thank you for your help! To begin, we’d like to know your general opinion about social media. Please answer the question below. Warm-up question What is your least favorite thing about social media? (unstructured) Please describe in 1-2 sentences in the space below. Fundamental questions - Funnel Now we’d like to find out your opinions about specific social media platforms. For each platform listed on the left, please indicate your feelings toward it by selecting a number between 1 and 5. Classification (demographics) Negative Positive Facebook 1 2 3 4 5 Twitter 1 2 3 4 5 LinkedIn 1 2 3 4 5 Catch-all (unstructured) Design for cooperation and clarity
LAYOUT Hello! As marketing researchers from Fordham University, we are interested in hearing about your usage and opinions of social media. This research will help advance our knowledge of consumer behavior, which will ultimately be passed on in the classroom and applied in the field. Your participation should take less than 5 minutes. If you have any questions, please contact Luke Strategically use fonts to Kachersky at kachersky@fordham.edu. Thank you for your help! indicate instructions, To begin, we’d like to know your general opinion about social media. Please answer the question below. questions, and response What is your least favorite thing about social media? areas Please describe in 1-2 sentences in the space below. Balance page breaks • After each question requires too Now we’d like to find out your opinions about specific social media many clicks platforms. For each platform listed on the left, please indicate your feelings toward it by selecting a number between 1 and 5. • If none, requires too much scrolling Negative Positive • Use a rule (e.g., by content or topic) Facebook 1 2 3 4 5 Twitter 1 2 3 4 5 LinkedIn 1 2 3 4 5 Design for cooperation and clarity
Declutter
Declutter Lean form: question item necessity • Addresses information needs? • How will I use data? Pilot test - e.g., a fundraising model?
Deploy to the right people
Deploy Customer surveys • In-person: higher response, higher social bias • Online: lower response, lower social bias • Recruitment appeals • “To better serve you” • Incentives: straight or drawing (www.random.org) • Protect anonymity and confidentiality by separating incentive fulfillment/ entry form Amazon MTurk.com for quick general population feedback or concept testing Professional panels for other targeted populations
Deduce insights from data
Unstructured (open- ended) responses Text analysis: Word cloud generators such as Wordle.net Deduce insights
Structured (scaled) reponses Frequency distribution/ histogram Descriptive statistics - Mean (average) - Mode (most common) - Range - Top/bottom X Deduce insights
Recommend
More recommend
