Office of Institutional Research Social Science Research Tyler Portis Design Considerations Meg Nyce Jimmy Huynh Larry Long Alexander Yin
Social Science Research Design 1. De ve lop your Re se ar c h Que stion(s) 2. Se le c t Re se ar c h Appr oac h 3. De te r mine Re c r uitme nt / Sampling Str ate gy 4. De ve lop Instr ume nts & Pr otoc ol 5. Conside r E thic al Implic ations & Biase s 6. Colle c t Data 7. Analyze and Communic ate Re sults 2
Social Science Research Design Research Research Collecting Analyzing Questions Approach Data Data Se le c t a ppro pria te E xe c ute re se a rc h Ana lyze re sults to All re se a rc h sta rts me tho do lo g y to a ppro a c h to dra w c o nc lusio ns with re se a rc h a nswe r q ue stio ns c o lle c t da ta . tha t a nswe rs the q ue stio ns tha t se t (i.e . surve ys, fo c us re se a rc h q ue stio ns. the me tho do lo g y g ro ups, inte rvie ws, a nd a na lysis. me ta -a na lysis). 3
Developing Research Questions • T he re se a rc h q ue stio n is the first ste p a nd ne e ds to b e de ve lo pe d a t the b e g inning o f the pro c e ss. T he da ta yo u c o lle c t sho uld b e dire c tly re la te d to yo ur re se a rc h q ue stio n. • Re se a rc h q ue stio ns sho uld b e c le a r, fo c use d, a nd inte re sting . • Yo ur re se a rc h q ue stio n will dic ta te yo ur re se a rc h a ppro a c h (i.e . fo c us g ro ups, surve ys, a rtic le a na lysis). • 6 Go lde n Rule s fo r Re se a rc h Ob je c tive s (https:/ / www.lipma nhe a rne .c o m/ ho w-to -d e fine - g o o d-re se a rc h-o b je c tive s/ ) 4
Examples • Wha t is the e xpe r ie nc e o f Ho no rs Co lle g e stude nts? (inte rvie ws, fo c us g ro ups) • Wha t is the diffe r e nc e in pe rso na l de ve lo pme nt skills o f intra mura l spo rts pa rtic ipa nts a nd no n-pa rtic ipa nts? (surve y) • Wha t is the r e lationship b e twe e n se x a nd hig h-risk drinking ? (surve y, inte rvie w, fo c us g ro up) 5
Interviews vs Focus Group vs Surveys Interviews Focus Groups Surveys Provide structure through guided Provide quantitative data • • facilitation and one-on-one time • May provide more candid • Removes potential bias from • Provide qualitative data responses since respondents Pros other participants that may occur Give flexibility to dive deeper into complete survey in a more • within a focus group participant responses private, anonymous setting Are flexible in method (i.e. Reach more people • • telephone, in-person, online) simultaneously Lack of anonymity within group Cannot ask follow-up questions • • Can be susceptible to open- may cause participants to not to responses • endedness (and lack of speak up on issues • Do not always give respondents a generalizability in analysis) May provide skewed data due to way to make comments on • Cons • Require careful consideration and outspoken participants who matters not directly asked training of observers to avoid dominate group discussion Can yield low response rates (i.e. • biasing participants Require more investment of time survey fatigue) - thus requires • • Are time consuming and effort in planning, recruiting, proper planning to recruit and facilitating groups participants 6
Determine Recruitment / Sampling Strategy T hing s to c o nside r o nc e yo u’ ve se le c te d a re se a rc h a ppro a c h (thing s yo u ha ve to c o nside r fo r I RB): • Ho w will yo u re c ruit pe o ple / g e t pe o ple to pa rtic ipa te ? • Ho w will yo u g e t e ma ils if yo u ne e d to surve y pe o ple ? • Ho w a re yo u g o ing to pro te c t c o nfide ntia lity / a no nymity? • Do yo u ne e d multiple pe o ple to run a fo c us g ro up? And ho w will yo u e limina te / minimize b ia s? • Ho w muc h mo ne y a re yo u willing to spe nd to re c ruit pe o ple ? • Ho w / whe re will yo u sto re yo ur da ta ? 7
Why send survey to only a sample size instead of population? • Po pula tio n re fe rs to a ll me mb e rs in a g ro up o f inte re st in a study • Sa mple re fe rs to a sma lle r sub se t o f the po pula tio n tha t the study will b e imple me nte d o n • Sa mpling : • Re duc e s c o st o f da ta c o lle c tio n (Po pula tio n studie s re q uire o ne to g e t a re spo nse fro m e ve ryo ne in the de fine d po pula tio n) • Must b e c a re ful o f b ia s (i.e . a re the pe o ple re spo nding to the surve y re pre se nta tive o f the po pula tio n? ) • T his vide o pro vide s mo re info rma tio n a b o ut po pula tio n a nd sa mple s • https:/ / yo utu.b e / e I ZD1BF fw8E 8
Sample Types & Sizes Sample T ype s Sample Size s Clic k the b utto n to g e t a n o nline re so urc e to c a lc ula te sa mple size . Note , not e ve r yone will r e spond to your sur ve y, so you may ne e d to adjust your sample size to ac c ount for r e sponse r ate s. https:/ / yo utu.b e / pT uj57uXWlk 9
Good Survey Design/Questions – Likert Questions Likert questions enable for respondent to rate their opinion about something as opposed to eliminating it to a black and white answer 10
Good Survey Design/Questions – Skip Logic Skip logic is when the designed survey skips survey questions that do not apply to the respondent. If we have a set of questions about living on campus, we want students who live off-campus to not have to answer them. 11
Good Survey Design/Questions – Avoid Leading Questions A leading question is a question that suggests the answer the research is looking to have confirmed. Avoid these questions as they can create untrustworthy data. Image source: https://aea365.org/blog/objection-thats-a- leading-survey-question-by-carla-hillerns-by-carla-hillerns/. 12
Survey Research Resource (Considered the Source for Survey Research) 13
Preparing for Observations, Interviews, and Focus Groups • Ma ke sure yo u ha ve the pro pe r supplie s (i.e . no te pa d, pe ns, ta pe re c o rde r, c o nse nt fo rms, inte rvie w pro to c o l) • Pla n yo ur inte rvie w q ue stio ns a nd de te rmine the struc ture o f yo ur inte rvie w • De te rmine if yo ur inte rvie w is fully-struc ture d o r se mi-struc ture d • F ully-struc ture d inte rvie ws a re stric t a nd fo llo w a sc ript • Se mi-struc ture d inte rvie ws a llo w fo r fo llo w-up q ue stio ns o r a dditio na l pro mpts b ut still fo llo w so me fo rm o f struc ture • Whe n writing inte rvie w q ue stio ns, g o fo r q ue stio ns tha t e lic it a tho ro ug h re spo nse (i.e . a vo id ye s/ no re spo nse s) 14
Consider Ethical Implications & Bias • Pro te c t the c o nfide ntia lity o f yo ur pa rtic ipa nts (i.e . use pse udo nyms fo r q ua lita tive studie s a nd a g g re g a te da ta fo r q ua ntita tive studie s) • Cra ft yo ur q ue stio ns a nd da ta c o lle c tio n struc ture in a wa y tha t pa rtic ipa nts c a n a nswe r ho ne stly a nd o pe nly • I f yo ur to pic is b a nkruptc y, c o nside r ho w yo u will g e t yo ur pa rtic ipa nts to fe e l c o mfo rta b le a nswe ring yo ur q ue stio ns. • Co lle c t yo ur da ta in a wa y tha t is se c ure a nd do e s no t ide ntify yo ur pa rtic ipa nts • Co nside r if yo u a re re q uire d to ha ve a c o unse lo r a t ha nd de pe nding o n yo ur inte rvie w q ue stio ns o r to pic • Be c o nsiste nt with ho w yo u c o lle c t yo ur da ta to a vo id b ia s. • I f yo u ha ve two inte rvie we rs, ho w a re yo u ta king no te s fo r e a c h a nswe r? Are yo u tra nsc rib ing o r inte rpre ting a nswe rs? 15
Strategies to Increase Response Rates • Ma ke yo ur a udie nc e fe e l spe c ia l b y e xpre ssing g e nuine a ppre c ia tio n fo r the ir pa rtic ipa tio n. • “We wa nt to kno w wha t yo u think.” • Pro vide inc e ntive s whe n po ssib le • Gift c a rds, e ntry into ra ffle s, a nd o the r prize s a re e ffe c tive in g e tting re spo nde nts to c o mple te surve ys. • Ma ke yo ur surve y a s sho rt a s po ssib le 16
Collecting Data • F o llo w yo ur e sta b lishe d pro to c o l fo r c o lle c ting da ta . 17
Analyzing Survey Data • Be fo re vie wing re sults, g o b a c k to yo ur ma jo r re se a rc h o b je c tive s a nd write a n o utline fo r yo ur a na lysis to a nswe r tho se q ue stio ns. • Cle a n a nd tra nsfo rm the da ta a s ne c e ssa ry • Use da ta te c hno lo g y like Mic ro so ft E xc e l, SPSS, o r SAS to a nswe r the q ue stio ns yo u ha ve . • E .g . Ca lc ula te me a ns, c ro ss-ta b ula te da ta , a nd c o nduc t t-te sts b a se d o n yo ur re se a rc h q ue stio n • Dra w c o nc lusio ns a b o ut the da ta . • Mo re info rma tio n o n a na lyzing re sults c a n b e fo und a t https:/ / g e tthe ma tic .c o m/ insig hts/ a na lyze -surve y-da ta -surve y-a na lysis/ . 18
Recommend
More recommend
