S u r v e y s in marketing research SU R VE Y AN D ME ASU R E ME N - PowerPoint PPT Presentation

S u r v e y s in marketing research SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R George Mo u nt Data anal y tics ed u cator

" On a scale of 1 to 5..." SURVEY AND MEASUREMENT DEVELOPMENT IN R

Anatom y of a s u r v e y SURVEY AND MEASUREMENT DEVELOPMENT IN R

Steps to s u r v e y de v elopment Step 1: Item generation Do the e x perts agree w e ha v e strong items ? 1 Hinkin , Timoth y R . " A brief t u torial on the de v elopment of meas u res for u se in s u r v e y q u estionnaires ." _ Organi z ational research methods _ 1.1 (1998). SURVEY AND MEASUREMENT DEVELOPMENT IN R

Inter - rater reliabilit y: base agreement Base agreement library(irr) agree(experts) Percentage agreement (Tolerance=0) Subjects = 10 Raters = 2 %-agree = 50 SURVEY AND MEASUREMENT DEVELOPMENT IN R

Inter - rater reliabilit y: Cohen ' s kappa Acco u nt for chance : Cohen ' s kappa library(psych) cohen.kappa(experts) Cohen Kappa and Weighted Kappa correlation coefficients and confidence boundaries lower estimate upper unweighted kappa -0.202 0.24 0.69 weighted kappa -0.078 0.42 0.92 Number of subjects = 10 SURVEY AND MEASUREMENT DEVELOPMENT IN R

Interpreting Cohen ' s kappa Weighted kappa interpretation 0 - .39 Poor agreement .4 - .59 Moderate agreement .6 - .79 S u bstantial .8 - 1 Ver y strong SURVEY AND MEASUREMENT DEVELOPMENT IN R

Content v alidit y Ho w essential is each item to meas u ring the concept of interest ? La w she ' s Content Validit y Ratio ( CVR ): What percent of e x perts j u dge that item " essential " to w hat ' s being meas u red ? E − ( N /2) CV R = N /2 N = total n u mber of e x perts E = n u mber w ho rated the item as essential SURVEY AND MEASUREMENT DEVELOPMENT IN R

Meas u ring CVR in R Calc u lating CVR in R : library(psychometric) # Expert 1: Essential # Expert 2: Essential # Expert 3: Not necessary # Expert 4: Useful # Expert 5: Essential CVratio(NTOTAL = 5, NESSENTIAL = 3) [1] 0.2 SURVEY AND MEASUREMENT DEVELOPMENT IN R

CVR interpretation CVR v al u e Interpretation # of e x perts CVR threshold - 1 Perfect consens u s against 5 .99 0 No consens u s 15 .49 1 Perfect consens u s for 40 .29 SURVEY AND MEASUREMENT DEVELOPMENT IN R

E x pert assessment : let ' s practice ! SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R

Meas u rement , v alidit y & reliabilit y SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R George Mo u nt Data anal y tics ed u cator

What is meas u rement ? A process of obser v ing and recording Done w ith an instr u ment Is o u r instr u ment " calibrated ?" SURVEY AND MEASUREMENT DEVELOPMENT IN R

Meas u rement reliabilit y Are w e meas u ring consistentl y? Can w e reprod u ce o u r meas u rements ? SURVEY AND MEASUREMENT DEVELOPMENT IN R

Meas u rement v alidit y Are w e meas u ring w hat w e claim / intend to meas u re ? SURVEY AND MEASUREMENT DEVELOPMENT IN R

Step 2.5: e x plorator y data anal y sis SURVEY AND MEASUREMENT DEVELOPMENT IN R

Step 2.5: e x plorator y data anal y sis SURVEY AND MEASUREMENT DEVELOPMENT IN R

S u r v e y EDA : response freq u encies # Plot response frequencies Checking for : plot(c_sat_likert) Response freq u encies # c_sat is a customer satisfaction survey # Convert items to factors library(dplyr) library(likert) c_sat_likert <- c_sat %>% mutate_if(is.integer, as.factor) %>% likert() SURVEY AND MEASUREMENT DEVELOPMENT IN R

S u r v e y EDA : re v erse coding Checking for : Re v erse - coded items SURVEY AND MEASUREMENT DEVELOPMENT IN R

S u r v e y EDA : re v erse coding library(car) # Recode item 6:"difficult to use" # Not dplyr::recode()! # A "3" response will stay a "3"! c_sat$difficult_to_use_r <- car::recode(c_sat$difficult_to_use, "1 = 5; 2 = 4; 4 = 2; 5 = 1") SURVEY AND MEASUREMENT DEVELOPMENT IN R

S u r v e y EDA : response freq u encies / re v erse coding library(psych) library(dplyr) # Confirm reverse coding c_sat %>% select(difficult_to_use, difficult_to_use_r) %>% response.frequencies() %>% round(2) 1 2 3 4 5 miss difficult_to_use 0.02 0.18 0.57 0.21 0.03 0 difficult_to_use_r 0.03 0.21 0.57 0.18 0.02 0 SURVEY AND MEASUREMENT DEVELOPMENT IN R

S u r v e y EDA : let ' s practice ! SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R

Describing s u r v e y res u lts SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R George Mo u nt Data anal y tics ed u cator

Missingness If < 5% non - response , omit missing cases . If > 5%, back to item generation . # Total number of incomplete cases nrow(na.omit(bad_survey)) # Number of incomplete cases by item colSums(is.na(bad_survey)) item_1 item_2 item_3 item_4 item_5 82 98 78 98 82 75 SURVEY AND MEASUREMENT DEVELOPMENT IN R

Missingness library(Hmisc) # Hierarchical plot -- what values # are missing together? plot(naclus(bad_survey)) SURVEY AND MEASUREMENT DEVELOPMENT IN R

Item correlations Do gro u ps of items correlate highl y w ith each other ... ... and not so high w ith those o u tside the gro u p ? SURVEY AND MEASUREMENT DEVELOPMENT IN R

Item correlations Probability values (Entries above the library(psych) diagonal are adjusted for multiple corr.test(brand_qual[,1:3]) tests.) trendy latest tired_r Call:corr.test(x = brand_qual[, 1:3]) trendy 0 0 0 Correlation matrix latest 0 0 0 trendy latest tired_r tired_r 0 0 0 trendy 1.00 0.76 0.55 latest 0.76 1.00 0.57 To see confidence intervals of the tired_r 0.55 0.57 1.00 correlations, print with the Sample Size short=FALSE option [1] 505 SURVEY AND MEASUREMENT DEVELOPMENT IN R

Item correlations # Visualize item correlations library(corrplot) corrplot(cor(brand_qual), method = "circle") SURVEY AND MEASUREMENT DEVELOPMENT IN R

Spotting " gro u ps " in o u r correlation SURVEY AND MEASUREMENT DEVELOPMENT IN R

Spotting " gro u ps " in o u r correlation SURVEY AND MEASUREMENT DEVELOPMENT IN R

Spotting " gro u ps " in o u r correlation SURVEY AND MEASUREMENT DEVELOPMENT IN R

Let ' s practice ! SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R