Employee Engagement Surveys: Improving your scores without really - PowerPoint PPT Presentation

1 Employee Engagement Surveys: Improving your scores without really trying Garry Gelade Business Analytic Ltd

2 Outline • How to improve your employee engagement scores without really trying. • How not to be so stupid as to think you could do such a thing.

3 Managing Employee Engagement Annual Survey Develop Reward Intervene Action Improvements Plans Departmental level

4 Focus the Effort! “ One emerging best practice for improving survey effectiveness is to aim follow-up actions more precisely where they are needed rather than across the organization at large. This means making action plans required only for employee units or teams that perform poorly in key areas but not for other groups where survey results reveal a generally healthy productive work environment ”

5 Hopi Rain Dance

6 Rainfall in the Southwest USA [illustrative!] 1.6 Average rainfall in the month following 1.4 the dance is higher than in the month preceding the dance. .... 1.2 ... Because the Indians never perform the rain dance when it’s raining. 1 This gives the false impression 0.8 that the dance is effective. 0.6 0.4 0.2 Average rainfall 0 Before Dance After Dance

7 Engagement Rain Dance “The HR team requests that managers whose scores declined the most or are farthest below regional norms send their impact plans directly to the CEO and Chief operation officer. ‘We have found that by doing this, it’s unlikely for these managers to have low scores again the following year ’, says Church. ‘Managers know we take all plans seriously and hold them accountable for executing them”. www.towerswatson.com/newsletters/strategy-at-work/2830

8 Engagement Rain Dance with Attitude While the top 10 th Bottom 10 th percentile percentile is “remaining stable” has improved an average of 5 points “XYZ’s lowest scoring units received coaching and support last year. Engagement scores improved an average of 14 points in these units (vs 1 point in all other units)”

9 Regression Towards the Mean

10 Galton: “Regression Towards Mediocrity” Tall Parents Population Children Mean Children Short Parents Galton, F. (1886). Regression towards mediocrity in hereditary stature. Journal of the Anthropological Institute, 15, 246-263.

11 Plate IX from Galton’s paper.

12 Mechanism of RTM

13

14 RTM in Best Companies Results http://money.cnn.com/magazines/fortune/bestcompanies/2011/full_list

15 Best Company Survey: 89 of the top 100 US companies in the 2011 assessment also participated in the 2012 assessment The average Top 20 company fell down The average Bottom 20 company rose the rankings by 7.9 places up the rankings by12.6 places

16 RTM in Engagement Survey Data UK retail bank: 495 branches. 5-item engagement scale (Alpha= .94) 3-item Computer Satisfaction scale (Alpha= .79) Year 1 Year 2 difference Engagement Overall Mean (SD) 3.23 (0.47) 3.25 (0.40) +0.02 Top 50 branches 4.04 3.63 -0.40 Bottom 50 branches 2.40 3.00 +0.60 Computer Satisfaction Overall Mean (SD) 3.11 (0.39) 3.59 (0.37) +0.48 Top 50 branches 3.77 3.77 0.00 Bottom 50 branches 2.40 3.62 +1.21

17 Recognizing RTM and correcting for it

18 Recognizing RTM Plot change vs. baseline Low scoring units improve Mean Score Year 1 High scoring units drop

19 Correcting for RTM - ANCOVA Adjusts each unit’s follow -up (e.g. Year 2) measurement according to their baseline (e.g. Year 1) measurement. 𝐺𝑝𝑚𝑚𝑝𝑥𝑣𝑞 = 𝛾 0 + 𝛾 1 𝑐𝑏𝑡𝑓𝑚𝑗𝑜𝑓 − 𝑐𝑏𝑡𝑓𝑚𝑗𝑜𝑓 𝑛𝑓𝑏𝑜 + 𝛾 2 𝑕𝑠𝑝𝑣𝑞 where group = 1 for intervention group, and 0 for no intervention group. The coefficient β 2 is the estimated treatment effect adjusted for RTM. Twisk JWR. Applied Longitudinal Data Analysis for Epidemiology: a Practical Guide . Cambridge University Press, 2003.

20 Correcting for RTM - Mee & Chua t-test for the existence of a treatment effect in the presence of RTM Y 2 = μ + τ + ρ(Y 1 - μ) + ε 1. Calculate X = Y 1 - μ 2. Estimate the parameters β 0 and ρ from the linear regression model of Y 2 on X 3. Estimate the treatment effect by subtracting μ from τ , the estimate of β 0 4. Calculate the test-statistic where s 2 is the mean squared error in the regression analysis of variance, X i denotes the value of X in the i -th case, i = 1,..., n , and is the mean of all X i 5. Compare t with the appropriate t-distribution with ( n -2) degrees of freedom to obtain a p- value p = p(μ). Mee, R. W., & Chua, T. C. (1991). Regression toward the mean and the paired sample t test. The American Statistician , 45 (1), 39-42.

21 Thanks for Listening. We’re Done. Any Questions?