Leveraging the Power of Surveys Lillian Thomas Analytics Manager - PowerPoint PPT Presentation

Leveraging the Power of Surveys Lillian Thomas Analytics Manager (Section Chief) HR Systems Analytics & Information Division (HR SAID) Office of Human Resources (OHR) National Institutes of Health (NIH)

Why Use Surveys?  Get more context  Fill the ‘gaps’ of quantitative data  Provide insight into employee opinions  Understand issues and develop solutions  Assess the effectiveness of programs  Improve processes  Track trends 4/3/2018 Leveraging the Power of Surveys 2

Take Kick-Off  Kick-Off Action  Design Survey Survey  Develop Online Analyze/ Life  Implement/Administer Design Report Cycle  Analyze Data/Report Findings  Take Action Implement Develop 4/3/2018 Leveraging the Power of Surveys 3

CASE EXAMPLE Developing the OHR Pulse Survey Get Leadership Buy-In Develop Survey - Questions development - Provide draft survey for leadership review - Test survey Launch Survey Determine Topics - Monitor survey response rates - Continue to promote survey & of Interests encourage participation Promote Survey - Leadership announcing survey coming - Sharing previous survey results/impact if applicable 4/3/2018 Leveraging the Power of Surveys 4

CASE EXAMPLE Reporting for the OHR Pulse Survey Analyze Results Communicate Findings Debrief Leadership Implement Changes Action Planning 4/3/2018 Leveraging the Power of Surveys 5

Survey Life Cycle Best Practices Discuss survey Evaluate Reduce survey- Sponsor to Ensure data is Communicate Kick-Off Design Develop Implement Analyze/Report Take Action free from errors. objectives up survey taker burden. advertise survey results. front and make questions and surveys. Conduct Ensure Take analyses sure they align ensure they confidentiality. appropriate appropriate to with questions are free from action. objectives. and the errors and Communicate analyses. pitfalls. As reports steps taken as mature, find ways to a result of automate the prior surveys. report (or segments) for quicker dissemination. 4/3/2018 Leveraging the Power of Surveys 6

What analyses can we do with engagement surveys?  Basic summary report  Significance testing  Cross-tabulation  Open-text analysis  Trend analysis  Cross analysis with other survey and HR data 4/3/2018 Leveraging the Power of Surveys 7

Examples of Analysis What are people saying in their open text responses? What can we extract from the numerous lines of What is the difference in Thematic/Open-Text texts? supervisor satisfaction between Analysis employees likely to leave in a year and employees intending to stay? Is that difference statistically significant or is that due to chance? How is the organization doing compared to last year? Are there any Significance Testing increases or decreases worth noting? Trend Analysis 4/3/2018 Leveraging the Power of Surveys 8

Survey Data & Analytics OUTCOMES Workforce Planning SURVEY RESULTS DATA/ANALYTICS Retention Strategies Action Planning o Pulse Survey o Workforce o NIH Exit Demographics Employee Engagement Survey o Separation Improved Organizational Process o FEVS Data 4/3/2018 Leveraging the Power of Surveys 9

CASE EXAMPLE Data Driven Action Planning Initiatives/Programs Resulting from Action Planning Based on OHR Pulse Survey HR Opportunity Program OHR Director’s Awards OHR Training Program Ceremony ConnectMore Events 4/3/2018 Leveraging the Power of Surveys 10

Key Takeaways Survey Participation Survey Utility Survey Validity Keep It Simple Advertise Incorporate Multiple Sources - Use succinct questions - Have someone important sponsor the - Consider incorporating other sources - Reduce instrument bias to get more survey of data to gain further insights valid data - Send reminder emails Share Results Provide Confidentiality - Share findings with participants - Preserve confidentiality - Increase participation and valid responses Take Action/Show Commitment - Show participant input is useful and actions will be taken based on survey results - Increase customer buy-in 4/3/2018 Leveraging the Power of Surveys 11

Lillian Thomas Contact Information Analytics Section Chief NIH/OHR/HR SAID thomaslm@mail.nih.gov 301.594.0924 4/3/2018 Leveraging the Power of Surveys 12

Additional Resources: Types of Survey Questions Demographic Questions Select Best & Multi-Select Questions Likert Scale Questions Baseline/Overall Rating Questions Open-Ended Questions 4/3/2018 Leveraging the Power of Surveys 13

Additional Resources: Types of Survey Questions Demographic Questions • Questions that asks respondents their demographic information • IC, Series, Grade, Pay Plan, Tenure • Include demographic questions if they are needed for analyses Please select the employee type that best describes you. o Intern/ Fellow o NIH Employee o Contractor 4/3/2018 Leveraging the Power of Surveys 14

Additional Resources: Types of Survey Questions Select Best & Multi-Select Questions • Select Best: Respondents forced to select only one answer Please select your preferred training delivery method. o Classroom o Online • Multi Select: Respondents can select multiple answers What training areas are you interested in? Select all that apply.  Project Management  Information Systems  Human Resources Management 4/3/2018 Leveraging the Power of Surveys 15

Additional Resources: Types of Survey Questions Likert Scale Questions • Rating questions • Degree of agreement, satisfaction, likelihood, & etc. Please rate your agreement with the following statement: The instructor communicated the objectives effectively. o Strongly Disagree o Disagree o Neutral o Agree o Strongly Agree 4/3/2018 Leveraging the Power of Surveys 16

Additional Resources: Types of Survey Questions Baseline/Overall Rating Questions • Most commonly are likert scale questions • Ask the “overall experience What is your overall satisfaction with the training session. o Very Dissatisfied o Dissatisfied o Neutral o Satisfied o Very Satisfied 4/3/2018 Leveraging the Power of Surveys 17

Additional Resources: Types of Survey Questions Open-Ended Questions • Provides respondents an opportunity to provide some general comments. Please provide any additional comments you have regarding the training session. • Can be used to ask why respondents selected a specific answer. You have indicated dissatisfaction with the statement above, please elaborate on your experience. 4/3/2018 Leveraging the Power of Surveys 18

Additional Resources: Common Survey Question Issues/Pitfalls Double/Multi-Barreled Questions Biased/Leading Questions Categories not Mutually Exclusive Categories are not Exhaustive Double Negative Questions Open-Ended Question that can be answered with yes/no 4/3/2018 Leveraging the Power of Surveys 19

Additional Resources: Common Survey Question Issues/Pitfalls Double/Multi-Barreled Questions Bad example: Please rate how satisfied you were with the training content, instructor and materials. Good example: Please rate your satisfaction with the following:  Training content  The instructor(s)  Training materials provided Tips on how to avoid error • Look for ‘and’, ‘or’ in the question • Break down the question and test whether you could answer each piece separately 4/3/2018 Leveraging the Power of Surveys 20

Additional Resources: Common Survey Question Issues/Pitfalls Biased/Leading Questions Bad example: On average, a hands-on training format is rated the most preferred. What type of training format do you prefer? o Hands-On o Lecture o Online Good example: What type of training format do you prefer the most? o Hands-On o Lecture o Online Tips on how to avoid error • Avoid including opinions and supporting information • Keep it simple; Only include the question at hand 4/3/2018 Leveraging the Power of Surveys 21

Additional Resources: Common Survey Question Issues/Pitfalls Categories not Mutually Exclusive Good Example: Approximately how many in Bad Example : Approximately how many in classroom trainings do you attend per year? classroom trainings do you attend per year? o 0-1 o 0-1 o 1-3 o 2-3 o 3-5 o 4-5 o 5 or more o 6 or more Tips on how to avoid error • Look for response options which overlap or repeat • Identify questions where a single response could fall into more than one response option/category; separate options into independent categories 4/3/2018 Leveraging the Power of Surveys 22

Additional Resources: Common Survey Question Issues/Pitfalls Categories are not Exhaustive Bad Example: How many years have you Good example: How many years have you worked at NIH? worked at NIH? o 1-4 o Less than 1 year o 6-8 o 1 year – Less than 5 years o 10-12 o 5 years – Less than 8 years o 8 years – Less than 12 years o 12 years or more Tips on how to avoid error • Look for gaps between options Look for limits which may exist in the range of response • • Look for a potential to include an “Other” option • When applicable, add a text box (“Please Specify”) if interested in gathering more detailed information 4/3/2018 Leveraging the Power of Surveys 23