BUILDING A CROWDSOURCING PROGRAM at FDA FROM MARKET RESEARCH TO A - PowerPoint PPT Presentation

BUILDING A CROWDSOURCING PROGRAM at FDA FROM MARKET RESEARCH TO A SYSTEMATIC PROGRAM Mark Ascione and Kim Taylor Office of Strategic Programs (OSP) Center for Drug Evaluation and Research (CDER) Food & Drug Administration (FDA)

Here’s what we want you to learn from our story 2 1 3 Building Leveraging Increasing value excitement around standardized tools through program and engagement for consistent results improvements in a crowdsourcing and increased program bandwidth

Today’s agenda 1 Introducing CDER Crowdsourcing 2 Leveraging early successes to establish best practices 3 Developing tools to support increased demand 4 Using learning from all of you to increase value 5 Q&A

Our program was designed to enhance intra-center communication and engagement • Idea sparked by Partnership for Public Service’s “Effective Internal Background Communication Strategies Workshop” • Crowdsourced the federal ideation community in designing program • Drive ideas and information up to senior leadership Goals • Engage employees Operate as part-time crowdsourcers • Execution Focus on short-term challenges (two • weeks) sponsored by senior leaders

E a r l y S u c c e s s Our early success was driven by thoughtful challenges, communications and incentives Example: Office Retreat Planning Challenges Challenge • Engaged executive sponsors and moderators Objective: Better understand • • Focused topics with specific questions office issues to tee up appropriate for crowdsourced conversation discussion at upcoming all- • Interested target audiences office retreat Incentive: Office-wide one- • Communications hour early dismissal • Developed by communications professional • Consider target audience Participation rate: 80% • • Include multiple modes and touch points Tangible results: New office • newsletter, recognition process and recurring All- • Targeted toward participation and/or quality Incentives Hands events idea generation • Include creative approaches appropriate for federal space

C o n t i n u e d S u c c e s s Increased demand and limited resources required standardized tools for continued success Example: Pharmacy Preceptor Challenge Initiation Template • Outlines critical components of each challenge: challenge objective, Objective: Solicit ideas to • audience, questions, incentive, etc. enable more pharmacy • Informs planning conversations with preceptor students at CDER executive sponsor and moderator Incentive: Three four-hour • time-off awards Communication Plan Template • Contains suggested communication Evaluation criteria: Focus, • touch-points customizable to challenge timeliness, feasibility, • Spans pre and post challenge, including popularity communication of any implementation plans Tangible results: overhauled • preceptor program process Moderator Checklist • Contains moderator best practices

Going forward we plan to evaluate our program, connect with customers, and increase value Customer Connection Value Evaluation Marketing segmentation “Stages” Focus groups, interviews and SharePoint implementation and and surveys integration enhanced community design

We will leverage our core P E I S strength by evaluating CDER Program Evaluation and Crowdsourcing! Implementation Staff Crowd • Objective: Understand mindset of different • Consists of five expert user segments, how to drive engagement, evaluators (three of and how to enhance CDER crowdsourcing whom form • Method: Surveys/Focus groups Crowdsourcing team) Moderators • Conducts and • Objective: Learn what went well, what we manages major can improve, and impact of implemented ideas internal evaluations • Method: Surveys/Focus groups related to key programs and Sponsors initiatives • Objective: Discuss value of crowdsourcing, how to increase presence within CDER, and impact of implemented ideas • Method: Interviews

We will utilize the “ lurker ” U s e r s e g m e n t s metric as we increase user • Lurkers: Users who sign engagement in to a challenge but do not participate • Participants: Users who submit ideas, comment, or vote Lurker once or twice but no Participant more Super • Super-crowdsourcers: crowdsourcer Users who log in repeatedly and participate more than twice Engagement

We will bring crowdsourcing O p e n i d e a t i o n to where staff does its work • CDER’s large staff (5000) … SharePoint ! and bureaucratic culture not ripe for open ideation • Offices are smaller (500- 1000) and have staff dedicated to employee IdeaScale feedback SharePoint • Several offices are developing message board / suggestion box processes • SharePoint integration brings power of IdeaScale right to users

We plan to implement S t a g e s IdeaScale’s “ Stages” to help • Ideate: Generate ideas ideas move past ideation and comments from staff • Staff assessment: Adjudicate ideas through voting (currently combined with ideate) Action Sponsor • Sponsor assessment: Assessment Review and elevate top ideas using ReviewScale Staff (currently done offline) Assessment • Action: Choose ideas Ideate for implementation, consideration, and parking lot

We will create one area on “ A b o u t u s ” our site for crowdsourcing • Our crowdsourcing resources story • Top-line statistics (# of How to participate About us challenges, # of ideas, etc.) Describes how to submit Provides background on an idea, vote, comment the program and • Infographics and participate includes infographics confidentially (overview and how- to) Past challenges Host a challenge Includes previous Educates potential challenge questions, sponsors and moderators ideas, votes, and about how to host a comments challenge

QUESTIONS? • Kim Taylor: Kimberly.Taylor@fda.hhs.gov • Mark Ascione: Mark.Ascione@fds.hhs.gov • Emily Ewing: Emily.Ewing@fda.hhs.gov

Appendix

C D E R C R O W D S O U R C I N G checklist // M O D E R A T O R S ENGAGE MONITOR TRANSFER RESPOND Transfer off-topic or irrelevant Thank users for idea submissions ideas to the “additional input” (comments as warranted) category Ask for elaboration of ideas or comments on submissions that are NOTIFY vague or about which you are curious Notify crowdsourcing team of any inappropriate or negative Specify in a responding comment submissions if an idea submission is already in practice MERGE + DIVERGE Merge ideas that are similar SEND Diverge comments that contain Send relevant ideas to SMEs and new ideas into their own separate ask them to engage on the ideas crowdsourcing site Version 2.0 | 6/1/2018

CDER CROWDSOURCING CHALLENGE INITIATION TEMPLATE What do you plan to get out of this challenge? How will the results be OBJECTIVES used? This list should contain bullets of your overall goals for this challenge. What senior leader sees the need for and will act as the face of this SPONSOR Insert your response here challenge? MODERATOR(S) Who will monitor submissions to make sure they are appropriate and Insert your response here respond to idea submissions? DATES When do you want to hold the challenge? Insert your response here Which office(s) are you targeting? AUDIENCE Insert your response here Is there an incentive for this challenge (based on participation, idea INCENTIVE Insert your response here quality, or some other factor)?

CDER CROWDSOURCING CHALLENGE INITIATION TEMPLATE INTRO PARAGRAPH What is this challenge about? Why is it important? This paragraph will be posted on the home page of the crowdsourcing website and will be used to develop communications plan. Please keep your paragraph to 200 words or less. DRAFT QUESTIONS Ask open-ended questions that require non-’yes’/’no’ answers Limit yourself to 3-5 questions and provide titles for each question.

CDER CROWDSOURCING CHALLENGE INITIATION TEMPLATE OPTIONAL FEATURES Please highlight below the features you would like to utilize for your challenge. - Executive sponsor photo on crowdsourcing site and in communications - Turn off voting - Turn off down voting - Close voting until the last week of challenge after ideas have been reviewed and selected - Add profile questions to track demographics - Incentive for challenge participation - Send final challenge report to participants - Using stages - Posting flyers for challenge promotion