Common Measures: Planning & Implementing a Successful CYFAR - PowerPoint PPT Presentation

Common Measures: Planning & Implementing a Successful CYFAR Evaluation Kyle R. Hawkey krhawkey@umn.edu

Planning for Successful CYFAR Evaluation

Evaluation: What’s in it for me? • Concrete evidence about whether young people are benefiting from your SCP project • Project Improvement • Project Resources • Current/potential participant support • Community relationships • Funding/initiative requirements

CYFAR SCP Evaluation requirements • 5 year logic model • 5 year work plan • Programs are REQUIRED to evaluate their programs using the CYFAR Common Measures, including: • PRE/POST evaluation • Reporting outcomes • Sharing outcomes

Required Common Measures This instrument measures behavior and attitudes related to five constructs Core that are common to the goals of all CYFAR programs: social conscience, Competencies personal values, caring, decision making, and critical thinking. These two separate instruments collect program participant data including Youth & Adult gender, age, grade level, ethnicity, race, military affiliation for both youth Demographics and adults These two separate instruments collect program participant dosage data Youth & Adult including: frequency and length of participation in program, length of participation in 4-H, in-school activities, out-of-school activities, and Participation other activities. This 22-item instrument measures different aspects of program quality, Program including physical/psychological safety, supportive relationships, positive social norms, support for efficacy/mattering, and opportunities for skill Quality building.

Outcome-Specific Common Measures Construct/Topic Version Type(s) Leadership Development High School Nutrition Elementary, Middle School, & High School Parenting Adults Physical Activity Elementary, Middle School, & High School Science Elementary, Middle School, & High School Elementary, Middle School, High School, & Technology Adults Workforce Preparation Middle School & High School

Implementing a Successful CYFAR Evaluation

Resource Page

Survey Builder 1 2 3 Create a Download custom Edit and save your survey: survey using surveys to • Pre/Post vetted your account • PDF files measures

Add a New Survey • Access the “Survey Builder” from the top menu • Click “Add New Survey”

Identify User Status Indicate whether you are: • CYFAR Grantee • YEAK Survey user (4-H Science programs) • Guest

Title Your Survey • Title your survey to locate it in your account once saved • Click “Add Survey” to move to the next step

Select Target Audience Select from: • Middle School • High School • Elementary • Adult

Select Your Measures • Required measures are pre-selected • Select from Common Measures approved for CYFAR

Preview & Reorder • Preview your survey • Drag and drop to reorder • Scales stay in-tact

How will you collect your data? The CYFAR PDTA Team supports data collected using the on-line survey or printable paper/pencil surveys.

Download Survey Files File formats: • Online survey link • PDF • Clicker survey

Elevated Account Privileges • Your account must be elevated by the Evaluation Team in order to: • Download survey files • Access application and software • Upload data files • Access data outputs and reports • Send your username to the CYFAR PDTA Team (cyfarpdta@umn.edu) to have your account elevated

IRB Requirements • CYFAR grantees must consult with their local institution to determine if the collection of CYFAR Common Measures will require IRB approval • UMN’s IRB will not provide multi-institution level oversight and Human Subjects approval for the Common Measures data collection process • Each site will safeguard their data by submitting only de-identified data to the CYFERnetSEARCH.org website

IRB Requirements UMN’s role in the CYFAR Common Measures data collection process does not constitute human subjects research because: • CYFAR Common Measures data will be collected as part of routine program evaluation and is not part of a specific intervention (program or activity) proposed by UMN • CYFAR Common Measures data will be de-identified (stripped of personal information and coded with a unique identifier provided by the site) when it is submitted by CYFAR SCP grantees to the UMN; therefore, UMN will not be able to trace data back to specific individuals and will not be able to identify individual persons based on their data

Common Measures Data Collection Process Phase One Select short- Save and term outcome manage survey Build Logic indicators from Build files (clicker, Model approved survey(s) PDF, online) in CYFAR Common your account Measures Phase Two Review evaluation process including Determine how survey Download survey files will be administered from CYFERnetSEARCH checking with your institution’s IRB

Common Measures Data Collection Process Phase Three Upload data file(s) to Administer CYFERnetSEARCH.org pre-surveys Download data set for grantee use Administer Upload data file(s) to post-surveys CYFERnetSEARCH.org Phase Four GRANTEES: CYFAR PDTA Team: Use CYFERnetSEARCH.org to Use common measure data to create basic site-level reports of provide aggregate-level analysis of common measure data CYFAR initiative

Questions? cyfarpdta@umn.edu Kyle R. Hawkey krhawkey@umn.edu