130 5 Evaluation July 18 , 20 17 Erica Smith, MS Evaluation, - PowerPoint PPT Presentation

130 5 Evaluation July 18 , 20 17 Erica Smith, MS Evaluation, Epidemiology, and Data Team Manager Center for Chronic Disese Prevention and Control

Evaluation, Epidem iology, and Data Team • Erica Smith, MS – Manager • Carly Stokum, MPH - 1305 Epidemiologist • Elizabeth Funsch, MPH, MA - 1422 Program Evaluator • Alicia Vooris, MSPH - Program Evaluator • Georgette Lavetsky, MHS - BRFSS Coordinator • Marshall Washick - Graduate Research Assistant

Center Approach Integration and enhanced bidirectional communication to ensure strong evaluation/ epidemiology and program understanding: • Managers meet at least weekly • Data team engaged in program planning and implementation • Data team attends partnership meetings, site visits, monthly contract monitoring calls • Evaluation and program staff meet at least monthly

Role of the Data Team - Exam ples • Provide overall technical support and guidance on evaluation and data • Identify appropriate data sources and targets • Develop reporting templates/ guidance, surveys, and evaluation tools • Review RFPs and new contracts • Assist with IRB processes, as needed • Monitor performance measures and other evaluation indicators • Develop and contribute to plans, reports, and other publications • Support data visualization and presentation

130 5 Process Evaluation Questions 1. How has coordination with critical partners changed due to the implementation of 1305? b. How has working across categorical program areas enhanced coordination with critical partners? 2. How has your organizational structure and approach changed due to the implementation of 1305? b. How has working across categorical program areas increased or decreased operational efficiencies?

Process Evaluation Data Collection • Review the work plan, budget, contracts/ MOUs, and other 1305 documents • Review organizational chart and position recruitments • Key informant interviews • Preliminary findings shared with 1305 staff to cross-check data and fill gaps in information

How has coordination with critical partners changed? • 40 partners critical to successful implementation of 1305 strategies, including 25 contracts/ MOUs • 60% of critical partners worked across 2 or more categorical programs, with frequent overlap between: • School Health and DNPAO • HDSP and Diabetes

How has working across categorical program s enhanced coordination with critical partners? • Streamlined reporting • Aligned with the way partners approach chronic disease • Techniques to be used across strategies • Multiple interventions in one setting • Cross-promotion of programs • Expanded partnerships for individual categorical programs

How has your organizational structure and approach changed? • 1305-Funded Positions Center is organized into 3 teams: Evaluation, Health Systems Health Systems • Community-Clinical Linkages Epidemiology, and Team Manager Coordinator Data Team Manager (Kathleen Graham) (vacant) (Erica Smith) • Health Systems School Health Diabetes • Evaluation, Epidemiology, and Data Epidemiologist Coordinator Coordinator (Carly Stokum) (Caroline Green) (Sue Vaeth) • 90% of 1305-funded staff Partnership positions require work across 2 or Health Policy Analyst Fiscal Officer Coordinator (Berit Dockter) (Robert Bauer) (Christine Boyd) more categorical programs

How has working across categorical program s increased operational efficiencies? • Combined funding opportunities and contracts • Staff time • Shared resources and materials • Combined meetings, trainings, and events

How has working across categorical program s decreased operational efficiencies? • Burdensome budget requirements • Duplicate strategies in Basic and Enhanced made reporting difficult • Sub-contractors sometimes gave one categorical program less emphasis (until addressed by 1305 contract monitor)

Process Evaluation Data Utilization • Shared with CDC • Shared with Chronic Disease managers and leadership • Informed action planning and next steps • Planning for sustainability

DNPAO Evaluation Plan • Domain 2, Strategy 2: Implement Food Service Guidelines/ Nutrition Standards 1. What are the key activities and/ or resources considered critical to the successful implementation of food service guidelines? 2. What are the major facilitators and barriers in implementing this initiative? How were barriers overcome?

DNPAO Data Collection • CDC Worksite Health Score Card data collected through Healthiest Maryland Businesses (HMB) • Monthly Reports and Training and Technical Assistance Logs completed by HMB Regional Coordinators • Additional qualitative data (e.g. Regional Coordinator group calls, interviews, etc.)

DNPAO Data Utilization and Sharing • Informed program planning (e.g. annual goals for Regional Coordinators) • Center’s worksite wellness strategic plan and sustainability • HMB Report this fall to summarize 1305 HMB achievements • Score Card scores, changes to scores over time, and the impact of training and technical assistance

DNPAO Data Utilization and Sharing • 345 businesses, reaching 275,435 Maryland employees, completed the Score Card • Among 104 businesses that have taken the Score Card more than once: • 55 (53%) improved their score in the nutrition section • 24 (44%) received TA from a Regional Coordinator on nutrition • 59 (57%) improved their score in the physical activity section • 25 (42%) received TA from a Regional Coordinator on physical activity Source: HMB data 7/1/2013-1/31/17

School Health Evaluation Plan • Domain 2, Strategy 3: Create Supportive Nutrition Environments in Schools 1. What critical factors or activities influence the successful implementation of nutrition policy and nutrition practice? 2. To what extent has implementation of nutrition policies and nutrition practices increased access to healthier foods and beverages at school?

School Health Data Collection • Training Tracker System shared with Maryland State Department of Education (MSDE) • Local Health Department School and Childcare Wellness grants • Statewide Surveillance • School Health Profiles • Youth Risk Behavior Survey (YRBS) • Maryland Wellness Policies and Practices Project (MWPPP)

School Health Data Utilization and Sharing • MWPPP data briefs and in-person feedback sessions with each jurisdiction • "Implementation of Local Wellness Policies in Schools: Role of School Systems, School Health Councils, and Health Disparities” published in the Journal of School He a lth in October 2016 • Findings informed scope of work for 5 School and Childcare Wellness grants

School Health Data Utilization and Sharing • MWPPP encouraged schools to establish a school based wellness team • Percent of schools that reported having wellness teams increased from 44% in Wave I (2012-2013) to 53% in Wave II (2014-2015) • During the 2014-2015 school year (Wave II data collection): • 42% of schools organized or held activities for staff members to support and promote healthy eating and physical activity (compared to 27% in 2012-2013) • 26% provided training or education to encourage staff to model healthy eating and physical activity behaviors (compared to 20% in 2012-2013) • 26% held activities involving families to support and promote healthy eating and physical activity among students (compared to 21% in 2012-2013)

School Health Data Utilization and Sharing • School Health Interdisciplinary Program (SHIP) Conference • Presentations highlighted MDH-MSDE collaboration and included school health data • 2014 and 2016 YRBS Reports

HDSP Evaluation Plan • Domain 3, Strategy 1: Increase Implementation of Quality Improvement Processes in Health Systems 1. What were the major facilitators and barriers in promoting implementation of quality improvement processes, such as use of EHRs in health systems? How were barriers overcome? 2. How has the relationship between the state health department, health care systems, and other QI/ HIT partners in the state changed as a result of 1305?

HDSP Health System s Definition • Local Health Departments, reaching approximately 24 private practices • Mid-Atlantic Association of Community Health Centers (MACHC), reaching 15 Federally Qualified Health Centers (FQHCs) • Medicaid, reaching 8 Managed Care Organizations (MCOs) • Maryland Learning Collaborative, reaching 52 Patient-Centered Medical Home practices (project w as discontinued)

HDSP Data Collection • Contractor reporting (e.g. Plan-Do-Study-Act cycles (PDSAs), quarterly data reports (including NQF data), final reports) • Meeting evaluations • Academic partner evaluation studies • Payers (e.g. All Payer Claims Database, Medicaid, and Medicare) • Additional qualitative data (e.g. one-on-one monthly calls, monthly Community of Practice Calls, site visits, etc.)