Are we ready yet? Readiness of community-based organizations to - PowerPoint PPT Presentation

Are we ready yet? Readiness of community-based organizations to adopt and implement evidence-based home visiting programs Sara rah K Kaye ye, P PhD Kaye Implementation & Evaluation, LLC Debo borah F h F. P Perry, P PhD Georgetown University Center for Child and Human Development

Disclaimer/Disclosure This project was supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under grant number D89MC25207 Affordable Care Act–Maternal, Infant and Early Childhood Home Visiting Program (MIECHV). This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by DOH, HRSA, HHS or the U.S. Government. None of the authors have conflicts of interest to disclose.

Maternal, Infant, Early Childhood Home Visiting (MIECHV) • Nationwide expansion of home visiting Implications for capacities of • 18 evidence-based models approved by HRSA; selected by states community-based providers: • Targets high risk service populations  Recruitment and retention of high-risk families MIECHV Benchmark Domains:  Assessment of child family 1. Improvement in maternal and newborn health needs 2. Reduction in child injuries, abuse, and neglect  Referral networks 3. Improved school readiness and achievement  Data collection and reporting 4. Reduction in crime or domestic violence 5. Improved family economic self-sufficiency 6. Improved coordination and referral for other community resources and supports USDHHS, HRSA, MCHB. https://mchb.hrsa.gov/maternal-child-health-initiatives/home-visiting-overview

Evaluation Context R Reach (and Retention) E Effectiveness A Adoption I Implementation M Maintenance Glasgow et al. (1998).

Aims of the Adoption Study POLICY/PRACTICE RESEARCH/EVALUATION Understand strengths and limitations of local Pilot community-engaged research community-based organizations to inform city- methodology to study influences on wide capacity-building efforts community-based organizations’ decisions to adopt EBHV and progress through Think about what providers need to implementation stages implement a new EBHV model: Healthy Families America (HFA) Collect information about organizational-level factors to inform and contextualize other sub- studies Engage and build relationships with provider organizations

Evaluation Methods All community-based child & Phase 1: family serving organizations •Key informant interviews Adoption •Guided by TCU Program Change Model Request for Applications Phase 2: Selected MIECHV local implementing agencies •All staff interviews Implementation •Document Reviews •Focused on HFA essential elements

Community Engagement Methods Design  Selected guiding theory of organizational readiness  Reviewed and provided feedback on interview guides Recruitment  Identified organizations and key informants to include in sampling frame  Participated in qualitative interviews Analysis  Reviewed findings during member checking  Identified actionable recommendations

Sample Characteristics PHASE 1 PHASE 2 Key informants in organizations who provide All program staff in organizations awarded child and family services in the community EBHV contracts following RFA process (n staff =12, n org =10, RR=67%) (n staff =12, n org =2, RR=100%) 2 providers under MIECHV 1 Federally qualified community health center providing comprehensive maternal/child 5 providers of EBHV through other funding health care and social services sources 1 small non-profit providing residential and 3 have never provided EBHV case management services primarily for pregnant and parenting teens 4 had completed the HFA affiliate process

Phase 1 READINESS FOR ADOPTION OF HFA UNDER MIECHV

Phase 1: Readiness for Adoption TCU Program Model for Change—Adapted for DC EBHV TCU Program Model for Change – Constructs Adoption Study Phase 1 Planning and Practice Adoption Implementation Preparation Improvement Staff Program Motivation Resources Costs Attributes Climate Organizational Readiness & Functioning Simpson, D.D. & Flynn, P.M. (2007) Moving innovations into treatment: a stage-based approach to program change. Journal of Substance Abuse Treatment, 33 (2), 111-120.

Phase 1: Qualitative Analysis  Interviews audio recorded and professionally transcribed verbatim  TCU organizational readiness and functioning constructs used as sensitizing concepts during coding  Qualitatively coded in Nvivo to identify themes by one analyst (Co-PI)  Codes with coded quotes reviewed with PI, and codes were refined  Comparative analysis of perspectives based on sample characteristics (e.g., size, experience)  First draft of findings shared with interview participants as part of member checking  Second draft of findings shared with Community Advisory Board to identify actionable recommendations

Results: Motivation Influences on organizational decision making Internal External Pragmatic   Sustaining programs Meeting funding agency expectations   Self-assessed capacity Trend toward “evidence based”  Responsiveness to funding opportunities   Altruistic Fit with mission, values, target Meeting community needs, population  Responsiveness to target population:  Client needs changing demographics   Analytic Feedback from evaluations Community needs assessment   Feedback from clients Analysis of client needs  Feedback from stakeholders

Results: Organizational Capacity Construct Findings Resources -Wide variation in IT infrastructure and data collection/reporting experience -Office tech was sufficient; mobile tech was limited -Office space was mostly sufficient (with creativity) -Internal training resources typically a challenge; orgs partner and share -Some internal QA/QI/Evaluation, some contracted Staff attributes -Range: 1.5 case managers to 102 medical field staff; most have 3-9 home visitors -Partner with local universities for additional support -Many staff do not have college degrees, but many are working toward degrees -Some hire graduates of the home visiting programs -Important to “be from the community” and “ability to go into the home and be comfortable and engaging” Costs -Frequently raised as an area of concern—at start up and ongoing maintenance -Training was identified as a major expenditure; few trainers available

Results: Planning and Preparation All organizations, even with mature programs, identified planning and preparation steps that would need to be taken to successfully implement or scale-up with new MIECHV funding  Recruiting and hiring staff  Training staff  Supporting staff, especially paraprofessionals

Phase 1: Adoption and Implementation Adoption Implementation No Adoption Adoption Planning and Implementation Practice No application to Application for RFA Preparation Providing HFA Improvement RFA Completed the HFA services Using feedback to affiliate process improve HFA delivery 2 8 4 1 0 8 2 1 1 0

Provider-Recommended Supports for Increasing Readiness  Provide funding for EBHV  Provide training citywide  Support a network of organizations and meaningful referral process  Support data collection, data reporting, and evaluation support – and allow organizations to access their own data  Develop opportunities for cross-agency peer support for home visitors  Reinforce outreach attempts through citywide marketing to “get the word out” about home visiting

Phase 2 READINESS FOR IMPLEMENTATION OF HFA UNDER MIECHV

Phase 2 Mixed Methods Analysis  Guided by HFA essential elements; considers how TCU readiness and functioning constructs supports or hinders successful achievement of essential elements  Open-ended questions asked during semi-structured interviews with all staff and supervisors/ program managers  Recorded interviews professionally transcribed  Researchers operationalized indicators of fully ready, ready, and approaching readiness for all essential elements  Interview transcripts and supporting documents (e.g., implementation plan, job descriptions, policies/procedures) and coded by essential element and TCU construct; single analyst (Co-PI)  Codes and quoted text reviewed with PI  Both researchers assigned readiness ratings to both organizations for each HFA essential element

Readiness Rating Scale Symbol Rating Description Staff can articulate processes, procedures, and how they support outcomes Fully Ready + in a way that is consistent with the program theory of change. The organizational infrastructure has been developed to include supports which facilitate high-fidelity implementation. Ready Staff understand their role in supporting implementation. The organization  has sufficient resources to support staff in fulfilling roles and responsibilities. Staff have a general understanding of requirements, but do not demonstrate Approaching - an in-depth understanding of processes. Organizational resources do not Readiness provide the infrastructure necessary for quality and consistency in implementation.