Designing a Cross Model Quality Assurance System: Mission Impossible? 1
Introductions 2 Jennifer Torres Julia Heany Cynthia Zagar Tiffany Kostelec Study Principal MHVQAS Lead Home Visiting Coordinator Investigator HFA Reviewer Unit Manager
Agenda for Today 3 Provide a broad understanding of the evidence base behind this work Discuss the laws relative to this project Share the story of the journey Discuss the study and preliminary findings Share initial thinking about what next steps are to make use of the tool moving forward
The Research Base: Our Guiding Light 4
Finding clarity: What’s QA & what’s QI & why do we need both? 5 Quality Assurance: Quality Improvement: Reviewing performance against a set Adjusting processes to better meet of standards the needs of customers, both internal and external Determines whether a program meets Determines whether adjustments to a expectations for quality program result in improvements Uses staff expertise to ensure the program Uses staff experience to improve the provides quality services processes by which the program is implemented Relies on external review and internal Relies on internal teams monitoring Responds to expectations established by Responds to needs of customers and external bodies stakeholders Expectation of MIECHV grant recipients Expectation of MIECHV grant recipients Expectation of Michigan’s Home Visiting Law (PA 291)
Current QA & QI Activities 6 Quality Assurance Quality Improvement Model reviews MHVI Learning Collaborative Contractual monitoring Participation in HV Collaborative Innovation and Improvement Network (CoIIN) Benchmark reporting (under MIECHV) MHVI LIA, Local, and State CQI projects Training & TA Gaps in quality assurance: Does not communicate consistent expectations for home visiting program implementation from the State’s perspective Does not monitor the quality of home visiting implementation across Michigan’s HV system Does not provide a foundation for quality improvement Does not meet legislative or funding requirements
The Law and You 7
Building a System that Supports Quality 8 Quality Assurance Expectations for MI are established through: Michigan’s Home Visiting Law - PA 291 of 2012 Defines the Home Visiting System as the “infrastructure and programs that support and provide home visitation.” Built on the idea that “evidence-based programs have comprehensive home visitation standards that ensure high-quality service delivery and continuous quality improvement...” Requires that state funded HV programs “operate with fidelity to the program or model.” MIECHV Funding Requirements “States must provide a plan for ongoing monitoring of implementation quality” Includes “State’s overall approach to home visiting quality assurance” State’s approach to “maintaining quality assurance”
The Journey of the Development of the Quality System 9
The Shock: 10
Building a System that Supports Quality 11 To build an integrated approach to assuring and improving the quality of Evidence-Based and Promising Home Visiting Programs in Michigan: Formed a workgroup, including model representatives and state partners Explored the evidence base and existing tools Received consultation from national & model experts Identified domains of quality in home visiting implementation Developed cross-model standards and measures under each domain Developed a tool and process for assessing quality Received funding to field test the tool
Quality Domains 12 Selected domains based on indicators of quality and fidelity that were supported by the research and common across models Domains include: Recruitment and Enrollment Home Visitor Caseloads Assessment of Family Needs and Referral to Services Dosage and Duration Home Visit Content Staff Qualifications and Supervision Professional Development Organizational Structure and Support
Quality Standards and Measures 13
Quality Standards and Measures 14
MHVQAS: Supplementary Materials 15 Model Specific Guidance Provides guidance to reviewers for measures where models have specified requirements. Worksheets Reviewers – assists with reviewing site documentation prior to site visit Sites – assists with preparing data for site review Document Checklist List of self-assessment materials to be sent prior to site visit Document Gathering Questions Documentation of issues with gathering materials Site Visit Document Review List of materials to have ready on day of site visit Interview Questions Questions you will be asked during the interview portion of the site visit
Quality Report 16 On-site review summary – rating for each standard
Quality Report 17 • Ratings for each measure, including opportunities for improvement and/or special recognition
The Study Framework and Preliminary Findings 18
Study Questions 19 o Do the tool & procedure produce reliable and valid results? o Are the results similar when the tool is completed by different reviewers? o Do the tool & procedure really measure implementation quality? o Can the tool & procedure be applied across models implemented in Michigan? o What are the costs to the state and to local home visiting programs associated with preparing for and completing the review procedure? o How can the results of the tool be used to improve quality in home visiting at the local and state levels?
Participants 20 Home visiting implementation experts (n=6) Provide feedback on whether the tool captures key concepts in quality through an online survey Reviewers (n=5) Conduct site reviews, track time and resources, and complete satisfaction surveys Home visiting sites (n=8) Participate in site reviews, coordinate home visit observations, provide recent model review, track time and resources, and complete satisfaction surveys
Study Tools 21 Michigan’s Home Visiting Quality Assurance System Tool Implementation Expert Survey Home Visit Rating Scale Satisfaction Surveys Reviewer satisfaction Site satisfaction 6-month follow-up Cost Tracking Tool Reviewer Site
Procedures 22 Review On-site Quality Cost Training Scheduling Surveys Preparation Review Report Tracking • Reviewers • On site review • LIA collects • Documentation • Developed • Completed • LIAs and documentation through throughout reviewers - • LIAs • HOVRS • Interview reviewer process day after observations • Submit 3 • Tour consensus quality report weeks prior to • Submitted 1 • HOVRS process is sent review week after completed • Provided to review online • LIAs - 6 within a month LIA months after of the review review • Phone conference to • Experts discuss
Findings 23 Focus on research question #2 Can the tool and procedure be applied across models implemented in Michigan? To what degree do model review guidance overlap with MHVQAS standards? To what degree were model review findings consistent with MHVQAS findings? Were there areas of the MHVQAS tool where models performed differently? Did models differ in their perceptions of the MHVQAS tool’s reliability and validity, or in their satisfaction with the tool? Did models differ in the cost of preparing for and completing the MHVQAS process? What successes and challenges were associated with testing a cross- model set of standards and measures?
To what degree do model review guidance overlap with MHVQAS standards? 24 Domains Standards MHVQAS MIHP HFA EHS PAT Recruitment and Recruitment and X X X X Enrollment Enrollment Caseloads X X X Caseloads Assessment of Need and Referrals X X X X X Developmental Screening X X X X X Care Coordination Assessment of Need and X Tailoring of Services Design of Services to Address Needs X X X X Goal Setting and Visit X X X X Planning
To what degree do model review guidance overlap with MHVQAS standards? 25 Domains Standards MHVQAS MIHP HFA EHS PAT Dosage X X X X X Retention Dosage and Duration X X X Family Exit and Transition X X X X Plans Use of a Curriculum X X X X X Nutrition Services X Home Visit Content Documentation of Home X X X Visits Family Feedback X X X X X Accommodations X X Family Rights Voluntary Services X Consent X X
To what degree do model review guidance overlap with MHVQAS standards? 26 Domains Standards MHVQAS MIHP HFA EHS PAT Staff Qualifications X X X X X Staff Qualifications and Supervision Supervision X X X Training Professional Development X X X Environmental Health & X Safety Infrastructure X X X Quality Assurance Organizational Structure X X X X and Support Integration into Service X X X System Advisory Group X X X X
To what degree do model standards overlap with MHVQAS standards? 27 There is not a perfect overlap between MHVQAS standards and any other model’s standard The MHVQAS tool does not include any standards that are not included in at least one model Take Home Message: The MHVQAS standards are comprehensive by design, overlapping with and supplementing each of the other models reviewed.
Recommend
More recommend
