THE IMPORTANCE OF MEASURING OUTCOMES IN PBIS ORGANIZATIONS SERVING - PowerPoint PPT Presentation

THE IMPORTANCE OF MEASURING OUTCOMES IN PBIS ORGANIZATIONS SERVING INDIVIDUALS WITH I/DD 17 TH INTERNATIONAL CONFERENCE ON POSITIVE BEHAVIOR SUPPORT ROBERT PUTNAM, STEWART SHEAR, MEG DEPASQUALE, JENNIFER JEFFREY-PEARSALL MARCH 13, 2020 www.devereux.org

ABOUT Devereux Advanced Behavioral Health Devereux was founded in 1912 by special n education pioneer Helena T. Devereux, a teacher in a south Philadelphia public school Today, we are a recognized national provider for individuals, n families, schools and communities, serving many of the most vulnerable members of our society in the areas of: Autism £ Intellectual/developmental disabilities £ Behavioral Health £ Dual Diagnosis (I/DD and MH) £ Brain Injury £ Emotional Behavior Disorders £ Our unique model integrates the latest scientific and medical advancements, with a best practices framework to provide practical, effective and efficient care, for over 25,000 children and adults every year .

PROGRAMS AND SERVICES n As part of our continuum of care, our 7,500+ team members across 13 states support and empower children, adolescents and adults through services such as: £ Acute care £ Treatment in residential settings £ Community-based living £ Supportive living £ Special education day schools £ Foster care £ Employment and transitional services £ Vocational training £ School and Family consultation £ Outpatient and specialized programs

Devereux Positive Behavior Intervention and Supports (D-PBIS) n Devereux began development of PBIS in the schools in the local Philadelphia area in 1999 n PBIS was adopted by the organization in 2012 as a framework for service provision (D-PBIS) for all centers n Development of a D-PBIS framework for adults with intellectual and developmental disabilities began in 2013 n Roll out of the first D-PBIS I/DD pilot began in the late fall of 2014 n In 2020 there are six Devereux centers implementing the D-PBIS I/DD framework with two other centers in development

D-PBIS I/DD Impact on Outcomes When implementing D-PBIS I/DD there are at least two questions that need to be considered. £ Is D-PBIS I/DD being implemented with fidelity? £ What impact does D-PBIS I/DD have on targeted outcomes?

Question 1 Is D-PBIS I/DD being implemented with fidelity? Yes No Analyze Analyze Fidelity Outcomes Data

D-PBIS I/DD Fidelity Measures n Implementation Fidelity £ Program implementation - engagement, interactions, materials (Reid and Parsons 1990) £ Instructional fidelity – the teaching abilities of staff £ Acknowledgements – for individuals and staff £ Compliance checks – frequency of coaching per residence £ Home Spot Check – program generalization of D-PBIS I/DD £ Tiered Fidelity Inventory (TFI) n D-PBIS I/DD Satisfaction – every 90 days for first 2 years £ Individual £ Staff £ Consultation (monthly)

Scope of Fidelity Analysis n Organizational Fidelity – measurement across all centers n Center Fidelity - measurement within one center across all programs combined n Within Center Fidelity £ By Program Cluster £ By Individual Program

D-PBIS I/DD n The following examples compare the fidelity data between the Devereux Adult Services center and an individual residence noted for significant challenging behaviors £ The implementation goal is 80% for materials, interactions, and engagement £ The instructional goal for staff teaching abilities is 90% £ The staff, individual and consultation satisfaction goal is 80%

Positive Interactions of Staff with Individuals (4:1 Ratio or 80%)

Engagement Rates of Individuals (80%)

Instructional Integrity of Staff (90%)

Acknowledgements of Individuals and Staff

Staff Satisfaction with D-PBIS (80%)

Compliance Monitoring

Home Spot Check n Measures Tier1generalization of D-PBIS in the home n Can be an announced or unannounced visit to the home n Takes about 10 minutes to complete n Has total scores for the home environment and D- PBIS I/DD implementation n Can be used for an item analysis n Used in conjunction with the program and instructional data

Residence: _______________________ Date: ____________ Reviewer: _______________________________ Staff on shift (circle name if PRN): ______________________ Supervisor present: Y N Clinician present: Y N Yes = Element being implemented No = Element not being implemented NA = Element not applicable for residence or not yet implemented Category YES NO NA A. D-PBIS I/DD Information Is there a current weekly Program Schedule posted? Is there a current activity calendar? Are the activities marked on the calendar for the designated activity times? Is the Teaching Matrix posted? Are the identified lessons to be taught and the lessons completed marked on the Teaching Matrix? Are the nine Key Signs posted? Are the D-PBIS posters visible in the home? Calculate Percentage: # Yes / # Yes + # No = ________% B. D-PBIS I/DD Implementation Is the activity schedule being followed? Are materials available for all of the individuals? Are staff interacting positively with the individuals? Are the individuals engaged in an activity? Acknowledgement cards: Acknowledgement cards are carried by staff on shift Acknowledgement Cards are visible in the drawing container Staff reports monthly acknowledgement for using Acknowledgement cards Individual or staff report drawings for the individuals occur every 1 or 2 weeks House meeting minutes are recorded at least weekly – review the minutes Program Consultation includes Performance Feedback Errorless learning is correctly demonstrated when teaching with lesson plans System self-management is set up correctly (behavior goal, #days, tracking system, celebration) Questions to ask: If Performance training is used in the community, describe how(identify behavior, prompt/praise/summary) If Corrective instruction is used, ask when do you use it (in the moment when behavior occurs) C. D-PBIS I/DD Non-verbal Non-verbal – Picture activity schedule posted? Non-Verbal – Individual communication systems? Non-verbal – Minimal verbal instruction (gestural or modeled prompts) Calculate Percentage: # Yes / # Yes + # No = ________%

Tiered Fidelity Inventory n Based on the Tiered Fidelity Inventory form for I/DD £ (R. Horner and B. Putnam) n The Devereux adapted version of the TFI includes changes that reflect contextual variables, processes, and policies. £ In pilot stage for inter-rater reliability and internal consistency for Tier 1 £ Will be used as a an internal and external assessment annually £ Can be used as a repeated measure

Question 2 If D-PBIS I/DD is being implemented with fidelity then: n What impact is the D-PBIS I/DD framework having on the targeted outcomes? “Outcomes are the changes you expect to result from your program. These can be changes in individuals, systems, policies or organizations that you seek to achieve.” (Oxford Dictionary, 2020)

D-PBIS I/DD Outcomes n Outcomes at Devereux for D-PBIS I/DD are the same across all centers serving adults n Key Outcome Targets £ Physical interventions - restraints £ Major Incidents - Challenging behaviors £ Positive Behavior Support Plans with restrictive procedures £ Challenging and Positive Behavior trends £ Quality of Life £ Progress on ISP goals £ Staff Retention £ Workers Compensation

Outcomes and Evaluation n Evaluating outcomes and the impact of PBIS I/DD £ Frequency of data collection £ Frequency of data analysis £ Analysis methodology used at the Leadership and Treatment team levels £ Solutions focused £ Follow up and feedback £ Acknowledgements

Personal Emergency Interventions (PEIs) 2014-2019 70 65 60 50 41 40 40 36 32 30 26 25 19 20 17 16 14 10 10 0 2014 2015 2016 2017 2018 2019 Adult Services Ferdinand Linear (Adult Services) Linear (Ferdinand)

Personal Emergency Interventions 2019 2019 Personal Emergency Intervention Adult Services=32 Ferdinand=17 5 4 4 4 4 4 3 3 3 3 3 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 h l y e y y y t r r r r i s e e e e r r c r a n l u b a a p u b b b r M u a J g m o m u u A m J M u n r t b e c e e A a O t v c e p J o e F e D N S Adult Services Ferdinand Linear (Adult Services) Linear (Ferdinand)

Major Challenging Behaviors 2019 2019 Major Behaviors AWOL, Aggression, Property Destruction, Highly Risky Behaviors, Unsafe Vehicle Behavior, Sexual, Suicidal Adult Services=295 Ferdinand=48 40 34 35 30 28 30 25 25 25 25 23 23 23 21 20 18 20 15 10 6 6 6 5 4 4 4 4 5 3 2 2 2 0 h y e y y y t r r r r l s e e e e r r c i a n l r u b b a a u b b r p M u a g J m o m m u u J A M u n r t b e c e e A a O v t c e p J e o F e D N S Adult Services Ferdinand Linear (Adult Services) Linear (Ferdinand )

Frequency of Aggression Per Month 2019 2019 Aggression Adult Services=216 Ferdinand=36 30 26 25 23 22 21 20 17 17 17 17 16 14 14 15 12 10 6 5 4 4 4 5 3 2 2 2 2 2 0 y y h y e y t r r r r l s e e e e c r r i a n l r u a r u a u b b b b p a M g J J m m u u o m u M A n r t A b e c e e a t O v c e J p e o F e N D S Adult Services Ferdinand Linear (Adult Services) Linear (Ferdinand )