Tier 2 Fidelity Data: Strengthening your Tier 2 PBIS Implementation: - PowerPoint PPT Presentation

Tier 2 Fidelity Data: Strengthening your Tier 2 PBIS Implementation: Using Fidelity Measures to Promote Growth in Tier 2 Implementation March 2020 Celeste Rossetto Dickey, M.Ed. University of Oregon

Introductions and Welcome  School Staff Member  PBIS Coach  Family Member  Tier 2 Team Facilitator  Tier 2 Team Member  Other?

Objectives for today:  Understand the importance of fidelity data  Tier 2 Systems  Tier 2 Interventions  Understand how to use the TFI Tier 2 data for action planning  Become knowledgeable about various Tier 2 Interventions fidelity assessments

Multi-Tiered System of Supports (MTSS) Tertiar iary—intensive, individualized supports 5% of student population at high risk Secondary—targeted, group-based supports 5% - 15% of student population at risk Universal Un al—primary prevention provided to all students, effective for approximately 80% All spec ecialized i inter erven entions a are m e more e effec ective e and m more e durable e with unive vers rsal, s sch chool-wide b behavioral e expecta tations a as a foundation

Overarching Goals at Tier 2 (across interventions) Provide s students w with: o More time a and instruction f for s skill d development o More s structure a and p predictability (during a routine/subject or across the day) o Struct ctured p performance f feedback ck o Sel elf-man anag agement s skills (self-monitoring, self-instruction, self-correction)

Critical Components of Tier 2 1. Standardized r routines a and procedures 1. o Aligned with school-wide expectations o All school staff trained 2. Continuously available w with r rapid s student a acce ccess 2. o Standard is within 3 days of identification 3. Low t teach cher/staff e effort 3. o Typically, less than 10 minutes per teacher per day 4. Home/school lin l linkage 4.

Types of Information (Data) Fidelity Outcome Data Data Did we implement the systems Is the plan resulting in and strategies we agreed upon? progress toward our goals? 7

Connecting Fidelity & Outcome Data Lucky Sustaining Positive outcomes, low understanding of Positive outcomes, high understanding of how they were achieved how they were achieved Outcomes Replication of success is unlikely Replication of success likely Losing Ground Learning Undesired outcomes, low understanding of Undesired outcomes, high understanding of how they were achieved how they were achieved Replication of failure likely Replication of mistakes unlikely Fidelity 8

Fidelity Data at Tier 2 Examples  Tiered Fidelity Inventory (TFI) – Fidelity ty to to S SWP WPBIS F S Framework ork Tier 2 section at t Tie ier 2 r 2  Benchmark of Advanced Tiers (BAT)  CICO Self-Assessment Fidelit lity to s o specif cific ic T Tier 2  Group Assessments inter erven entions  Counseling Group Lesson Plan Quick “ ck “Mi Micr cro” C o” Check cks  Observations, interviews, surveys

How often should we collect & analyze fidelity? It depends…. o Initial implementation o Restarting or indications of low fidelity o Recent student and staff turn-over The l long-term g goal i is to to move ve to to annual fidel delity c chec hecks f s for bi big pi g picture a e and nd week eekly/monthly o or qua quarterly c chec hecks i s in- bet etween. en.

Fidelity Assessment of the Tier 2 System

How do you currently assess the fidelity of your Tier 2 system?  Tiered Fidelity Inventory (TFI)  Benchmarks of Advanced Tiers (BAT)  We use a different assessment  We aren’t currently assessing Tier 2  I don’t know Poll www.menti.com

SWPBIS Tiered Fidelity Inventory (TFI) TFI is available FREE at pbisapps.org (Applications > PBIS Assessment)

Why use the TFI?  The TFI was developed to provide one assessment across all three tiers that could be used for progress monitoring and annual evaluation.  Three separate studies show o Reliability: Consistent results among raters o Validity: Significant correlation with other measures (BoQ, TIC, SAS, and BAT) o Usability: Teams reported the TFI as efficient & useful for decision making

Preparing to Complete the TFI Time & Place o Schedule a time and place to complete the TFI o Allow approximately 30 minutes per tier Team Members & Coach o Be sure the correct team members are present and the coach is available Complete Interventions Reference Map o Appendix B o Complete before the meeting

Tier II: Targeted Interventions Reference Map (Appendix B) Targeted Interventions Reference Map Description Complete the Reference Map as a team • List the targeted interventions across the top row • Check off each of the functions provided by the intervention in the columns below • Allow 15-30 minutes to complete • Use the information to complete items 2.5 and 2.7 of the TFI

Targeted Interventions Reference Guide

Tier 2 Section Tier 2: TFI Feature: 13 Items Possible Data Sources Scoring Criteria: 0-2

Possible Data Sources for the TFI

Pos ossible le Data Sources for Tier 2 ▲ Tier 2 team meeting minutes (last two) ▲ Rubric for selecting students for Tier 2 support ▲ Tier 2 strategy handbooks, or procedures (i.e., CICO) ▲ Available Tier 2 data summaries (if possible for 2 months) ▲ Family communication systems ▲ Most recent fidelity measures for Tier 2 strategies ▲ Intervention Tracking Tool*

Intervention Tracking Tool Examples

TFI Total Score Report Total Score Report : Overall implementation percentage for Tiers 1-3 This is most helpful when the team is implementing all three tiers.

TFI Reports – Overall by Tiers 1. Is Tier 1 SWPBIS being implemented with high fidelity (at least 70%)? 2. If yes, is Tier 2 SWPBIS being implemented with high fidelity (at least 70%)? 3. If yes to both, is Tier 3 SWPBIS being implemented with high fidelity (at least 70%)?

TFI: S Subscale Report rt

TFI Tier 2 Team Subscales Team Composition Team Operating Procedures Screening Request for Assistance

TFI Reports: Interventions Subscale Options for Tier 2 Interventions Tier 2 Critical Features Practices Matched to Student Need Access to Tier I Supports Professional Development

TFI Reports: Evaluation Subscales: Level of Use Student Performance Data Fidelity Data Annual Evaluation

TFI Reports – Individual Items Use for team action planning

Action Planning Meeting  Provide I e Item ems s Repo eport f for all pa participants t s to review  On paper or electronically  Cel eleb ebrate a e all t the he 2’ 2’s! s!  Develop way to share results with staff & stakeholders  Choos oose 1 1-3 3 Items f for Action P n Plann nning ng  < 2  Priority for the team  Easier items first can be helpful

TFI Evaluation Brief (Kittleman, et.al. 2017) What Tier 2 subscales and items on the TFI have the highest and lowest scores ?

Tier 2 Items Tier 2: Highest: Team Composition Tier 2: Lowest: Fidelity Data

TFI Fidelity 2:12

Item Analysis for Action Planning Our TFI Tier 2 Total Score was below 70% Step 1  Which subscale( e(s)?  Which items ms?  Which i item tem(s) d ) do y you w want to t to f focus on? “Our Tier 2 r 2 Scale score is below 70%. The subscale of Eval aluation ion is the area we need to focus on, specifically with Item tem 2 2:12 o on f fidel elity data f for o our T Tier 2 2 inter terven entions. W We aren’t d t doing t g the f e fidel elity chec ecks for o our T Tier 2 2 inter terven entions consisten entl tly. The staff are reporting that t Step 2 the fidel elity c chec ecks t take e to too m much time to e to c complete. e.”

Action Planning Next Steps Items Actions Who When Evaluation Item 2:12 Group to revise fidelity check; Susan, Friday put in teacher’s boxes each Michael & (fidelity data for our Friday Rachelle Tier 2 interventions Step 3 scored 1.0). We aren’t Put the current checks in Paloma Quarterly doing the weekly Tier 2 teacher’s boxes quarterly fidelity checks consistently. The staff are reporting that the fidelity checks take too much time to complete.

TFI Activity Review the TFI Items & Information Be ready to share Choose up to 3 out your action items (0 or 1) for plan items action planning Look at the Develop an Action information sheet Plan with specific for additional 10 minutes information resources

Fidelity Assessments for Tier 2 Interventions

How do you mainly assess fidelity of Tier 2 interventions?  Assessment Tool  Observation  Survey  Feedback  We don’t have Tier 2 interventions  We don’t assess fidelity  Other

Which ch T Tier 2 2 Interventio ions d doe oes y your sch choo ool u l use? e? List the Tier 2 interventions at your school

Fidelity Check Examples

CICO Intervention Fidelity Available on www.pbisapps.org

BEP/CICO Fidelity Measure  Formal evaluation of the critical features of CICO (formerly known as the BEP)  There are 13 items scored from 0-2 based on review of permanent products, interviews, and observations Free Online Download Search term: BEP-FIM