5C Three Assessment Tools to Strengthen PBIS Implementation Ken - PowerPoint PPT Presentation

CALIFORNIA PBIS COALITION 5C Three Assessment Tools to Strengthen PBIS Implementation Ken Fitzgerald Stanislaus County Office of Education tinyurl.com/PBISassess

CALIFORNIA PBIS COALITION the Conference Tweet Out About… Research and Resources #PBISCA @ PBIS_CA Recognition Don’t forget the Hashtags and Handles! Impactful Quotes and Inspiration

Presentation Goals  Review the evidence-based outcomes of PBIS and implementation fidelity  Introduce three effective assessment tools for implementation and continuous improvement

Acknowledgements  National PBIS Technical Assistance Center Co-Directors: Dr. Rob Horner, University of Oregon Dr. George Sugai, University of Connecticut  Educational and Community Supports, College of Education at University of Oregon University of Oregon

PBIS Framework Framework for enhancing the adoption and implementation of A continuum of evidence- based interventions to achieve Important academic and behavioral outcomes For ALL students

PBIS Framework Intensive Intervention Few Students • Individualized Universal Prevention • Function-based All Students 1-5% • High intensity • Core Instruction 7-15% • Preventive Targeted Intervention • Proactive Some Students • Common Rules • Supplemental (to and Expectations reduce risk) • Common • High Efficiency Referral System • Rapid Response • Strength Based 80% Behavior System All students in school

Experimental Research on SWPBIS Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10 (2), 100-115 Bradshaw , C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23 (4), 462-473. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148. Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1-26. Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145. Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42 (8), 1-14. Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics . Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial . Archive of Pediatric Adolescent Medicine. 2012;166(2):149-156 PBIS.org/research

Implementation Fidelity  PBIS Implementation Blueprint Tinyurl.com/PBISblueprint pbis.org/blueprintguidestools/blueprint/implementation-blueprint  Part 1: Foundations and Supporting Information  Part 2: Self-Assessment and Action Planning (district focus)

Implementation Strategies Experimental data show these methods, when used alone, are insufficient:  Diffusion/dissemination of information  Training  Passing laws/mandates/regulations  Providing funding/incentives  Organization change/reorganization Data: 5% to 15% Realize Intended Outcomes Each of these are necessary but not enough if done alone

You You shou hould be e mor more e ex expl plicit her here in Step tep 2! 2!

Training Outcomes Related to Training Components Training Outcomes Knowledge of Skill Classroom Training Components Content Implementation Application Presentation/ Lecture 10% 5% 0% Plus Demonstration 30% 20% 0% Plus Practice 60% 60% 5% Plus Coaching/ Admin 95% 95% 95% Support, Data Feedback Joyce & Showers, 2002

Connecting Outcomes & Fidelity Lucky Sustaining Positive outcomes, low understanding of Positive outcomes, high understanding how they were achieved of how they were achieved Outcomes Replication of success is unlikely Replication of success likely Losing Ground Learning Undesired outcomes, high Undesired outcomes, low understanding understanding of how they were of how they were achieved achieved Replication of failure likely Replication of mistakes unlikely Fidelity

PBIS Assessments – PBISapps.org

Sample Assessment Timelines SWIS Who/When TFI SET SAS Data PBIS Team ✔ Initially, then quarterly, then annually (at 70%) External Evaluator ✔ After PBIS Tier I Rollout All Staff Conducted annually to ✔ assess features, prioritize improvement Data Analyst, PBIS Team ✔ At least monthly

Tiered Fidelity Inventory (TFI)  An assessment tool developed by the University of Oregon, PBIS Technical Assistance Center  Provides teams with a single, efficient, valid, reliable survey to guide implementation and sustained use of school wide PBIS  Primary purpose is to help school teams improve  Primary audience is the implementation team and other stakeholders  Effective use requires multiple assessments to show growth over time tinyurl.com/PBISTFI

Tiered Fidelity Inventory (TFI)  Addresses all three tiers of implementation  Reports include: Total Score, Subscale, Sub-Subscale and Individual Items Sample Sub-Subscale Report 70% is minimum goal www.pbisapps.org

TFI Tool Overview

Tips for Conducting a TFI  Review the Feature description and scoring criteria as a group  Does your team have any of the identified possible data sources available?  Goal is to find consensus but not let one person’s voice overpower others  Vote simultaneously using a hand signal then discuss more if needed  Invite a district coach or PBIS team leader from another school to help provide an objective view

TFI Action Planning

TFI Action Planning  Action planning tool is online at PBISapps.org Reflection: What would be an appropriate action item for 1.4? Who would be responsible, when will it be completed? 1 2 2 Complete teaching plan schedule for year using 1 Mike 01/01/17 teaching example from training website 2

SET: Schoolwide Evaluation Tool  An interview and walk-through tool designed to assess and evaluate the critical features of PBIS  Developed as a research tool, effective for program improvement, allowing you to:  Assess features in place  Determine annual goals  Evaluate on-going efforts  Design and revise procedures  Compare year to year efforts tinyurl.com/SETPBIS

SET: Schoolwide Evaluation Tool  Work with an outside evaluator, either a district coach, regional trainer or partner site  Interview portion  Administrator interview with 21 questions  Random selection interviews  10-15 staff, asking 7-10 questions  15 students, asking 2 questions  School tour and PBIS materials review  Including specific observations in 10 locations

SET Reports  Subscale

SET Reports  Individual Items

Self-Assessment Survey (SAS)  Survey for all school staff to assess the implementation of behavior support systems  Four main areas:  School-wide discipline systems  Non-classroom management systems  Classroom management systems  Systems for individual students tinyurl.com/SASPBIS

Self-Assessment Survey (SAS)  Completed independently by all staff initially  Schedule 20-30 minutes for completion  Email survey links or pen-to-paper in a meeting  Follow-up surveys can be conducted with representative groups  Summarize results then develop action plans for improvement

SAS Summary Sample

SAS Survey Tool Excerpt

Next Step: Get a FREE PBIS Assessment account  Visit PBISapps.org  Select PBIS Assessments in the Applications drop-down menu  At the bottom of the overview section on the PBIS Assessments page, click on the Get PBIS Assessment button  Download and complete the account request form, then email to support@pbisapps.org  Video tutorials are available under the Resources menu

Where to start?  Always look for the smallest change that will have the largest impact  Never stop doing what works

Thank you! Questions? Ken Fitzgerald Director II, Safe and Supportive Schools Stanislaus County Office of Education, Prevention Programs (209) 238-1381  kfitzgerald@stancoe.org