Riding Program Impact A Multicenter Approach and Feasibility Study - PowerPoint PPT Presentation

Measuring Therapeutic Riding Program Impact A Multicenter Approach and Feasibility Study 1 Ca Cathy y Smith th Hy Hybels, s, Ph.D. D. PATH H Intl ntl Re Region One Co Confere erence nce April 2015

Topics:  TR Program Impact  Measuring Impact  Collaboration  Feasibility Study Results  Expert panel 2

TR Program Impact We want to in TR 3 Hybels 2015

From success defined as “how much we do”… TR Program Impact # Participants # New Participants # Volunteers # Volunteer Hours # Lessons $ value of volunteer time Types of TR programs 4 Hybels 2015

…to also include “how well we perform” and to answer “is anyone better off?” TR Program Impact Impact on the Community How well we perform Results-Based How much Accountability Is Anyone we do Evidence-Based Better Practices Off? Research on TR Effectiveness 5 Hybels 2015

We want to advance accountability in TR at each of these levels TR Program Impact Participant TR program Instructor and the TR Industry TR Center 6 Hybels 2015

Changing direction requires Measuring Impact 7 Hybels 2015

Tools for measuring impact Measuring Impact 8 Hybels 2015

Session Report Measuring Impact Product of a Collaborative Effort 9 Hybels 2015

Session Report Measuring Impact Identifies TR Skill Areas Hor Horseman semanship ship Skill Skills Suppo Sup porting ting Skill Skills Mount Expressive Communication Dismount Attending to Task Back up Balance At halt--posture Coordination At halt--reins Posture At halt--halt-walk transitions Follows direction At walk--posture Social interaction At walk—hold reins At walk—steer with reins Safety Awareness At walk--walk-halt transitions Regulates Behavior At walk--half seat At trot/jog--posture At trot/jog--seated trot At trot/jog--half seat At trot/jog--posting At canter/lope Jump Tack Untack 10 Groom Lead Hybels Horse Safety 2015

Session Report Measuring Impact Defines TR Skill Areas Horsemanship Skills Uses Natural Aids: Uses hands, legs, seat, voice, ears, and eyes to effectively communicate with the horse • Example: Uses voice to “whoa” or “walk on.” Horse Safety: Recognizes potentially unsafe or hazardous circumstances both mounted and unmounted and reacts accordingly. • Example: When riding in a group, recognizes when approaching too close to another horse and directs horse to an open space in the arena. 11 Hybels 2015

Session Report Measuring Impact Tracks P Progress in Skill Areas Horsemanship Skill Area Section Level of Support Required Leader Involvement 1. Maximum physical assistance — hands on 80-100% of the time 1. Leader is totally in charge of the horse — rider does not have reins 2. Moderate physical assistance — hands on 50% of the time 2. Leader is totally in charge of the horse — rider is beginning to use reins 3. Minimal physical assistance — hands on less than 25% of the 3. Leader or rider may control horse time 4. Instructor supervision with verbal/visual cues 80-100% of the 4. Rider is in control of the horse — leader remains on-lead time 5. Rider is in control of the horse — leader is off- lead at horse’s head 5. Instructor supervision with verbal/visual cues 50% of the time 6. Rider is in control of the horse at all times — no leader needed 6. Instructor supervision with verbal/visual cues less than 25% of the time 7. Instructor supervision only — rider is independent 8. Instructor observation only — independent & proficient Mounted Skills Baseline: Jan/Feb Mar/Apr May/June July/Aug Sept/Oct Nov/Dec Uses Natural Aids Halt Back Up Walk – overall Walk – half seat Trot/Jog – Sit Trot/Jog – half seat 12 Trot/Jog – post Canter/Lope Jump Hybels Horse Safety 2015

Session Report Measuring Impact Tracks P Session Goals Session Goals & Progress Notes Section Session Goals ( March/April ): 1. 2. 3. Progress notes on goals for this session (Indicate progress and to what degree the goal was met) Recommended Future Goals: 1. 2. 3. 13 Please note anything that could be helpful to know about this rider: Hybels 2015

Session Report Measuring Impact As a Tool for Measuring Impact  Defines TR skill areas  TR instructor use only  Tracks P skill progress over a year  Instructor rates the progress in several TR skill areas of their students in broad skill areas — vulnerable to bias  Tracks P progress on individual  Does not allow aggregation of goals data to determine program  Designed by instructors for effectiveness instructors  Limited value outside of TR  Field tested because there is no research 14  Facilitates communication to establish its credibility as between TR instructors Hybels an assessment tool 2015

Measuring Impact Goal Attainment Scaling (GAS) is a widely implemented and systematic goal setting & evaluation process that measures the extent to which a participant’s goal(s) is achieved Assess P Baseline Define Assess goal Performance Identify Goal Expected attainment in a TR Skill Outcomes Area 15 Independent Rater TR Instructor TR Instructor & others TR Instructor Hybels 2015

GAS Process Measuring Impact • At the beginning of each TR session for each participant Set Goals • Providing opportunities to practice skills Deliver Instruction • Independent rater attends lesson at the end of the TR session to observe and assess performance Measure in goal areas Performance 16 Hybels 2015

GAS Encourages: Measuring Impact Collaboration in Goal Setting Participant 17 Hybels 2015

GAS Encourages: Measuring Impact Support Team Collaboration 18 Hybels 2015

GAS Requires: Measuring Impact SMART Goals 19 Hybels 2015

Goal Scale Development Measuring Impact Start with Defining Baseline Performance Anticipated goal attainment at Performance Level Descriptions end of session Baseline/Much Participant performs a sitting trot down the long side Less Than Expected Less than expected Expected Better than expected Much better than expected 20 Baseline performance as observed by the TR Hybels instructor at the beginning of the session 2015

Measuring Impact Goal Scale Development Define the Expected Performance Goal Performance Level Descriptions Attainment Baseline/MLTE Participant performs a sitting trot down the long side Less than expected Expected Participant performs a posting trot down the long side Better than expected Much better than expected 21 Expected performance at the end of the TR session based upon the TR instructor’s understanding of the p, and Hybels drawing upon the instructor’s training and past experience 2015

Goal Scale Development Measuring Impact Define the other possible levels Goal Performance Level Descriptions Attainment Baseline/ Participant performs sitting trot down the long side MLTE Less than Participant performs posting trot for less than half the long side expected Expected Participant performs posting trot down the long side Better than Participant performs posting trot for half the entire ring expected Much Participant performs posting trot for the entire ring better than expected 22 Other possible outcomes then are established: less than expected, better than expected level, and much better Hybels than expected 2015

Goal Scales In Use This is what the Rater Sees Please check the box next to the description below that best describes the participant’s performance that you observed today: Participant performs sitting trot down the long side Participant performs posting trot for less than half the long side Participant performs posting trot down the long side Cathy Hybels 2015 Participant performs posting trot for half the entire ring Participant performs posting trot for the entire ring Please describe any circumstances present that you feel may have affected the participant’s performance : 23 Hybels 2015

Goal Attainment Scales Measuring Impact How Scores Are Assigned Goal Score Performance Level Descriptions Attainment -2 BL/Much Participant demonstrates sitting trot down the long less than side expected -1 Less than Participant demonstrates posting trot for less than half expected the long side 0 Expected Participant demonstrates posting trot down the long side 1 Better than Participant demonstrates posting trot for half the expected entire ring 2 Much better Participant demonstrates posting trot for the entire than ring expected 24 Hybels The independent rater selects the level that best How to interpret the rating 2015 matches the performance they observed

TR Goal Attainment Scale Example #2 Measuring Impact Goal Rating Attainment Level Description BL/MLTE -2 Participant does not hold reins Less than Participant picks up the reins after a verbal cue, then -1 expected immediately puts them down Expected Participant picks up the reins after a verbal cue and 0 holds them for 1 minute, then puts them down Better than Participant picks up the reins after a verbal cue and 1 expected holds them for 2 minutes, then puts them down Much better 25 than Participant picks up the reins after a verbal cue and 2 expected holds them for 3 minutes, then puts them down Hybels 2015