9/23/2016 Learning to Administer and Interpret Assessments to Support your Continuing Professional Development (CPD ) Shelley Hughes Occupational Therapist 22 Sept 2016 CPD and Standardised Assessments 1 Shelley Hughes Occupational Therapist Senior Product Manager Pearson Clinical Assessment Twitter: #OTLearn CPD and Standardised Assessments | 2 1
9/23/2016 By the end of this session, attendees will be able to… … explain the CPD requirements for Occupational Therapists in the UK … describe the steps required to begin learning to use a standardised assessment … share ideas for working with colleagues to learn to use standardised assessments as a CPD activity … explain some of the concepts of standardised assessments to evidence this as a CPD activity CPD and Standardised Assessments 3 Agenda CPD requirements UK Occupational I Therapists The assessment process II The essentials of getting started III Learning with colleagues IV Graduating from basic to advanced V administration and interpretation 4 CPD and Standardised Assessments 2
9/23/2016 CPD Requirements CPD can be defined as: “ a range of learning activities through which health and care professionals maintain and develop throughout their career to ensure that they retain their capacity to practice safely, effectively and legally within their evolving scope of practice” Health and Care Professions Council (HCPC) 6 CPD and Standardised Assessments 3
9/23/2016 CPD Requirements in the UK 1. Standards of proficiency ‒ Meeting the standards ‒ Scope of practice 2. CPD and Audit Process ‒ Need to document CPD ‒ Demonstrate a variety of learning activities ‒ Seek to ensure that CPD contributes to quality of practice ‒ Must benefit the service user[s] ‒ May be audited 3. Examples of CPD activities ‒ Work based learning ‒ Professional activities ‒ Formal education ‒ Self-directed learning ‒ Other CPD and Standardised Assessments 7 COT Guidelines 1.College of Occupational Therapists Code of Continuing Professional Development – 2014 2. Aligned with and complimentary to the HCPC guidelines (does not replace these) 3.Underpinning concept = critical reflection 8 CPD and Standardised Assessments 4
9/23/2016 The Assessment Process The Assessment Process 1.Hypothesis testing ‒ A problem solving process ‒ Identifying strengths 2.Types of Assessment ‒ Screeners ‒ Non-standardised ‒ Criterion-referenced ‒ Standardised ‒ Full battery ‒ More specific ‒ Predictors for problems in everyday life 10 CPD and Standardised Assessments 5
9/23/2016 The Assessment Process develop hypothesis background information full battery screening specific assessment assessment assessment revise prove / disprove hypothesis hypothesis Intervention planning / outcome measure CPD and Standardised Assessments Standardised Assessments: Key Features 1. Allows you to compare your patients score to a criterion ‒ the patient’s score is compared to the norms - the scores of the reference group 2. Is always administered and scored in exactly the same way 3. Comparisons made using standardised scores ‒ Raw Scores (non-standardised) ‒ Percentile Ranks ‒ Standard (or Scaled) Scores 4. The aim is to decide whether: ‒ patient’s performance is evidently different to most or all of the typical sample ‒ the patients performance is significantly different to what you would expect in view of age, ability etc 12 CPD and Standardised Assessments 6
9/23/2016 The Essentials of Getting Started Where to Start… 1. Understand the population for which the assessment is intended 2.What service needs will the assessment meet 3.How was the assessment developed 4.How are assessment findings reported 5. Practice administration 14 CPD and Standardised Assessments 7
9/23/2016 How was the Assessment Developed… 1. Theoretical Underpinnings 2. Data Collection CPD and Standardised Assessments 15 Evidence of Consistency & Usefulness…Reliability Is the test a reliable and consistent measure? Inter-rater reliability ‒ are the scores likely to be the same no matter who is administering the test? Test-retest reliability ‒ are the scores likely to be the same if the test is given on two occasions separated by a short interval? Parallel form reliability ‒ where the test has parallel forms, are these equivalent in difficulty? 16 CPD and Standardised Assessments 8
9/23/2016 Evidence of Consistency & Usefulness…Validity Does the test measure what it claims to measure? Construct validity ‒ Is the test measuring the theoretical concept it says it is measuring? Content validity ‒ Does the test assess a reasonable range of behaviour related to the concept being measured? Criterion validity ‒ Are the test results consistent with results from other measures of the concept being measured? Face validity ‒ Does it appear as if it is measuring what it is supposed to be measuring? Ecological validity ‒ Do test results predict behaviour in everyday situations? CPD and Standardised Assessments 17 How we Measure Reliability and Validity: Correlation Coefficients • Measure of the degree of relationship between 2 variables e.g. age and scores on a test • Range +/- 1 to 0 • +1 and –1 are perfect correlations in different directions • 0 = no relationship between the variables • Correlation of .6 = acceptable, .8 or higher = very good 18 CPD and Standardised Assessments 9
9/23/2016 How are assessment findings reported ‒ The patient’s score is compared to the norms - the scores of the reference group The aim is to decide whether: ‒ the patient’s performance is evidently impaired i.e. worse than most or all of the typical sample ‒ the patients performance is significantly different to what you would expect in view of age, ability etc CPD and Standardised Assessments 19 Normal Distribution: ‒ Distribution of test scores for large groups of heterogeneous test takers ‒ 68.26% of population should fall within 1 Standard Deviation (S.D.) of the mean 2 S.D.’s = 95.44% ‒ 3 S.D.’s = 99.73% ‒ ‒ Few extremes at both ends ‒ Scores must be normally distributed for these to be useful 20 CPD and Standardised Assessments 10
9/23/2016 Normal Distribution: CPD and Standardised Assessments 21 Interpretation of Scores ‒ A score on a test does not prove or disprove anything ‒ It just gives an estimate of performance in a particular area of behaviour ‒ Must be interpreted in the wider context of results from a range of tests ‒ Take into account previous abilities and achievements 22 CPD and Standardised Assessments 11
9/23/2016 Practice Administration CPD and Standardised Assessments 23 The Learning Process Questions to ask yourself 1.Understand the basics 2.How would the assessment support your decision making and practice 3.What is the evidence base for use of the assessment 24 CPD and Standardised Assessments 12
9/23/2016 The Learning Process Documenting 1. Reflection ‒ How important is this activity to your professional practice? ‒ How will this influence your practice? ‒ Your next steps for learning ‒ Review your learning after several months 2. Sharing with others ‒ Teaching students ‒ Share a case study with colleagues ‒ Share with other professions CPD and Standardised Assessments 25 Learning with Colleagues 13
9/23/2016 Teamwork 1. Practice administration 2. Inter-disciplinary 3. Application to different clinical populations 4. Establish a community of practice CPD and Standardised Assessments 27 Graduating from Basic to Advanced Administration and Interpretation 14
9/23/2016 The Next Steps 1. Plan for your next steps 2. Review the research 3. Become more advanced in administration and interpretation CPD and Standardised Assessments 29 Advanced Administration 1. Understanding the assessment in context 2. Interpreting the assessment in detail 30 CPD and Standardised Assessments 15
9/23/2016 Confidence Bands Reflect the uncertainty in scores arising from measurement error i.e. – you would not expect the same score on every single occasion . CPD and Standardised Assessments 31 Confidence Bands 1. Reported as 68% or 95% confidence band 2. 95% confidence band = you are 95% confident that the true score will lie within this band 3. Based on the reliability measurements of the test 4. The higher the confidence interval is the wider the band will be 5. Which band is better for your situation? 32 CPD and Standardised Assessments 16
9/23/2016 Confidence Bands Overlapping Confidence Bands 86 (Standard Score) 79 Verbal Memory 93 95 (Standard Score) 91 Visual Memory 99 CPD and Standardised Assessments 33 Composite vs Subtest Level Interpretation 1. Composite more robust 2. Some subtests are sufficiently robust to use in isolation 3. Comparing subtest level data helps build understanding about an individual’s profile 4. Statistical difference vs clinical importance 34 CPD and Standardised Assessments 17
9/23/2016 Pairwise comparisons BOT-2 Example CPD and Standardised Assessments 35 Pairwise comparisons 36 CPD and Standardised Assessments 18
Recommend
More recommend
