Understanding standardised assessments in everyday practice - PowerPoint PPT Presentation

Understanding standardised assessments in everyday practice Claire Parsons Assessment Solutions Consultant Pearson Clinical 17 th June 2019 Rookwood Driving Battery (RDB) 1

Claire Parsons Assessment Solutions Consultant Pearson Clinical Assessment

Aims of the workshop • Explain the principles of administering and interpreting standardised assessments • Gain an understanding of the considerations to be made when selecting appropriate standardised assessments • Understand and apply the principles of standardised assessment within the occupational therapy process • Understand the principles of test interpretation • Gain a clearer understanding of statistical concepts • Describe the learning of standardised assessments as a Continuing Profession Development (CPD) activity 3

Agenda • CPD requirements UK Occupational Therapists • The Assessment Process • Getting Started • Understanding scoring • Interpretation of scores • Questions

CPD Requirements

CPD can be defined as: “ a range of learning activities through which health and care professionals maintain and develop throughout their career to ensure that they retain their capacity to practice safely, effectively and legally within their evolving scope of practice” Health and Care Professions Council (HCPC)

CPD Requirements in the UK 1. Standards of proficiency ‒ Meeting the standards ‒ Scope of practice 2. CPD and Audit Process ‒ Maintain a record ‒ Demonstrate ‒ Contributes to quality of practice ‒ Must benefit the service user[s] ‒ Written profile 3. Examples of CPD activities ‒ Work based learning ‒ Professional activities ‒ Formal education ‒ Self-directed learning

Guidelines • Principles for continuing professional development and lifelong learning in health and social care – January 2019 • Aligned with and complimentary to the HCPC guidelines (does not replace these) • Underpinning concept = critical reflection

The Assessment Process

Selecting the right test • Depends on • Tests are designed for different purposes • The tests should

The Assessment Process 1. Hypothesis testing ‒ A problem solving process ‒ Identifying strengths 2. Types of Assessment ‒ Screeners ‒ Non-standardised ‒ Criterion-referenced ‒ Standardised ‒ Full battery ‒ More specific ‒ Predictors for problems in everyday life

The Assessment Process

Standardised Assessment: Key Features 1. Allows you to compare your patients score to a criterion 2. Is always administered and scored in exactly the same way 3. Comparisons made using standardised scores 4. The aim is to make a decision

The Essentials of Getting Started

Where to Start… 1. Understand the population for which the assessment is intended 2. What service needs will the assessment meet 3. How was the assessment developed 4. How are assessment findings reported 5. Practice administration

How was the Assessment Developed… 1. Theoretical Underpinnings 2. Data Collection

Evidence of Consistency & Usefulness… Reliability • Inter-rater reliability • Test-retest reliability • Parallel form reliability

Evidence of Consistency & Usefulness… Validity • Construct validity • Content validity • Criterion validity • Face validity • Ecological validity

How we Measure Reliability and Validity: Correlation Coefficients • Measure of the degree of relationship between 2 variables e.g. age and scores on a test • Range +/- 1 to 0 • +1 and – 1 are perfect correlations in different directions • 0 = no relationship between the variables • Correlation of .6 = acceptable, .8 or higher = very good

Understanding Scoring

How are assessment findings reported? The patient’s score is compared to the norms - the scores of the reference group The aim is to decide whether: • patient’s performance is evidently impaired • the patients performance is significantly different to what you would expect in view of age, ability etc

Normal Distribution • Distribution of test scores for large groups of heterogeneous test takers • Mean, median, mode should all have same value • 68.26% of population should fall within 1 Standard Deviation (S.D.) of the mean • Few extremes • Scores must be normally distributed for these to be useful

Normal Distribution:

Standard Scores • Mean of 100 • or • Mean of 10 Advantages of standard scores: • allow comparison • take account of standard deviation • see where differences are extreme, abnormal or very unlikely • clear cut definition of impairment • most commonly used form of scoring on tests

Age & Standard Scores - Example Two patients with identical raw scores on the Doors and People test Patient A - aged 29 years Patient B- aged 81 years Subtest Raw score Standard Standard Score A Score B Verbal recall 14 4 9 Visual Recall 22 3 10 Verbal Recog 12 4 8 Visual recog 14 5 10

Interpreting a Composite Standard Score Less than 70 More than 2 standard Significant Impairment deviations from the norm 70 - 85 More than 1 standard Moderate impairment deviation from the norm 85 - 115 Within 1 standard Average range deviation of the norm 115 - 130 More than 1 standard Above average range deviation from the norm Over 130 More than 2 standard Very high / superior deviations from the norm

Interpreting Subtest Level Scores Less than 4 More than 2 standard Significant Impairment deviations from the norm 4 - 7 More than 1 standard Moderate impairment deviation from the norm 7 - 13 Within 1 standard Average range deviation of the norm 13 - 16 More than 1 standard Above average range deviation from the norm More than 16 More than 2 standard Very high / superior deviations from the norm

Percentiles • Reflect a kind of rank ordering • indicate the proportion of people obtaining a score EQUAL TO OR LESS THAN a given raw score 1 st percentile - low level • 99 th percentile - high score •

Interpreting Percentiles Less than 2 More than 2 standard Significant Impairment deviations from the norm 2 - 16 More than 1 standard Moderate impairment deviation from the norm 16 - 84 Within 1 standard Average range deviation of the norm 84 - 98 More than 1 standard Above average range deviation from the norm More than 98 More than 2 standard Very high / superior deviations from the norm

Interpretation of Scores

Defining Impairment • With raw scores • With standard scores • With percentile ranks

What do scores tell us? • What does a score in the impaired range tell us? • What does a score in the average range tell us?

Interpretation of Scores • A score on a test does not prove or disprove anything • It just gives an estimate of performance in a particular area of behaviour • Must be interpreted in the wider context of results from a range of tests • Take into account previous abilities and achievements

Confidence Intervals • Reflect the uncertainty in scores arising from measurement error i.e. – you would not expect the same score on every single occasion. • 95% confidence band = you are 95% confident that the true score will lie within this band. • Based on the reliability measurements of the test • The higher the confidence interval is the wider the band will be

Composite vs Subtest Level Interpretation • Composite more robust • Some subtests are sufficiently robust to use in isolation • Comparing subtest level data helps build understanding about an individual’s profile • Statistical difference vs clinical importance

The profile 38

Communication of Results • Exercise caution • Support suggestions • Rule out • Adapt your style of reporting • For other professionals who are familiar with cognitive assessment • Take appropriate steps regarding confidentiality.

Questions?

There’s so much more to learn Find out more about us at Pearsonclinical.co.uk