A Rational Approach to Outcome Measurement Duncan Raistrick Leeds - PowerPoint PPT Presentation

A Rational Approach to Outcome Measurement Duncan Raistrick Leeds Addiction Unit Leeds and York Collaborations for Leadership in Applied Health Research and Care (CLAHRC) Funded by the NIHR Payment by Results Conference Friday 28 th September 2012

NIHR Nine CLAHRCs approx £10million each CLAHRC for Leeds/York/Bradford Addiction Research in Acute Settings (ARiAS) 1 of 5 themes ARiAS outcome measurement 1 of 6 strands CLAHRC research group : Duncan Raistrick, Gillian Tober, Christine Godfrey, Charlie Lloyd, Steve Parrot, Jude Watson, Veronica Dale Co-opted Expert Group convened March 2011 : Owen Bowden-Jones, Alex Copello, Ed Day, Eilish Gilvarry, Don Lavoie, Damian Mitchell, Julia Sinclair, John Strang, and Alex Whincup

Summary o Views and expectations of different stakeholders  Service user and carers  Politicians and commissioners  Practitioners o Substance use and scales  Self report  Biological measures  Scales EQ5D LDQ CORE SSQ  Societal Impact Measures o Further interesting things to find out  Relationship between measures  Clinically significant change

Outcome measures answer the question “ how do I know this person is getting better?” and they also... summarise complex information in a clinically meaningful and ‘real world’ way communicate complex information in a clear and simple way using minimal data can be integrated into routine practice in a clinically useful way but.............

......different stakeholders have different ideas of ‘getting better’ ...... the same stakeholders have different needs o Service users and carers – abstinence o Politicians and commissioners – costs and benefits o Practitioners  Health workers – physical and mental health  Criminal justice workers – offending behaviour  Social workers – safeguarding children ......so, there is no ideal measure, just the mix of measures that best suits the purpose.

Service user and carer views on what outcomes are important Agency Type Type of Activity SU F&F FG1 NHS Treatment 7 4 FG2 NHS/3rd sector DRR 3 2 FG3 NHS/3rd sector Harm reduction 0 0 FG4 SMART group Recovery mutual aid 7 n/a FG5 3rd sector Recovery SU only 7 n/a FG6 3rd sector Recovery F&F only n/a 6 TOTAL 24 12 Six focus groups (FGs) were held to elicit service user (SU) views and views of Family and Friends (F&F) on what constitutes a good outcome. Source: unpublished CLAHRC study 2012

Analysis of Focus Groups FG1 recovery is abstinence, social life, support for family and friends, relationships, lifestyle with partner, being in control, life style changes during recovery process, physical and mental health, confidence FG2 positive environment to maintain abstinence and move away from drugs and alcohol, no good outcome – it’s with you for life, not using methadone or buprenorphine, social support, doing something to fill time, safe accommodation, children back from care, confidence, physical health and appearance FG4 physical and mental health, ability to distract and not act upon cravings, self-worth, energy, not feeling isolated, personal responsibility, living a normal life, engaging in treatment, improved relationships FG5 self esteem, eating healthily, abstinence, sleep, physical health, relationships, social networks, feeling valued, new interests and skills, courses. FG6 employment, safe and supportive social networks, something to do during the day, self-worth, physical and psychological well-being, having a good addiction, not feeling ashamed, financial situation There was a distinct view that abstinence or perhaps moderation is the first step but the social support to maintain change is all important. Other gains such as health and relationships were seen as good but not the essence of a good outcome . Source: unpublished CLAHRC study 2012

Substance Use as the Primary Outcome

Justification for substance use as the primary outcome measure.......  Viewed by service users as most important  Viewed by careers as most important  Most convincing for general public and policy makers  Substance misuse is the condition  Correlation with societal costs

Different Ways to Measure Substance Use Objective Frequency Actual Quantity / Episodes Testing Categories Frequency Frequency of Use blood tests ASSIST & ASI MAP & TOP OTI toxicology AUDIT Composite measures are restrictive: • always a compromise – too much/too little detail eg ASI • typically rater completed eg TOP, ASI, MAP • item selection bias eg TOP • predetermined categories not suited to particular needs eg AUDIT • not easy to update/modify and revalidate eg TOP, ASI

Birmingham Heavy Drinkers 10yr Follow-up Uses UNITS of alcohol grouped in ranges n=259 followed at 10yrs: >50% reported major life events in previous 2yrs: typically health, employment, shift of attitude. 18 known deaths: 3 CVS, 4 liver, 3 cancer, 1 suicide, 2 diabetes. Abstinence increased from 0-10%, weekly units decreased from 90-59 (m) and 60-36 (f) 60 % of sample in drinking volume group 50 40 30 20 10 0 Interview year Abstinent Sensible (<14/21 units) Hazardous (14-35 / 21-50) Heavy (35-70 / 50-100) Very heavy (70+ / 100+) Source: Rolfe et al.(2009) Report to DH

UK Alcohol Treatment Trial FREQUENCY and UNITS of alcohol 30 100 Percent of Days Abstinent Drinks per Drinking Day 25 80 20 60 15 40 10 20 5 0 0 Baseline 12 Months Baseline 12 months MET SBNT MET SBNT Source: UKATT Research Team, BMJ (2005)

Problems of the substance itself as an outcome measure…… o Is it how much (quantity) or how often (frequency) that matters? How reliably can these be measured? o Is it the range of substances used or just the presenting main substance that should be the outcome? Does this include prescribed drugs (eg methadone, diazepam)? o Are some drugs associated with more harmful routes of administration than others (eg heroin)? o Does substance use itself matter or is it the related harms? Are some harms clearly attributable to particular substances (eg alcohol) some much less so (eg heroin)? o How to measure it?

Toxicology Screening as an Indirect Measure Days (mean ± sd) before a positive test would be expected given different drug use frequency and different testing frequency 72-hour detection window (drugs that can be detected up to 72hrs) Frequency of testing Frequency of drug use 2x /wk 1x /wk 2x /mth 1x /mth 8x /yr 3 ± 2 7 ± 2 15 ± 10 30 ± 13 46 ± 40 Every day 4 ± 3 8 ± 3 18 ± 12 35 ± 17 51 ± 50 Every other day 5 ± 4 11 ± 7 23 ± 18 48 ± 31 71 ± 66 1x /wk 1x /wk 9 ± 9 18 ± 14 40 ± 33 80 ± 64 118 ± 106 2x /mth 19 ± 21 39 ± 35 91 ± 88 160 ± 124 272 ± 260 1x /mth 36 ± 42 71 ± 66 150 ± 141 306 ± 283 560 ± 598 Source: Crosby et al ., (2003) J Addictive Diseases

Correlations with Societal Costs example data for units of alcohol and health status health as % of best possible health 100 moderate drinking not 90 associated with major health problems 80 70 60 one2one 50 increase 40 decrease 30 health may not recover once damage is done 20 10 0 0 10 20 30 40 50 60 weekly units of alcohol

Conclusion There is a scientific and political case for the primacy of substance use as an outcome measure.... Correlations with Substance Use.... Baseline Follow-up EQ5D LDQ CORE SSQ EQ5D LDQ CORE SSQ Alcohol -0.32** 0.48** 0.28** -0.08* -0.30** 0.62** 0.39** -0.20** freq Alcohol -0.38** 0.47** 0.25** -0.19** -0.30** 0.62** 0.38** -0.20** units Heroin -0.11 0.39** 0.09 -0.10 -0.21 0.40** 0.40** -0.60** freq **p<.001 *p<.01 Source: unpublished CLAHRC clinical sample but....  substance use is difficult to measure – LDQ good proxy  other scales are needed to paint a picture of outcome

Choosing Scales as Secondary Outcome Measures

Types of Outcome Scales Generic Measures (Health) Is treatment cost effective? How ill are people with addiction problems compared to other users of health care? How complex are the health problems? What is the illness profile of people with addictions? Dimension Measures (Addiction) How severe is the addiction? How difficult is treatment likely to be? How good is one addiction service compared to another? Do problems persist? Condition Specific Measures (Depression, Pregnancy) How severe is the specific condition? How do services targeting the condition compare? How effective is treatment for this specific problem? Personal Goal Measures (Me) The personal outcome goals agreed with each service user . Source: Fitzpatrick et al. (1998) Health Technology Assessment

Societal Impact Measures - SIMs  Treating the individual has benefits at the societal level – SIMs are service level  Political interest is in societal costs (a research exercise) – SIMs are a headline contribution  SIMs need to be: objective, easy to collect, capable of showing variability, reflect the impact of the service in the fewest possible measures............... Pregnancy and Parenting i) birth weight ii) child with mother at 12months Hospital In-reach i) A&E attendances ii) in-patient admissions in last 12months Detoxification i) % completing ii) % supervised disulfiram