Analysis of clinical reports published in the context of Policy 0070 - PowerPoint PPT Presentation

Analysis of clinical reports published in the context of Policy 0070 Technical Anonymisation Group (TAG) meeting, London, 29-30 November 2017 Agenda point 06 Presented by Ada Adriano Documents Access and Publication Service An agency of the European Union

Acknowledgments Kanako Sasaki (Visiting Expert from Japanese MHLW) and EMA Clinical Data Publication team 1 Analysis of clinical reports published in the context of Policy 0070

Analysis of published anonym isation reports • 54 anonymisation reports published (cut-off date: 06 October 2017) Product category Product category Product category (with patient identifiers) (without patient identifiers) (11) 20% (5) (2) 21% (9) 12% 18% (1) 2% without generics generics patient identifiers biosimilars hybrids full applications with patient orphans identifiers non- * (7) generics (2) 64% (43) 18% 80% ( 28) 65% Non-orphans/ non-generics * (literature data) 2 Analysis of clinical reports published in the context of Policy 0070

Orphans • Mainly small/ very small study size (e.g. n= 2, n= 3 subjects); • Size of study population mostly accounted for in the anonymisation process (8/ 9); • Attacks envisaged linked to the type of product (e.g. gene therapy). 3 Analysis of clinical reports published in the context of Policy 0070

Orphans Methodology applied Anonymisation technique Anonymisation assessment 11% (1) 11% (1) 22% (2) risk assessment (qualitative) redaction transformation risk assessment (quantitative) fulfillment 3 criteria (6) 67% (8) 89% N(orphans)= 9 4 Analysis of clinical reports published in the context of Policy 0070

Orphans Anonym isation applied • Redaction of medical history and demographic characteristics throughout CSRs (8/ 9); Full redaction of case narratives Redaction of adverse events (2) (2) 22% 22% (3) 34% yes (1) 11% no, selected identifiers only 78% (7) (3) Anonym isation of: 33% • Dem ographic characteristics; • Medical history; • Verbatim text; 5 Analysis of clinical reports published in the context of Policy 0070

Orphans Exam ples of quantitative approaches Alprolix:  Redaction of quasi-identifiers to remove unique combinations of quasi- identifiers;  Full redaction of narratives performed;  For subgroups ≤11*, median, minimum and maximum values redacted. Darzalex:  Same approach used for non-orphan/ non-generic product (i.e. Afinitor);  Case narratives NOT fully redacted! * N patients= 1/ 0.09 6 Analysis of clinical reports published in the context of Policy 0070

Non-orphans/ non-generics • Usually large studies (i.e. > 100 subjects); • Few studies with < 100 subjects (e.g. Phase I studies); Anonymisation assessment Anonymisation technique (1) 3% 3% (1) 7% (2) risk assessment (qualitative) redaction risk assessment (quantitative) redaction and transformation fulfillment 3 criteria 90% (26) (28) 97% N(non-orphans/ non-generics)= 29 7 Analysis of clinical reports published in the context of Policy 0070

Qualitative approach (non-orphans/ non-generics) • Qualitative risk threshold to be set (e.g. low, very low); • No calculation of re-identification risk; • Risk assessment based on subjective evaluation; • Analytical approach? • Redaction as preferred technique; • Study categorisation driven by sample size (12/ 26): what is small/ big? • Heterogeneity in the anonymisation performed. 8 Analysis of clinical reports published in the context of Policy 0070

Qualitative approach (non-orphans/ non-generics) Anonym isation applied Full Redaction of case narratives Redaction of adverse events (3) 12% (8) (8) Redaction 31% 31% 50% (13) (4) 15% Dem ographic characteristics ( 2 6 / 2 6 ) Medical history ( 2 0 / 2 6 ) I n-text narratives (3) 11% 19% (5) 15% (4) ( 1 4 / 2 6 ) 4% (1) (3) 12% 9 Analysis of clinical reports published in the context of Policy 0070

Qualitative approach (non-orphans/ non-generics) Uniqueness of variable Uniqueness values ( 1 1 / 2 6 ) : of variable values • Criterion for identifiers Num bers of quasi identifiers selection; per trial participant • Redaction of specific ( 1 8 / 2 6 ) : Size of variable values; • Combination of variables study • Non-uniqueness population considered. considered. Num bers of quasi-identifiers Size of study population per trial participant ( 1 8 / 2 6 ) : • Study categorisation based on study characteristics; • Lack of harmonisation in the identifiers/ sections redacted. 10 Analysis of clinical reports published in the context of Policy 0070

Quantitative approach (non-orphans/ non-generics) • Quantitative risk threshold to be set (0.09); • Calculation of re-identification risk; • Transformation as additional technique (e.g. pseudo-anonymisation, offset dates, randomisation, generalisation of medical history to MedDRA HLT, HLGT and SOC); • Less conservative assumptions (data set considered, attacker knowledge); • Different methodologies applied. 11 Analysis of clinical reports published in the context of Policy 0070

Quantitative approach (non-orphans/ non-generics) Zinbryta : Afinitor : • Population in sim ilar trials used in the analysis; • Full com bined population of all studies used in the analysis; • Quasi-identifiers that are caught and those missed • Subjects grouped into equivalence classes accounted for in the risk calculation; (minimum equivalent class size= 12); • Local recoding: different transformation based on the level of risk; • Verbatim terms and sensitive data not included in the risk assessment; • Transform ation as anonymisation technique • Redaction as anonymisation technique. (dates, age, medical history). • Suppression applied to some identifiers (e.g. • No full redaction of case narratives (subject race); ID, dates, age); • Subject IDs pseudo-anonymised; • Adverse events redacted when in com bination • Full redaction of case narratives prior to risk and/ or unique ; assessment; • Redaction selected frequencies in table • Serious adverse events redacted in narratives. summarizing adverse events by body weight. 12 Analysis of clinical reports published in the context of Policy 0070

Data utility • Not integrated in the risk assessment; • Linked to aggregated data only; • Expectations of end users not clearly addressed; • Impact of full redaction of narratives not always addressed. 13 Analysis of clinical reports published in the context of Policy 0070

Conclusions • Disease and/ or study population driving the anonymisation process; • Limited experience (public release, potential adversaries, unstructured text); • Limited confidence with the assumptions (threshold, data set, type of attacks). 14 Analysis of clinical reports published in the context of Policy 0070

Any questions? European Medicines Agency 30 Churchill Place • Canary Wharf • London E14 5EU • United Kingdom Telephone + 44 (0)20 3660 6000 Facsim ile + 44 (0)20 3660 5555 Send a question via our w ebsite www.ema.europa.eu/ contact Follow us on @EMA_ New s