Guideline Development GRADE Martin Howell martin.howell@sydney.edu.au KDIGO Glomerulonephritis Guideline Update August 2018 Trusted evidence. Informed decisions. Better health.
Outline • Foundations of trustworthy guidelines • AGREE II reporting checklist • GRADE as applied to clinical practice guidelines
Acknowledgment • This presentation draws on training material provided by the GRADE working group. – www.GradeWorkingGroup.org • The on-going series of articles published from 2011 in the Journal of Clinical Epidemiology. • Key papers to read: • Andrews, J., et al. (2013). "GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations." Journal of Clinical Epidemiology 66(7): 719-725. • Andrews, J. C., et al. (2013). "GRADE guidelines: 15. Going from evidence to recommendation— determinants of a recommendation's direction and strength." Journal of Clinical Epidemiology 66(7): 726-735.
Trustworthy Guidelines 1. Transparent process 2. Conflict of interest – open and managed 3. Multidisciplinary – experts and key stakeholders – patients/consumers 4. Evidence from systematic review 5. Clear process for evidence → recommendations 6. Clearly articulated recommendations 7. Externally reviewed 8. Process for updating 9. Implementation – facilitators, barriers, resource implications ……
Foundations of trustworthy guidelines
GRADE(ing) CPGs The ultimate goal for the guideline group is to grade the recommendations Recommendations take into account • Overall quality of evidence –Study bias; inconsistency; indirectness; imprecision other biases. –Magnitude of effect; dose-response; residual confounding • Importance; balance between harms and benefits; values and preferences; costs/resources. • Strong – “We recommend….” • Weak or conditional – “We suggest….”
GRADE is outcome-centric PICO Systematic review Outcome #1 QUALITY High Outcome #2 Moderate Low Outcome #3 Very Low Study #4 Study #1 Study #2 Study #3 The quality of evidence may vary with across QUALITY the outcomes. The study quality (risk of bias) is only one part of the assessment.
Formulating key questions and systemic review Topic e.g. IgA nephropathy and IgA vasculitis 1. Key questions • Who should receive immunosuppressive therapy and who not? • What are the most effective immunosuppressive therapies to treat patients with IgA nephropathy? • What are the harms of immunosuppressive therapies in patients with IgA nephropathy? …………………etc. 2. PICO – population, intervention, comparator and outcomes for each question • Defines the scope of the systematic review • Identifies the critical outcomes
Hierarchy of Outcomes 9 Critically important outcomes Critical for decision 8 • Evidence profiles and summary of findings tables making • Overall quality determined by the most important outcome 7 6 Important but not critical outcomes Important but not • critical for decision 5 May not be included in evidence profiles • making Should play little or not part in formulation of recommendations 4 Not important outcomes 3 • Low importance Not included in evidence profiles • for decision 2 Should not be used to formulated recommendations making 1
Hierarchy of Outcomes IgA nephropathy 9 • All-cause mortality • Critical End-stage kidney disease (need for dialysis/ eGFR <15 ml/min/1.73m 2 ) • for decision 8 50% loss of GFR • making Infection • Malignancy Patient important outcomes? 7 SONG-GN Project* Fatigue, anxiety, quality of life, ability to work 6 Important but not • Complete remission (as defined by the investigator) critical for decision 5 • Annual GFR loss (minimum 3 year follow-up required) making 4 3 Low importance • for decision 2 Urine/serum biomarkers making * www.songinitiative.org 1
Population Intervention Comparator(s) Patients with IgA Immunosuppressive Placebo/other nephropathy medication immunosuppressive medication Patients with IgA Non-immunosuppressive Placebo/other non- nephropathy medication* immunosuppressive therapies Patient with IgAN in Immunosuppressive Placebo/other IgA vasculitis therapy immunosuppressive medication OUTCOMES Critical •All-cause mortality •End-stage kidney disease (need for dialysis/ eGFR <15 ml/min/1.73m 2 ) •50% loss of GFR •Infection •Corticosteroid-related adverse events, in particular diabetes induction •Malignancy Important •Complete remission (as defined by the investigator) •Annual GFR loss (minimum 3 year follow-up required)
Randomization increases initial quality 1. Risk of bias Grade down Outcome Critical P 2. Inconsistency High 3. Indirectness I Outcome Critical Moderate 4. Imprecision Low C Outcome Important 5. Publication Very low O Outcome Not bias Summary of findings 1. Large effect Grade up & estimate of effect 2. Dose for each outcome response 3. Confounders Systematic review Guideline development Formulate recommendations : Grade • For or against (direction) overall quality of evidence • Strong or weak/conditional across outcomes based on (strength) lowest quality By considering: of critical outcomes Quality of evidence • “We recommend using…” Balance benefits/harms • “We suggest using…” Values and preferences • “We recommend against using…” Revise if necessary by considering: • “We suggest against using…” Resource use (cost) www.GradeWorkingGroup.org
Quality Assessment Focus is on identifying factors that influence confidence in the magnitude of the effect. Study design Lower if… Higher if… Quality of evidence Study limitations Large effect (e.g., RR 0.5) High Randomized (design and execution) Very large effect (e.g., RR 0.2) trials Inconsistency Evidence of dose-response Moderate gradient Observational Indirectness All plausible confounding Low studies would reduce a demonstrated effect Imprecision Very low Publication bias
Quality of evidence: beyond risk of bias Definition: The extent to which the confidence in an estimate of the treatment effect is adequate to support a particular recommendation Methodological Inconsistency Indirectness Imprecision Publication limitations of results of evidence of results bias Sources of Risk of bias: indirectness: Allocation Indirect concealment comparisons Blinding Patients Intention-to-treat Interventions Follow-up Comparators Stopped early Outcomes etc.
Evidence Profiles • Provide a record of judgements made by the reviewers and/or guideline authors. • Aim to provide transparency of review process and evaluation of quality of evidence for each outcome. • Should be provided in a useable format for readers of guidelines. • The link between recommendations and profiles should be clear. – Often poorly done • KDIGO are using MAGICapp to provide transparency and make the links • https://www.magicapp.org/
Conceptualising quality ⊕⊕⊕⊕ We are very confident that the true effect lies close to High that of the estimate of the effect. We are moderately confident in the estimate of effect: ⊕⊕⊕ Moderate The true effect is likely to be close to the estimate of effect , but possibility to be substantially different. Our confidence in the effect is limited: The true effect ⊕⊕ Low may be substantially different from the estimate of the effect. We have very little confidence in the effect estimate: ⊕ Very low The true effect is likely to be substantially different from the estimate of effect.
Evidence to recommendations Recommendation A High 1 Strong Strength of Moderate Strength of B Quality of recommendation recommendation evidence C Low 2 Weak D Very low
Strength of recommendation “The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.”
Determinants strength Factors that can weaken the Explanation strength of a recommendation Lower quality evidence The higher the quality of evidence, the more likely is a strong recommendation. Uncertainty about the The larger the difference between the desirable balance of benefits versus and undesirable consequences, the more likely harms and burdens a strong recommendation is warranted. The smaller the net benefit and the lower certainty for that benefit, the more likely is a weak recommendation warranted. Uncertainty or differences The greater the variability in values and in patients’ values preferences, or uncertainty in values and preferences, the more likely weak recommendation warranted. Uncertainty about whether The higher the costs of an intervention – that is, the net benefits are worth the more resources consumed – the less likely the costs is a strong recommendation warranted.
Recommend
More recommend
