Use of GRADE grid to reach decisions on clinical practice guidelines - PDF document

ANALYSIS Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive The large and diverse nature of guideline committees can make consensus difficult. Roman Jaeschke and colleagues describe a simple technique for clarifying opinion Guidelines have become an important vehi- to the idiosyncrasies of small or moderate In this article, we briefly review consensus cle for influencing clinical practice. Many sized group interaction. Factors including development techniques, 9 describe a qual- local, national, and international societies time pressure; fatigue; lack of expertise in ity improvement and guideline development now go through the process of identifying content, methods, or group leadership; and, group (the Surviving Sepsis Campaign), and relevant clinical areas, formulating specific most importantly, dominance by individuals introduce the GRADE grid—an instrument clinical questions, reviewing the applicable with powerful personalities and intimidating recently developed and implemented by the evidence, and formulating recommendations reputations threaten the integrity of the Surviving Sepsis Campaign for use within that they believe clinicians and their patients process. the GRADE approach. should follow. Those interested in the science of Over the years, in recognition of the guideline development have developed Formal consensus development techniques diversity of individuals required to pro- two strategies to deal with these problems. The most popular techniques for developing duce optimal recommendations (content The first uses structured approaches consensus are the Delphi method, the nom- experts, methodologists, front line clini- to collect, analyse, and summarise the inal group technique, and a combination of cians, patients’ representatives), guideline relevant evidence and to use that evidence these two approaches. The Delphi method, panels have grown in size. The resulting to produce and grade recommendations. which was originally used to forecast the large and diverse panels present challenges These approaches are epitomised by influence of technology on warfare, sys- for decision making, such as ensuring that all the method suggested by the Grading of tematically gathers opinion from a number participants have a voice and can influence Recommendations Assessment, Develop- of stakeholders or experts. Large numbers the results of the debate, ensuring transpar- ment and Evaluation (GRADE) working of participants can be included in this ency, dealing with disagreement, achieving group, which has developed an increas- process, during which contributors answer consensus, and resolving situations in which ingly widely adopted structure for devel- questionnaires in two or more rounds, usu- consensus is not possible. oping guidelines. 1-6 The second relies on ally working independently without meet- Guideline panels often use only informal somewhat formalised processes to encour- ing in person. After each round, a facilitator processes to deal with these challenges. age a consensus to which all panellists can provides an anonymous summary of the Informal processes are, however, vulnerable contribute more or less equally. 7 8 contributors’ opinions from the previous BMJ | 9 AUGUST 2008 | VOLUME 337 327

ANALYSIS GRADE grid for recording panellists’ views in development of guidelines (including examples of propositions from the Surviving Sepsis Campaign and number of panellists who voted for each option) GRADE score 1 2 0 2 1 Balance between Desirable clearly outweigh Desirable probably Trade-offs equally balanced Undesirable probably Undesirable clearly desirable and undesirable undesirable outweigh undesirable or uncertain outweigh desirable outweigh desirable consequences of intervention Recommendation Strong: “definitely do it” Weak: “probably do it’” No specific Weak: “probably don’t Strong: “definitely don’t recommendation do it” do it” For each proposition below, please mark with an “X” the cell that best corresponds to your assessment of the available evidence, in terms of benefits versus disadvantages Use of (as opposed to no use of): Low dose steroids in patients 0 5 4 8 4 with septic shock responsive to fluids and vasopressors Low dose steroids in patients 5 16 0 0 0 with septic shock poorly responsive to fluids and vasopressors SDD in ventilated patient 0 9 4 8 1 (local and systemic) rhAPC in patients with septic 6 15 1 0 0 shock and high risk of death SDD=selective digestive decontamination, rhAPC= recombinant human activated protein C. *Participants were provided with guidance on factors to be taken into account in formulating a recommendation (box 1) and the implications of strong versus weak recommendations (box 2). round, as well as the reasons they pro- each technique can vary in design and to a recommendation will outweigh the vided for their judgments. Participants are implementation. Other methods, specific undesirable effects, and the strength of a encouraged to revise their earlier answers for guidelines developers, have been recommendation reflects the group’s degree in light of the replies from other members proposed. 9 13 of confidence in that assessment. A strong of the group. In general, during this pro- recommendation in favour of an interven- cess the range of the answers decreases, Surviving Sepsis Campaign tion reflects the collective judgment that the and the group converges towards a com- Over 50 experts from more than 10 desirable effects of the intervention (benefi- mon answer. The process terminates after countries participated in the development cial health outcomes, less burden on staff a predefined stop criterion (such as number of guidelines on managing severe sepsis and patients, and cost savings) will clearly of rounds, achievement of consensus, stabil- and septic shock as part of the Surviving outweigh the undesirable effects (harms, ity of results). 9 10 Sepsis Campaign. 14 The first edition of more burden, and greater costs). A weak The nominal group technique elicits the campaign’s guidelines was published recommendation reflects the collective opinions from a smaller number of experts in 2004 and the most recent in 2008. The opinion that the desirable effects will out- who meet in person. Each person is given 2008 guidelines were developed using the weigh the undesirable effects but the panel equal opportunity to speak, and there is GRADE approach to classify the quality formal feedback by the organisers to the of underlying evidence and the strength Box 1 Factors that influence the strength of participants, structured face to face interac- of recommendations. 1 GRADE classifies recommendation tions, periods of private (non-interacting) quality of evidence as high, moderate, low, Balance between desirable and undesirable activity such as development of ideas or or very low. The system allows the quality effects —The larger the difference between the ranking opinion, and an explicit method for of evidence derived from observational desirable and undesirable effects, the more final resolution. One method of resolution data to be upgraded from low to moderate likely a strong recommendation is warranted. involves definition of several options that or high categories and the quality of evi- The narrower the gradient, the more likely a weak recommendation is warranted are ranked from most to least acceptable by dence coming from randomised trials to all participants. be downgraded depending on the details Quality of evidence —The higher the quality Both these techniques are used in a vari- of design and execution of such studies. of evidence, the more likely a strong recommendation is warranted ety of situations where consensus needs The approach to deciding on the quality of to be built and not just for guidelines. For evidence, while in its optimal application Values and preferences —The more variability in values and preferences, or more uncertainty in example, they have been shown to be highly structured, nevertheless requires values and preferences, the more likely a weak valuable in establishing national research subjective judgment and thus invites dif- recommendation is warranted priorities 11 and in developing international ferences of opinions. Costs (resource allocation) —The higher the training programmes. 12 Modifications of Subjective judgment is also involved in costs of an intervention (that is, the more these methods are common—for example, classifying recommendations as strong or resources consumed) the less likely a strong voting on options in the nominal group weak. The guideline panel has to decide recommendation is warranted technique rather than ranking—and whether the desirable effects of adherence 328 BMJ | 9 AUGUST 2008 | VOLUME 337