Process steps and suggestions for creating drug monographs and drug class reviews in an evidence-based formulary system

Key Points

Abstract

Given the high estimates of inappropriate healthcare decisions in this country and the well-documented evidence of inappropriate variations in healthcare, a well-functioning formulary system is of great importance. Bias in research tends to inflate or falsely report benefit in clinical trials of interventions by up to a relative 40% to 50%. Therefore, to ensure the adoption of products with valid and clinically useful outcomes, formulary systems should include in their processes the key steps used in evidence-based medicine, including the evaluation of therapeutic studies for bias and clinical usefulness to inform formulary decisions. This article outlines requirements and suggestions for obtaining studies with appropriate designs and acceptable methodology and execution, and emphasizes why such studies should be the basis for developing drug monographs and class reviews and informing decisions. (Formulary. 2008;43:135–145.)

Historically, many physicians have objected to formularies because of the perception that formularies limit prescribing choices. However, it is estimated that 20% to 50% of all healthcare decisions in the United States are inappropriate.^1–4 Therefore, it is reasonable to conclude that an evidence-based formulary is an important tool that provides users with high-quality information to aid decision making and ensure the inclusion of only agents that have been demonstrated via research that is valid (ie, probably true) to be clinically useful.⁵ This paper discusses the importance of an evidence-based formulary process and outlines the steps required for creating high-quality, evidence-based drug monographs and drug class reviews.

Clinical judgments should be informed, when possible, by valid and clinically useful evidence that is likely to result in improved outcomes in the areas of morbidity, mortality, symptom relief, physical or emotional functioning, and/or health-related quality of life. Unfortunately, many physicians, clinical pharmacists, nurses, and other healthcare professionals, along with many of those providing information to these practitioners, such as pharmaceutical manufacturers, researchers, editors, peer reviewers, publishers, and other medical information content providers, lack the skills necessary to adequately differentiate high-quality studies from poor ones; this fact has been strongly supported by our direct experience working with numerous groups nationwide.^6,7 The problem of poor research and critical appraisal skills is compounded by the weekly publication of thousands of flawed and misleading studies and opinions, even in the best medical journals. Each week, approximately 12,000 articles are added to the National Library of Medicine. It is our estimate, having evaluated thousands of published studies, that approximately ≤10% of published studies in the healthcare literature are both valid and clinically useful; others have estimated that this number is <5%.⁸ In discussions concerning the state of scientific knowledge, the Institute of Medicine concluded that it was plausible that only 4% of interventions used in healthcare have strong evidence to support them.⁹

Critical appraisal matters. In a classic study, Chalmers et al¹⁰ demonstrated that a lack of concealment of allocation inflated the appearance of benefit, as did a lack of randomization, in studies with an outcome of fatality from acute myocardial infarction (MI). Studies that were randomized with concealment of the allocation sequence, on average, reported nonsignificant findings. As studies diminished in quality in these 2 dimensions, benefit was inflated as high as an absolute 10% in favor of the intervention. This translates into a number-needed-to-treat (NNT) of 10, which would be highly clinically significant if only it were true. Results of subsequently published studies have supported the conclusion that bias tends to favor the intervention, inflating benefits by up to a relative 40% to 50%.^11–15 These high rates of falsely inflated results have been demonstrated in studies in which methods such as generation of the randomization sequence, concealment of allocation, blinding, and assessing outcomes through statistical modeling are omitted or not done correctly.^11–15

Although FDA is charged with ensuring the safety and efficacy of drugs and biologic products, the agency cannot be relied on as medical decision-makers' sole source of information about the safety and efficacy of drugs. Skilled formulary groups can help solve this quality information deficit by conducting rigorous critical appraisals of the medical literature for validity and usefulness for all agents being considered for formulary inclusion. An evidence-based approach to the evaluation of drugs is the only reliable way to know if the drug being evaluated is likely to be responsible for the reported outcomes in a study (ie, the study is valid) or if the results are likely due to bias, confounding, or chance (ie, the study is invalid). A good formulary system continuously evaluates new evidence, incorporates valid and useful information, and monitors drug usage. Such monitoring, coupled with considerations of value, is the hallmark of a high-quality formulary system.

The first key to an evidence-based formulary system, therefore, is to ensure an understanding of the relevant science. Groups can then consider other factors, such as clinical attributes; community standards; risk management considerations; patient, clinician, and employer satisfaction; cost; marketing implications; utilization; and other factors when making formulary decisions.

BUILDING A DRUG MONOGRAPH OR DRUG CLASS REVIEW

Background information. Drug monographs and drug class reviews are the foundation of high-quality formulary systems. These summaries of a drug's efficacy, advantages and disadvantages, FDA status, and other information are a central component of the pharmacy & therapeutics (P&T) process. If monographs and class reviews are created using a rigorous evidence-based process, they can be used by decision-makers as trusted resources for guiding clinical care decisions, weighing alternatives, and evaluating claims made about drugs and users' opinions or judgments.

Identifying potentially useful evidence. The next step is to create a focused clinical question and then attempt to identify potentially useful scientific evidence. An efficient way to find high-quality information is to start with the most potentially reliable secondary sources, such as those listed in Table 2, to obtain systematic reviews. In addition to these sources, the Database of Abstracts of Reviews of Effects (DARE) ( http://www.york.ac.uk/inst/crd/crddatabases.htm) can be helpful in identifying useful systematic reviews. DARE, which is maintained by University of York, identifies potential systematic reviews, assesses them for methodological quality against a set of inclusion criteria, and summarizes the results.

At the time of publication, we are unaware of any secondary source that is fully reliable. Therefore, critical appraisal auditing must be done for each systematic review being considered for inclusion in the drug monograph or class review. When auditing a secondary source, select ≥1 studies determined by the secondary source to be of the highest quality and see if they pass the appraisal. If so, select ≥1 studies determined to be of the lowest quality and see if they pass the appraisal. If both the higher- and lower-quality studies pass appraisal, it is reasonable to conclude that only quality primary sources were used in the secondary source.

If a secondary source from the originally selected sources cannot be used, the National Library of Medicine, accessed through PubMed ( http://www.pubmed.gov/), can be useful to find other systematic reviews, including meta-analyses or randomized controlled trials (RCTs). The type of article to search for can be specified using the "Type of Article" variable (found under the "Limits" section). It is advisable to use the agent's generic and proprietary name when searching PubMed. It is necessary to critically appraise all studies obtained from a PubMed search. Systematic reviews obtained from PubMed can be crosschecked by determining whether DARE has already evaluated the reviews. Editorials, comments, and related articles can provide additional information to aid in the critical appraisal process.

If any secondary source does not pass a critical appraisal audit, a source might still be usable as a basis for a monograph or class review if it is agreed that the search and exclusions have been performed rigorously. In this instance, the conclusions of the review would not be used, but rather all studies selected for inclusion in the review should be critically appraised; any that are deemed to not be both valid and clinically useful should be discarded. In this case, it is necessary to update the review with subsequent research published after the date of the secondary study's search.

Information from any secondary sources almost always requires updating, which entails a search for any studies published after the search date of the systematic review and critical appraisal of the studies determined to be potentially valid and relevant. To update, it is important to use the same search date that was used by the secondary source, rather than the publication date, which may be much later than the search date.

When documenting your search, it is helpful to include information about the clinical question or focus point of the search, sources, date of search, search terms, limits used, number of hits, number of irrelevant studies (documented with citation), and number of studies selected for review (documented with citation). Documenting the PubMed Identification Number (PMID number) and using it as a search variable is an efficient way to relocate studies.