Because of an increased use of herbal medicinal products (HMPs) and awareness among healthcare professionals that adverse effects are possible, the reporting of adverse effects has increased substantially. Case reports, which vary in quality, play a crucial role in reporting adverse effects of therapeutic interventions. The authors, from the Universities of Exeter and Plymouth in Exeter, United Kingdom, developed a point-based rating scale for assessing the quality of case reports. In this study, they used the scale to evaluate the quality of published case reports of adverse effects of herbal medicines. They hypothesized that the quality of these reports has improved during the last 3 decades.
The scale estimates the quality of case reports by providing a score ranging from 0 (lowest quality) to 36 (highest quality). The 18 items on the scale are grouped into 4 key aspects: (1) information about the medicinal product and treatment; (2) patient history, diagnosis, medical condition(s), and medications; (3) concomitant medications; and (4) adverse event/drug interaction information. For case reports of HMPs, 3 additional items addressed specific issues related to that kind of therapy.
The authors searched Medline, Embase, AMED, and CINAHL for the following time periods: July 16, 1986-July 16, 1988; July 16, 1996-July 16, 1998; and July 16, 2006-July 16, 2008. Case reports were included if adverse effects suspected to be caused by an HMP were described. Case reports were excluded if they described more than one case of adverse effects, reported adverse effects associated with suicide or drug abuse, involved an HMP known to be illegal in the United Kingdom, or reported adverse effects in animals.
Two authors evaluated all case reports. They coded data for all 21 items (18 general and 3 specific for herbal medicine) and calculated a total score for each case report. Disagreement between the 2 evaluators was resolved through discussion. Each report was categorized as low quality (score between 0 and 14), lower medium quality (score between 15 and 21), upper medium quality (score between 22 and 28), or high quality (score between 29 and 42).
The authors included 137 case reports in the study: 15 published between 1986 and 1988, 43 between 1996 and 1998, and 79 between 2006 and 2008.
For case reports from 1986-1988, the median score was 20.9; for 1996-1998, it was 26.3; and for 2006-2008, it was 25.3. The difference between the first 2 periods was statistically significant (P=0.015), but not between the last 2 periods (P=0.528).
The 3 best-reported items were age, description of adverse event/interaction, and sex. The worst-reported items were description and use of a formal instrument for causality assessment, description of concurrent diseases/medication conditions, and analysis/consideration of concurrent disease regarding their relevance to the adverse effect. These results are consistent with those from conventional medicine, showing that important features of case reports are frequently omitted.
A vitally important aspect of this paper added to these worst-reported items was an overall lack of clear information on product names or manufacturers of the HMPs, with almost 80% of all reports failing to state the product name or manufacturer. The lack of reporting on product and manufacturer names also accompanies and results in inadequate reporting of descriptions of the herbal preparations used, e.g., type and/or concentration of the herbal extract(s) in the preparation (i.e., if extracts are included), the list of herbal ingredients (if the HMP is a combination product), etc. This lack of clarity and detailed description makes full assessment of the case report difficult, and can greatly hinder any medical, toxicological, and/or pharmacovigilance-related significance that such report(s) may have. Also, the lack of description and definition of the HMP is inconsistent with the recommendations of the CONSORT statement for reporting of clinical trials on herbal preparations.1
Of all the case reports, 72.3% were of upper-medium (43.8%) or high quality (28.5%). Only 2.9% ranked as low-quality reports. High-quality case reports increased over time (from 0% in the first period to 27.9% in the second and, finally, to 34.2% in the third period), while the percentage of low-quality reports decreased (from 13.3% to 0% to 2.5% during the three periods). The authors noted that the median scores of the last 2 periods were not significantly different, suggesting that the quality reached a plateau in 1996.
Among the study’s limitations is the fact that the construct validity of the scale has yet to be tested and certain items may need to be reconsidered (some items may be difficult to answer without a formal discussion by a healthcare professional). Also, restricting case reports to those published only in English is a limiting factor.
“Our results are consistent with the hypothesis that the quality of case reports of adverse effects of HMPs has improved during the last three decades,” the authors concluded.
1.Gagnier JJ, Boon H, Rochon R, Moher D, Barnes J, Bombardier C. Improving the quality of reporting randomized controlled trials evaluating herbal interventions: implementing the CONSORT statement.