Table 5 Evidence grading tools used by included systematic reviews
Tool used | Number of reviews using approach | Number of reviews that used published modified versions | Number of reviews that modified/ deviated from guidelines | Originally developed for | Initial grading based on study type | Main considerations | Reproductive/ children’s health- related considerations of directness, heterogeneity, confounding | Timing of exposure and outcome assessment considered? | Considerations of absence of effects |
|---|---|---|---|---|---|---|---|---|---|
1 [71] | Assessing quality of evidence in clinical practice | Yes (observational studies start at low, or moderate in modified version by WHO) | Study design, risk of bias among primary studies, inconsistency, imprecision, indirectness, publication bias, magnitude, dose–response association, residual opposing confounding (see modified versions for additional considerations) [99, 100] | Evidence in adults vs. children mentioned under “indirectness” [67] Provided child-specific examples regarding confounding | Timing of outcome assessment (general) considered under “indirectness” [67] | No explicit guidance provided | |||
0 | 0 | Systematic reviews in environmental health | Yes (observational studies start at moderate) | Study design, risk of bias among primary studies, inconsistency, imprecision, indirectness, publication bias, magnitude, dose–response association, residual opposing confounding) | Provided child-specific examples regarding confounding | Yes, both considered as part of rating quality and strength of evidence | Yes, in terms of publication bias and in terms of “strength of evidence (i.e., evidence for no effect)” | ||
0 | 1 [81] | Literature-based risk evaluations of environmental substances | Presence of study-design features considered (Controlled exposures, exposure prior to outcome, individual outcome data, comparison groups used) | Study-design features, risk of bias, inconsistency, imprecision, indirectness, publication bias, magnitude of effect, dose–response, residual opposing confounding, consistency (across species, populations, study designs), other | For inconsistency, the lifestage at exposure and assessment considered | Timing of exposure and outcome mentioned (in general). For inconsistency and indirectness, exposure duration and timing relative to outcome considered | Yes, in translating quality to level of evidence: “evidence of no health effect” | ||
Preamble for monographs by the International Agency for Research on Cancer (IARC) (2006) [100] | 1 [68] | 0 | 1 [68] | Identification and evaluation of potential carcinogens (etiology) | No, but limitations of different study designs are considered in relation to research question | Assignment based on study design, quantity, and quality of included studies, statistical power, and consistency between studies Preceded by considerations including exposure assessment methods, temporal effects of exposure, use of biomarkers, criteria for causality (based on Bradford Hill criteria) [105]. In the most recent version from 2019, this is replaced by “considerations for assessing the body of epidemiological evidence” [13, 21, 105] | No | Temporal effects and latency considered prior to evidence grading | Yes (assigned for several unbiased, consistent, precise study results based on exposures covering the full expected range in humans, with adequate length of follow-up available) |
Centre for Evidence-Based Medicine (CEBM) guidelines (2009, 2011) [95, 96] | 1 [70] | 0 | 0 | Grading quality of evidence for use in clinical decision-making | Yes (experimental studies are rated higher than observational ones) | Study design and quality, precision, consistency, directness, magnitude of effect | No | Length of follow-up considered (in general) | No |
Scottish Intercollegiate Guidelines Network (SIGN) (2011) [97] | 1 [69] | 0 | 1 [69] | Guideline development in clinical care research | Yes (observational studies cannot score higher than “B”) | Previously determined level of evidence (based on study design, risk of bias, and likelihood that “relationship is causal”), consistency of studies, and applicability of the evidence to the target population | No | No | No |
The Best Evidence Synthesis (BES) System [101] | 1 [78] | 0 | 0 | Clinical management guidelines for acute lower back problems | No | Number, relevance, and quality of available studies | No | No | No |