Skip to main content

Table 3 Criteria included in evidence-to-decision frameworks

From: Evidence-to-decision frameworks: a review and analysis to inform decision-making for environmental health interventions

   Priority Benefits and harms   Values Economc implications
  Organization Priority of the problem Desirable effects Undesirable effects Certainty of evidence regarding desirable and undesirable effects Balance of effects Values Certainty of evidence regarding values Resource considerations Certainty of evidence regarding resources Cost-effectiveness
1 Advisory Committee on Immunization Practices (ACIP) (US Centers for Diseased Control and Prevention) [20, 39] Is the problem of public health importance? How substantial are the desirable anticipated effects? How substantial are the undesirable anticipated effects? What is the overall certainty of this evidence for the critical outcomes? Do the desirable effects outweigh the undesirable effects? Does the target population feel that the desirable effects are large relative to undesirable effects? Is there important uncertainty about or variability in how much people value the main outcomes? Is the intervention a reasonable and efficient allocation of resources? NI Is the intervention a reasonable and efficient allocation of resources? (Cost-effectiveness is included in the explanatory text.)
2 Breast Cancer Prevention Partners (BCPP) [21] Does the intervention address cross-cutting, systemic problems? Is there evidence that the intervention has been successful in the past or does it show potential for success? Was there general agreement that the intervention would do no harm, i.e. not create unintended consequences? NI NI NI NI NI NI NI
3 California Environmental Protection Agency (CalEPA) [22] NI NI Deciding whether the proposed or current use of a pesticide results in an unacceptable risk; identifying options to minimize those risks NI NI Evaluating those options according to a value system that includes scientific, social, legal and economic factors, as well as practicality and enforceability. NI Evaluating those options according to a value system that includes scientific, social, legal and economic factors, as well as practicality and enforceability. NI NI
  California Environmental Protection Agency (CalEPA) [23] Exposure assessment (identify exposure pathways and estimate the exposure impact, including chemical quantities and household, workplace and market presence across the lifecycle) Product function (service or utility the product provides) and performance: includes: Principal manufacturer-intended uses or applications; functional and performance attributes, and relative function and performance; applicable legal requirements; useful life of the product; whether an alternative exists that is functionally acceptable, technically feasible, and economically feasible. Adverse impacts (Adverse environmental impacts; Adverse public health impacts; Adverse waste and end-of-life impacts; Environmental fate Materials and resource consumption impacts; Physical chemical hazards; Physicochemical properties, Associated exposure pathways and life cycle segments). Uncertainty analysis performed for individual factors assessed (sensitivity analysis or scenario analysis) NI NI NI Economic impacts (costs) Public health and environmental costs; cost to government agencies and non-profit organizations; internal cost Materials resource and consumption impacts NI NI
4 Evidence and Values Impact on DEcision Making (EVIDEM) [24] Disease severity; size of affected population; unmet needs; population priorities and acess Comparative effectiveness; comparative patient-perceived health / patient-reported outcomes, type of preventive or therapeutic benefit Comparative safety/tolerability Quality of evidence (validity, relevance, completeness of reporting, type of evidence) NI NI NI Comparative cost consequences - cost of intervention, other medical costs, non-medical costs; Opportunity costs and affordability NI NI
5 Grading of Recommendations, Assessment, Development and Evaluation (GRADE) (clinical - individual perspective) [17] Is the problem a priority (from the perspective of an individual patient)? How substantial are the desirable anticipated effects? How substantial are the undesirable anticipated effects? What is the overall certainty of the evidence of effects? Does the balance between desirable and undesirable effects favour the intervention or the comparison? Is there important uncertainty about, or variability in, how much people value the main outcomes? (combined with values criterion) How large are the resource requirements (costs)? What is the certainty of the evidence of resource requirements (costs)? Does the cost-effectiveness of the intervention (the out-of-pocket cost relative to the net desirable effect) favour the intervention or the comparison?
  GRADE (clinical - population perspective) [17] Is the problem a priority? Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Ni Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Does the cost-effectiveness of the intervention favor the intervention or the comparison?
  GRADE (coverage decisions) [9] Is the problem a priority? Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Does the balance between desirable and undesirable effects favour the option or the comparison? Is there important uncertainty about how much people value the main outcomes? Ni How large are the resource requirements (costs)? What is the certainty of the evidence of resource use? Does cost-effectiveness favor the option or the comparison?
  GRADE (health system or public health decisions or recommendations) [6] Is the problem a priority? Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Ni Same as clinical recommendations, individual perspective Same as clinical recommendations, individual perspective Does the cost-effectiveness of the intervention favor the option or the comparison?
6 Guide Community Preventive Services (US Centers for Diseased Control and Prevention) [26] ___ Benefits Harms “Strength of evidence” based on number of studies, study design, quality of execution, consistency, and meaningful effect Not explicit ___ Ni NI NI (Examined but does not contribute to decisions.)
7 Institute for Clinical and Economic Review (ICER) [27] ___ Comparative clinical effectiveness (involves weighing the benefits and harms/burdens of one treatment option versus another) Potential other benefits or disadvantages Confidence in the body of evidence and the accuracy of estimates of risks and benefits; certainty of net benefit Comparative clinical effectiveness (involves weighing the benefits and harms/burdens of one treatment option versus another) ___ Ni Potential budget impact (for short-term affordability assessment) NI Incremental cost-effectiveness; long-term value for money
8 International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [29, 30] Severity of disease Assessed as QALYs Assessed as QALYs ___ ___ Value of hope; Also (paraphrased): value as incorporated into QALYS Ni Net costs (resulting directly from the intervention) NI QALYs gained
9 National Institutes for Health and Care Excellence (NICE) (UK) [32] __ Benefits Harms Quality/certainty of the evidence Balance of benefits and harms; magnitude and importance of the benefits and harms of an intervention, and the potential for unintended consequences. Relative values placed on outcomes Ni Costs, resource use and economic considerations NI Cost effectiveness and other types of economic analysis
10 Navigation Guide [2, 16, 40] Exposure prevalence Benefits Assessment of risk of adverse health outcomes from (paraphrased) “exposure to a chemical or class of chemicals or other environmental exposure” Assess quality of evidence (on risk or toxicity) __ Values and preferences Ni Costs and benefits NI NI
11 Scottish Intercollegiate Guideline Network (SIGN)a [34] Is this question a priority? What benefit will the proposed intervention/action have? What harm might the proposed intervention/action do? Quality of evidence (Subcriteria: How reliable are the studies in the body of evidence? Are the studies consistent in their conclusions? Are the studies relevant to our target population? Are there concerns about publication bias?) Balancing benefits and harms How do patients value different outcomes? Ni Is the intervention /action implementable in the Scottish context? Consider existing SMC advice, cost effectiveness, financial, human and other resource implications. NI Is the intervention/action implementable in the Scottish context? Consider existing SMC advice, cost effectiveness, financial, human and other resource implications.
12 US Preventive Services Task Force (USPSTF) [36] NI Benefits Harms Certainty of net benefit Magnitude of net benefits Ni Ni NI NI NI
13 World Health Organization (WHO) [37] Priority of the problem NI NI Quality of the evidence Balance of benefits and harms Values and preferences Ni Resource implications NI NI
14 WHO INTEGRATE [38, 41]) NI NI NI Quality of evidence Balance of benefits and harms NI Ni Financial and economic considerations NI NI
Other
Equity Acceptability Feasibility Autonomy Sustainability Legal and regulatory considerations Political considerations Human rights Other considerations
NI Is the intervention acceptable to key stakeholders? Is the intervention feasible to implement? NI NI NI NI NI Framework includes additional criteria “balance of consequences”
NI NI NI NI NI NI NI NI Does the intervention support the science-based intervention goals? Is the intervention in alignment with the Guiding Principles of Paths to Prevention? Mentioned in the text only: Can the intervention be implemented in a wide range of realms?
NI Evaluating those options according to a value system that includes scientific, social, legal and economic factors, as well as practicality and enforceability. Evaluating those options according to a value system that includes scientific, social, legal and economic factors, as well as practicality and enforceability. NI NI Evaluating those options according to a value system that includes scientific, social, legal and economic factors, as well as practicality and enforceability. NI NI Identifying options to minimize those risks Selecting an effective course of action to reduce or eliminate unacceptable health or environmental risks
NI “Whether an alternative exists that is functionally acceptable..” - Table 3-1D (page 233) “Whether an alternative exists that is ... technically feasible…” (page 233) NI NI “to identify the product function, performance, and the legal requirements of the Priority Product and alternatives to ensure the selected choice is feasible” NI NI Life cycle impacts (from raw materials extraction through end-of-life disposal)
(Subcriteria of “population priorites and acess”: special populations, rare diseases, etc.) NI Mandate and scope of the health system; System capacity and appropriate use of intervention; Political, historical and cultural context (Sub-criteria of”comparative patient-perceived health”: impact on autonomy) NI NI NI NI Expert consensus/clinical practice guidelines Common goals and specific interests Environmental impact
What would be the impact on health equity? Is the intervention acceptable to patients, their caregivers, and healthcare professionals? Is the intervention feasible for patients, their caregivers and healthcare providers? NI NI NI NI NI NI
Same as clinical recommendations, individual perspective Is the intervention acceptable to key stakeholders? Is the intervention feasible to implement? NI NI NI NI NI NI
Same as clinical recommendations, individual perspective Is the option acceptable to stakeholders? Is the option feasible to implement? NI NI NI NI NI NI
Same as clinical recommendations, individual perspective Is the option acceptable to key stakeholders? Is the option feasible to implement? NI NI NI NI NI NI
What would be the impact on health equity? NI “Barriers” - a secondary consideration NI NI NI NI NI Evidence gaps Applicability to US populations, US settings of implementation, and intervention characteristics
Degree to which the intervention differentially benefits a historically disadvantaged or underserved community Delivery mechanism or relative complexity may affect adherence and outcomes NI NI NI NI NI NI “...Impact of the condition on family and caregivers...” “...Impact on improving return to work/or overall productivity...” “…potential effectiveness of future treatments…” Similarity of mechanism of action to that of other active treatments
Equity NI NI NI NI NI NI NI Labor productivity Adherence-improving factors Value of reducing uncertainty due to a new diagnostic Fear of contagion (benefit in reducing anxiety of future disease spread) Insurance value (physical and financial risk protection from new treatments) Value of hope (uncertainty: benefit may be greater than the mean) Real option value (opportunity to benefit from future advances in medicine) Scientific spillover (impact of a new technology on future patients)
Impact on health inequities NI Feasibility of implementation NI NI NI NI NI Effect on population health, if appropriate to the topic; any ethical issues, social value judgements, equity considerations and inequalities in outcomes, and policy imperatives, as well as equality legislation
NI NI NI NI NI NI NI NI Availability of safer alternatives
Equity Is the intervention/action acceptable to patients and carers compared to comparison? Consider benefits vs harms, quality of life, other patient preferences. Is the intervention/action implementable in the Scottish context? Consider existing SMC advice, cost effectiveness, financial, human and other resource implications NI NI NI NI NI “Are there any common comorbidities that could have an impact on the efficacy of the intervention? “(in the “considered judgement pro-forma 2014″ form). “Equity” and “How do patients value the different outcomes?” are mentioned in the SIGN 50 manual but not in the “considered judgement pro-forma 2014″ form.
NI NI NI NI NI NI NI NI Other factors can be considered for determing the adequacy of evidence for a recommendation: prevalence or natural history of the target condition, and biological plausibility, clinical relevance and applicability of the evidence, among others. “Grade changes may also result from changes in context (clinical context, societal values for specifc outcomes, and context of intervention and treatment.”.
Equity and human rights Acceptability Feasibility NI NI NI NI Equity and human rights NI
Health equity, equality and non-discrimination Human rights and sociocultural acceptability Feasibility and health system considerations NI NI NI NI Human rights and sociocultural acceptability Societal implications
  1. (*) The criteria listed here are from both the “Considered judgement pro-forma 2014” form and the headings in the SIGN 50 guideline handbook as these differ.
  2. Abbreviations: NI not included
  3. Footnotes
  4. (a) The criteria listed here are from both the “Considered judgement pro-forma 2014” form and the headings in the SIGN 50 guideline handbook as these differ.