Skip to main content

Toolkit for detecting misused epidemiological methods

Abstract

Background

Critical knowledge of what we know about health and disease, risk factors, causation, prevention, and treatment, derives from epidemiology. Unfortunately, its methods and language can be misused and improperly applied. A repertoire of methods, techniques, arguments, and tactics are used by some people to manipulate science, usually in the service of powerful interests, and particularly those with a financial stake related to toxic agents. Such interests work to foment uncertainty, cast doubt, and mislead decision makers by seeding confusion about cause-and-effect relating to population health. We have compiled a toolkit of the methods used by those whose interests are not aligned with the public health sciences. Professional epidemiologists, as well as those who rely on their work, will thereby be more readily equipped to detect bias and flaws resulting from financial conflict-of-interest, improper study design, data collection, analysis, or interpretation, bringing greater clarity—not only to the advancement of knowledge, but, more immediately, to policy debates.

Methods

The summary of techniques used to manipulate epidemiological findings, compiled as part of the 2020 Position Statement of the International Network for Epidemiology in Policy (INEP) entitled Conflict-of-Interest and Disclosure in Epidemiology, has been expanded and further elucidated in this commentary.

Results

Some level of uncertainty is inherent in science. However, corrupted and incomplete literature contributes to confuse, foment further uncertainty, and cast doubt about the evidence under consideration. Confusion delays scientific advancement and leads to the inability of policymakers to make changes that, if enacted, would—supported by the body of valid evidence—protect, maintain, and improve public health. An accessible toolkit is provided that brings attention to the misuse of the methods of epidemiology. Its usefulness is as a compendium of what those trained in epidemiology, as well as those reviewing epidemiological studies, should identify methodologically when assessing the transparency and validity of any epidemiological inquiry, evaluation, or argument. The problems resulting from financial conflicting interests and the misuse of scientific methods, in conjunction with the strategies that can be used to safeguard public health against them, apply not only to epidemiologists, but also to other public health professionals.

Conclusions

This novel toolkit is for use in protecting the public. It is provided to assist public health professionals as gatekeepers of their respective specialty and subspecialty disciplines whose mission includes protecting, maintaining, and improving the public’s health. It is intended to serve our roles as educators, reviewers, and researchers.

Peer Review reports

Background

Educated in the application of epidemiological methods, epidemiologists study where diseases occur, what causes them, and how to prevent them. According to A Dictionary of Epidemiology [1], the knowledge derived from epidemiological inquiry is not used solely for discovery purposes. It is also applied to control and prevent health problems and is used to restore, promote, and protect population health across all levels of society. Hence, by virtue of their focus on protecting the public’s health, epidemiologists, as a profession, are expected to serve the public, with the public interest trumping all others [2].

As an applied interventionist science, epidemiology is used not only to study health problems, but also to provide evidence to inform rational policy debate among interested stakeholders [3]. This evidence provides the scientific basis for correcting and, ideally, preventing health problems through government-driven health and social policy. Aside from informing policy, epidemiological data also provide the basis for individuals’ choices about lifestyle, diet, and other critical factors that influence health. Whether working as scholars, researchers, public health, or non-government agency professionals, as consultants, or even as expert witnesses in legal proceedings, the work-product and ultimate goal of the epidemiologist should be to promote and protect the public’s health, both at the population as well as the individual level.

Yet, in a world of conflicting interests, some parties may use the methods and language of epidemiology for personal gain or for corporate profit. They do so by manufacturing and casting doubt [4, 5] to confuse both policymakers and the public to the detriment of the public’s health. Goldberg and Vandenberg [6] have most recently identified commonly applied tactics used to misrepresent scientific discovery: spinning the facts to manufacture doubt, generating or perpetuating falsehoods. They point out that deceit can result in confusion that delays action by calling into question the scientific basis for concern.

Documents presenting best practices and ethics guidelines have been developed and adopted by the major epidemiology professional organizations to support the discipline and protect its integrity [7,8,9]. These provide the moral basis for epidemiology’s mission; they guide the normative practices of the discipline. While professionals who are not adherent to the guidelines can be called to account, there is no mechanism to ensure their implementation; moral suasion through peer pressure is the only enforcement mechanism.

In this commentary, our focus is on the discipline of epidemiology. The problems resulting from conflicting interests, and the strategies that can be used to protect public health from them, however, also apply to other public health disciplines, including risk assessment, toxicology, and exposure assessment.

The role of undue influence in increasing uncertainty

Policy decisions are influenced by factors and inputs related not only to health risk assessments based on epidemiological data; they are also influenced by economic, political, social values, and special interest stakeholder considerations [10]. When policies informed by epidemiological evidence are debated in government, the preference is to make policy decisions in the presence of the greatest possible certainty. However, absolute certainty is not possible in science, given the inherent uncertainty that accompanies scientific inquiry. Consequently, epidemiologists are usually cautious and provide caveats for their findings. This creates an entry point for those bent on manipulating policy to promote confusion and engage in disinformation [11].

Poorly or inappropriately designed and executed epidemiological research that makes its way into the scientific literature serves to increase uncertainty. This renders the policy maker less likely to vote in favor of a policy change in support of public health. If the science can be muddied to foment uncertainty, or perhaps to mislead, a policy could ensue that leads to even more adverse population health risks.

Aware of this, a well-developed strategy among those with a vested self-interest in influencing and undermining policy, in a manner that is not consistent with the health of the public, is to find ways to increase scientific uncertainty, or to outright mislead. Science can be misused, either intentionally, through error, or from bias. In epidemiology, bias is defined as “an error in the conception and design of a study—or in the collection, analysis, interpretation, reporting, publication, or review of data—leading to results or conclusions that are systematically (as opposed to randomly) different from truth [1].”

Financial conflict-of-interest (COI), including author financial ties, review sponsorship, and journal funding, introduces a bias at all levels of the research and publication process [12]. Contrary to what many scholars might believe, this bias is not prevented by the peer review process [12]. Distortion and disinformation practices regarding scientific methods and evidence were intentionally employed by the lead industry in the early 1900s [13] and, since the 1950s, by the tobacco industry [14], and have since been honed by the asbestos industry [15,16,17,18]. The methods have become more sophisticated over time as played out from one industry to the next [19]. The goal is to pollute the scientific literature with studies designed to serve the interests of powerful sponsors and special interests. While scientists routinely disagree, the most intransigent disagreements arise when financial [20, 21], as well as political [22] interests are at play. When people become ill, die prematurely, and the health of future generations is placed in jeopardy, then transparency about the stakes involved becomes even more pressing.

Most recently, COVID-19 has taught us the importance of following epidemiological evidence in policy and health decision-making, especially in a global pandemic [3]. It has starkly revealed science’s politicization, corruption, and suppression [22]. Indeed, the pandemic has exposed relationships that need to be confronted if profiteering is to be contained, calling on values that support the public interest rather than self-serving relationships with industry [23]. Health harms are likely when the public is exposed to misinformation. Confusion ensues, which in turn creates a space for the mistrust of science, and the amplification of conspiracy theories through social media—resulting in aberrant behaviors that hamper vital public health measures [24].

Recognizing the range of factors affecting the policy process, and how they compete with one another, would help public health scientists appreciate the vulnerability of their discipline to being perverted for manipulating science, misguiding policy development, and supporting special interests. By following the money, one can identify the role that influence has played, and how this has encouraged the misuse of epidemiology [25]. The conduct of invalid science for generating “evidence” involving researchers financially supported by special interests (e.g., [26,27,28,29]), is a common and worrisome practice.

Methods

Forces having direct or indirect financial stakes in policy interventions, especially those with a short-term focus on reports reflecting profits or personal gain to stakeholders, have been shown to be the most active in effectively working against the public’s health. Particularly, there has been a precipitous increase in the corporate funding of epidemiological research and an ever-growing reliance of academic institutions on such sources of funding. This has resulted in increasing instances of conflicting interests [30] which were brought to attention in the 2020 International Network for Epidemiology in Policy (INEP) Position Statement on Conflict-of-Interest and Disclosure in Epidemiology [31].

INEP is the major global network of epidemiologists with a focus on providing a bridge between epidemiological research and evidence-based, rational, government-formulated health policy that serves the public interest. It thus provides a unique forum to protect and promote public health, and works to ensure scientific integrity, promote ethical conduct in research, and support evidence-based research findings that are both independent and transparent. Its byline states: “Integrity, Equity, and Evidence in Policies Impacting Health.”

Thanks to investigative journalism, exposés of corporate and political influence in the United States (U.S.) in the period 2017–2021, reveal how the Environmental Protection Agency (EPA) under the Trump administration, the American Chemistry Council (ACC), and industry law firms colluded to weaken the EPA’s new chemical safety reviews [32]. The exposés also reveal how the fossil fuel industry has persisted over decades in influencing policy by obfuscating and denying negative impacts on human and planetary health [33,34,35,36]. Two seminal volumes, rich in well-established examples, were produced by the European Environment Agency [37, 38]. Furthermore, the harmful impacts of powerful influence through research sponsorship have been recently recognized, such that research and related professional sponsorship by Big Oil and Tobacco are being strongly discouraged [39].

On June 10, 2013, a few years prior to the aforementioned exposés, Dr. Margaret Chan, World Health Organization (WHO) Director General, made the following statement to the 8th Global Conference on Health Promotion, held in Helsinki, Finland: “… In the view of WHO, the formulation of health policies must be protected from distortion by commercial or vested interests [40].” INEP began to develop its Position Statement in 2014, soon after the WHO Director General’s pronouncement. What Dr. Chan noted indeed is an ongoing phenomenon.

With INEP working at the interface of research and policy, its mission includes recognizing and highlighting the misuse of data and potential corruption of the science practiced by epidemiologists. INEP comprises 24 national and international volunteer member associations and societies of epidemiology across five continents. It is registered as a 501(c)(3) public charity in the U.S. It is thus well positioned internationally to develop strategies to combat the misuse of epidemiological science. The INEP Position Statement [31] addresses two questions:

  1. a)

    How is it that public health policy remains under siege?

  2. b)

    Could public health be better protected through the improved management of Conflict-of-Interest and Disclosure in Epidemiology?

To address these questions, the INEP Position Statement [31] equips scientists with a set of tools to expose and root out so-called science that is designed to mislead and deceive. Hopefully, the actions of those drawing from the methods exposed in the Toolkit Table 1 (presented in the Results section below) to distort science should become less influential. Their influence will diminish because reviewers of epidemiological studies, be they peer reviewers or otherwise, should, by virtue of the toolkit, be more effective in identifying invalid science introduced to delay policy actions for protecting public health.

The toolkit’s role in the litigation process, from deposition to cross-examination in court proceedings, should also be helpful in both the pursuit of truth and for ensuring social justice. With the potential of the toolkit for bolstering the integrity of the discipline, we recognize that there are many journals with no or ineffective peer review; and, that industries have bought their own journals, limiting the extent to which the literature could be freed of corrupted science. The once-revered peer-review process is at risk, especially in journals controlled by vested interests.

Consolidation of the toolkit was thus included in the INEP Position Statement [31]. It is now made accessible as a standalone and expanded commentary. The compendium of tools provided in this commentary brings together work initially identified by Cranor [41, 42], and subsequently expanded upon by Soskolne [43,44,45] who saw the importance of expanding and consolidating this work to better arm epidemiologists, policymakers, and the scientific community with a greater appreciation for how epidemiological methods can be misused, abused, and perverted, counter to the advancement of knowledge and the public’s health.

This commentary has a role to play in not only bringing attention to, but also shining a light on, mechanisms of demonstrated influence and their harmful impacts on the advancement of science and the protection of public health. It therefore should be used as a teaching and training resource in graduate programs in epidemiology and other related public health disciplines. Every student emerging from any such program should be prepared to confront the world of malfeasance. Ideally, reviewers of manuscripts will be better positioned to separate public interest science from inappropriately designed studies that infiltrate the literature and hence the policy debate specifically to mislead science in the service of special interests.

Results

As noted above, application of the epidemiological method can be influenced by interests that manipulate it in ways to produce findings that cast doubt, foment uncertainty, and seek to mislead decision makers. Unfortunately, some epidemiologists are susceptible to incentives that induce unprofessional conduct, thereby undermining the integrity of science [46,47,48,49]. The increasing reliance of public health institutions and epidemiologists on corporate funding, as well as the influence of politics on public health research, further exacerbate harms resulting from misusing the methods of the discipline and/or misinterpreting research findings.

To counter the types of forces noted above, a listing of key methods/techniques, arguments, and tactics has been assembled in the Table 1. It is provided to help identify how epidemiologists, usually financially supported by or under the influence of vested interests, manipulate, misuse, or inappropriately apply the methods of epidemiology, or misinterpret findings, to skew results and produce invalid science. The Table 1 is a toolkit that can be used as follows:

  • By peer reviewers as a checklist of, or guide to key methodological parameters;

  • To train epidemiologists and other healthcare professionals on the ways in which epidemiology can be distorted;

  • To review the literature for invalid science or uninformative studies (e.g., underpowered studies, or misleading samples); and

  • To identify who is misusing epidemiology.

The benchmark against which the toolkit can be compared is assembled from a selection of 12 foundational epidemiological textbooks, developed since the 1970s, with more recent editions cited here and used in epidemiology training programs [50,51,52,53,54,55,56,57,58,59,60,61]. This selection is somewhat arbitrary; any well-established textbook should suffice to gain understanding about the correct use of epidemiological methods.

Biostatistical methods are relied on for the design of specific epidemiological studies. As such, statistical methods are a critical component of the epidemiologist’s toolkit. Statistics is a discipline that has been in play for a longer period than what epidemiology has. It is not surprising, therefore, to find in the statistical literature articles extending over a longer timespan that bring attention to statistical mistakes that both researchers and practitioners can make in their work [62, 63].

Inappropriate techniques applied in epidemiology, including those that manipulate findings in ways that bias them toward the null, are assembled in the Table 1. These techniques may apply to the full realm of epidemiologic inquiry, including descriptive and analytical study designs. They include the use of unbalanced discussion that emphasizes findings not supported by the data, selective disclosure of competing interests, and publication in ‘pay-to-play’ journals without appropriate peer review, and with issues involving undisclosed conflicting interests.

Table 1 Toolkit of inappropriate applications of the epidemiological method

In brief, the above Table 1, constituting the toolkit, is organized in three parts:

Part A of the Table 1 reflects on how the findings from epidemiological inquiry are affected by the design of studies, as well as on the how and what is being measured. We have compiled epidemiology-specific methods/techniques used to foment uncertainty and cast doubt about cause-and-effect through biased study designs and measurements producing invalid science.

Part B of the Table 1 reveals arguments that impose inappropriate standards and methods of suppression counter to the principle of openness and transparency. We have compiled arguments used to delay action, maintain the status quo, and create divisions among scientists by imposing inappropriate standards and methods of suppression.

Part C of the Table 1 identifies tactics imposed by those serving special interests to upset the very foundation of reason as it pertains to the core values and methods of the discipline. We have compiled tactics invoked to misdirect policy priorities through influence imposing undisclosed values from the positions taken by special interests.

Discussion

Since the compilation of this toolkit, the literature has, over the past year, seen many more examples of conflicting interests and failures to disclose them. Each example exposes the inappropriate role of influence-wielding at all levels of scientific inquiry and knowledge advancement.

In this commentary, we focus on the toolkit aspect of the INEP Position Statement [31], and thus limit ourselves in this discussion to one recent contribution to the topic of bias assessment because of its focus on methods. It appears in a 2020 commentary by Steenland et al. [102] in which they consider risk of bias (RoB) assessments and evidence syntheses for observational epidemiological studies of environmental and occupational exposures. RoB tools are used to evaluate epidemiological studies as part of evidence synthesis, the latter requiring a broader approach than simply evaluating RoB in individual studies. Those authors recognize the need to include classical considerations for judging causality in human studies, “as well as triangulation and integration of animal and mechanistic data.”

As with the INEP Position Statement [31], Steenland et al. [102] recognize conflict-of-interest, which can create the potential for bias, a bias that is not always assessed in RoB tools. They point to strong evidence that “studies authored by those with vested interest are generally favorable to those interests, hence the need to disclose potential conflict of interests.” In the view of Steenland et al. [102], if specific biases are present, reviewers should be able to detect them in evaluating studies. However, “generally not included in current risk of bias tools is potential bias because of problems in statistical methods. Concerns include choice of an inappropriate and badly fitting model, failure to model exposure–response or to evaluate different exposure–response models, incorrect use of mixed models, incorrect use of Bayesian techniques, violation of statistical assumptions (e.g., normal residuals in linear regression), overadjustment for covariates related to exposure but not to outcome, adjusting for causal intermediates, etc.”

We note that statistical models and methods are quite complex. As such, many epidemiologists and peer reviewers, as well as the general reader, may not be able to evaluate their appropriateness. Yet, bias due to COI has been increasingly considered and assessed in systematic review methodologies and RoB tools of epidemiological studies, including the Navigation Guide [103], and the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury [104].

Another domain of evidence synthesis that does not entail bias per se is “informativeness.” Consideration in this domain includes whether the study has a large enough sample size, whether the study has sufficient latency, whether results have been reported selectively, and whether the study has sufficient exposure contrast to see an effect of exposure on outcomes. This domain is sometimes called sensitivity in some evidence syntheses.

There is considerable overlap between the strategies identified in the toolkit of Goldberg and Vandenberg [6] and those independently identified in our Table 1 (above). This lends credence to our respective approaches for addressing the challenge of manufactured doubt. It adds a degree of validation to each of our respective Tables revealing strategies, arguments, and tactics used in doubt mongering. In the clinical realm, regarding disclosure as a mechanism for mitigating the effects of COI, Rimmer [105] notes that, until the introduction of a mandatory register of doctors’ interests, patients would have no idea who was funding their doctor’s voice, or who might be biased towards certain treatments. Related health professional bodies are thus calling out the biases to health and science induced by commercial interests.

In practice, broad opportunity exists to publish invalid science owing to: (1) the existence of predatory pay-to-play journals; (2) open access journals with little peer review; and (3) editors/peer reviewers who themselves have a COI and/or little-to-no knowledge of the topic under review. Given this, those who rely on the published literature, in both government and among the public, including the media, should be aware that research strategies exist that can be misleading. Above all, since professional epidemiologists are the gatekeepers of the discipline, they have the moral responsibility to execute its mission. It therefore behooves them, along with other healthcare professionals, to be familiar with this toolkit as but one mechanism for better ensuring the maintenance of professional standards of integrity [43] throughout the public health sciences.

Recommendations

Common practices to distort and misapply epidemiological science should be recognized and called out professionally when they occur. INEP member organizations, academic institutions, and other public health professionals can adopt INEP recommendations and strategies for COI management that include identification, avoidance, disclosure, and recusal [31]. It would be of added benefit to incorporate this commentary into the curriculum of graduate training programs in the health sciences and in medical schools to equip entry-level professionals to better serve as gatekeepers of the discipline.

The toolkit can be used as a guide in what to look for, to train epidemiologists and others on how epidemiology can be distorted, to evaluate the literature for invalid science or uninformative studies (e.g., underpowered studies), and to identify who it is that is misusing epidemiology along with their motivations. It can be used as a checklist for critically appraising descriptive or analytical studies pre- and post-publication, policies, and arguments in legal proceedings.

In summary, techniques to manufacture and cast doubt (i.e., irrational skepticism), targeted at policymakers and consumers through the misapplication of the epidemiological method, claim that:

  • The science is unclear

  • There is dissent (where the evidence is clear)

  • The data are inconclusive

  • Scientists are biased—“You can’t trust scientists”

  • Regulation is unjustified—“It’s a slippery slope.”

This is achieved through:

  • Delaying action

  • Influencing policy decisions—risk factors for bias

    • ◦ Pulls: Vested interest (stand to gain personally)

    • ◦ Pushes: Lobbying.

Defenses that work against epidemiology being misapplied include:

  • Correctly applying and clarifying the methods of statistical inference

  • Exposing undisclosed COI

  • Recognizing erroneous and misleading interpretations of underpowered studies

  • Acknowledging the scientific assessment of uncertainty

    • ◦ Bias; statistical (aleatoric) uncertainty; epistemic uncertainty

    • ◦ Model uncertainty; parameter uncertainty

    • ◦ Expected value = (value of outcome) X (probability of outcome)

    • ◦ Uncertainty intervals

  • Highlighting when the logic of an argument is invalid

    • ◦ False premises

    • ◦ Invalid argument

    • ◦ Misapply conclusions

  • Exposing the motives of researchers, journal editors, peer reviewers, decision makers and other stakeholders in the policy process

  • Critically appraising the evidence as presented

  • Publishing standards for good practice, e.g., the INEP Position Statement

  • Calling out malpractice.

While the Council on Publication Ethics (COPE) has guidelines designed to keep the literature free of corrupted or poor science, they are known to be inadequately enforced and are insufficient to stop the manipulation of the literature [16, 106, 107]. Actions on the part of the epidemiology community, as well as the broader health sciences, could help to change this as the problems are recognized and addressed. The scientific community should engage by recognizing and professionally calling out common practices used to distort and misapply epidemiological and other health-related sciences.

To demonstrate the seriousness of serving as gatekeepers with the moral responsibility to uphold professional standards, epidemiologists could expand upon the INEP Position Statement, using it as a launching pad to write other documents (e.g., other position statements, policy briefs, commentaries, letters, case studies, and editorials) to extend the reach of INEP’s Position Statement. Ultimately, exposing the public and policymakers to the INEP Position Statement will provide reassurance about the seriousness that professionals hold in protecting the public’s health. It is possible that, in return, with enhanced credibility in the profession, funding could be made available to support organizations like INEP as valued counterweights to the manipulation of this key public health science whose mission it is to serve the public interest above any other.

Epidemiologists and other health professionals must not be naïve. They need to remain vigilant to the variety of forces at play that influence both science and policy. In addition to vigilance, personal integrity is required to counter the influence of economically powerful entities and corrupt and/or morally bankrupt governments whose focus is not on protecting public health, but rather on protecting narrow, special interests.

This said, there are frailties in both human beings as well as in governmental structures. Sensitive to this reality, we provide specific short-term objectives that each epidemiologist could immediately implement: Recognize our professional obligation to be vigilant and especially careful in peer review to avoid contaminating the literature with invalid or poor science; and, support added oversight, as in Human Research Ethics Boards (HREBs) or Institutional Review Boards (IRBs), on the need to keep ourselves on track with the moral responsibility for being aware of and compliant with our profession’s ethics guidelines.

We recommend accepting that uncertainty is inherent in science. In our role as scientists, we strive to be value-neutral or value-free, but the human instrument is, in fact, incapable of achieving this point of neutrality or impartiality. Consequently, we need to look first to ourselves, because causal inference is a function of who it is that is making the inference which, in turn, is a function of how we apply our scientific methods. Anything that we can do to build protections into the system of self-governance that is expected of professions like epidemiology, we ought to engage with and embrace.

Conclusions

This novel toolkit exposes the negative impacts of the misuses of epidemiology. As such, it provides an essential foundation for expanding the science and methods of argumentation (i.e., disagreement) through formal logic and dialectics. While beyond the scope of this commentary, the challenge posed to develop an application (i.e., an app) based on the Table 1—to more efficiently review the literature and for rooting out invalid science and misleading conclusions—warrants further exploration in this philosophical context.

The toolkit, consistent with INEP’s mission, is made available to protect the public. It is provided to assist public health professionals whose mission includes protecting, maintaining, and improving the public’s health. Its utility lies in our more specific roles as educators, reviewers, and researchers. It is to be used to detect and professionally expose the misuse and distortions of epidemiology that result in misinformation that contaminates the literature, a domain on which the advancement of science and public policy rely.

Availability of data and materials

DOIs and hyperlinks are included throughout the literature cited.

Abbreviations

ACC:

American Chemistry Council

INEP:

International Network for Epidemiology in Policy

COI:

Conflict-of-Interest/conflicting interests

COPE:

Council on Publication Ethics

EPA:

United States’ Environmental Protection Agency

HREB:

Human Research Ethics Board

IARC:

International Agency for Research on Cancer

IRB:

Institutional Review Board

OR:

Odds Ratio

RoB:

Risk of bias

RR:

Relative Risk

U.S.:

United States

WHO:

World Health Organization

References

  1. 1.

    Porta M, editor. A dictionary of epidemiology. 6th ed. Oxford: Oxford University Press; 2014.

    Google Scholar 

  2. 2.

    Olsen J, Bertollini R, Victora C, Saracci R. Global response to non-communicable diseases—the role of epidemiologists. Int J Epidemiol. 2012;41:1219–20. https://doi.org/10.1093/ije/dys145. https://academic.oup.com/ije/article/41/5/1219/713559?login=true.

    Article  Google Scholar 

  3. 3.

    Oremus M, Taylor-Wilson R, Aldrich M, Bell K, Gaudino J, Palevsky S, Payne J, Raynes-Greenow C, Sim F, Smith M, Weiss S, Zhang Y. The role of epidemiologists in SARS-CoV-2 and COVID-19 research (Letter). Public Health. 2021;190(2021):e3–4. https://doi.org/10.1016/j.puhe.2020.10.006. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7568048/.

    CAS  Article  Google Scholar 

  4. 4.

    Michaels D. Doubt is their product: how industry’s assault on science threatens your health. New York: Oxford University Press; 2008.

    Google Scholar 

  5. 5.

    Michaels D. The triumph of doubt: dark money and the science of deception. New York: Oxford University Press; 2020.

    Google Scholar 

  6. 6.

    Goldberg RF, Vandenberg LN. The science of spin: targeted strategies to manufacture doubt with detrimental effects on environmental and public health. Environ Health. 2021;20:33. https://doi.org/10.1186/s12940-021-00723-0.

    Article  Google Scholar 

  7. 7.

    Soskolne CL, Light A. Towards ethics guidelines for environmental epidemiologists. Sci Total Environ. 1996;184(1,2):137–47. Adopted by the International Society for Environmental Epidemiology in 1999. Accessible at: http://colinsoskolne.com/documents/025.pdf.

    CAS  Article  Google Scholar 

  8. 8.

    American College of Epidemiology (ACE) ethics guidelines, January 24, 2000. https://www.acepidemiology.org/ethics-guidelines. Accessed 26 July 2021.

  9. 9.

    International Society for Environmental Epidemiology (ISEE). Ethics Guidelines for Environmental Epidemiologists, April 25, 2012. https://www.iseepi.org/docs/ISEE_Ethics_Guidelines_adopted_april_25_2012-English.pdf. Accessed 26 July 2021.

  10. 10.

    Silbergeld EK, Mandrioli D, Cranor CF. Regulating chemicals: law, science, and the unbearable burdens of regulation. Annu Rev Public Health. 2015;18(36):175–91. https://doi.org/10.1146/annurev-publhealth-031914-122654. PMID: 25785889.

    Article  Google Scholar 

  11. 11.

    Pearce N. Corporate influences on epidemiology. Int J Epidemiol. 2008;37:46–53. https://doi.org/10.1093/ije/dym270. https://academic.oup.com/ije/article/37/1/46/771539?login=true.

    Article  Google Scholar 

  12. 12.

    Mandrioli D, Kearns CE, Bero LA. Relationship between research outcomes and risk of bias, study sponsorship, and author financial conflicts of interest in reviews of the effects of artificially sweetened beverages on weight outcomes: a systematic review of reviews. PLoS One. 2016;11(9):e0162198.

    Article  Google Scholar 

  13. 13.

    Rosner D, Markowitz G. The politics of lead toxicology and the devastating consequences for children. Am J Industrial Med. 2007;50:740–56. https://doi.org/10.1002/ajim.20435.

    Article  Google Scholar 

  14. 14.

    Ong EK, Glantz SA. Constructing “sound science” and “good epidemiology”: tobacco, lawyers, and public relations firms. Am J Public Health. 2001;91(11):1749–57. https://ajph.aphapublications.org/doi/full/10.2105/AJPH.91.11.1749.

    CAS  Article  Google Scholar 

  15. 15.

    Marsili D, Terracini B, Santana VS, Ramos-Bonilla JP, Pasetto R, Mazzeo A, Loomis D, Comba P, Algranti E. Prevention of asbestos-related disease in countries currently using asbestos. Int J Environ Res Public Health. 2016;13(5):E494. https://doi.org/10.3390/ijerph13050494.

  16. 16.

    Algranti E, Ramos-Bonilla JP, Terracini B, Santana VS, Comba P, Pasetto R, Mazzeo A, Cavariani F, Trotta A, Marsili D. Prevention of asbestos exposure in Latin America within a global public health perspective. Ann Glob Health. 2019;85(1):49, 1–15. https://doi.org/10.5334/aogh.2341.

    Article  Google Scholar 

  17. 17.

    Baur X, Frank AL. Ongoing downplaying of the carcinogenicity of chrysotile asbestos by vested interests. J Occup Med Toxicol. 2021;16(1):6. https://doi-org.ezproxy.uniandes.edu.co:8443/10.1186/s12995-021-00295-2.

    Article  Google Scholar 

  18. 18.

    Ruff K, Mirabelli D. Conflict of interest, tailored science, and responsibility of scientific institutions and journals. New Solut. 2014;24(3):259–66. https://doi.org/10.2190/NS.EOV.

    Article  Google Scholar 

  19. 19.

    Van der Eijk Y, Bero LA, Malone RE. Philip Morris International-funded ‘Foundation for a Smoke-Free World’: analysing its claims of independence. Tob Control. 2019;28(6):712–8.

    Article  Google Scholar 

  20. 20.

    Bero L, Grundy Q. Why having a (non-financial) interest is not a conflict of interest (perspective). PLoS Biol. 2017;14(12): e2001221. https://doi.org/10.1371/journal.pbio.2001221.

    CAS  Article  Google Scholar 

  21. 21.

    Grundy Q, Mayes C, Holloway K, Mazzarello S, Thombs BD, Bero L. Conflict of interest as ethical shorthand: understanding the range and nature of ‘“non-financial conflict of interest”’ in biomedicine. J Clinical Epidemiol. 2020;120:1–7. https://doi.org/10.1016/j.jclinepi.2019.12.014.

    Article  Google Scholar 

  22. 22.

    Abbasi K. Covid-19: politicisation, “corruption,” and suppression of science. When good science is suppressed by the medical-political complex, people die. (Editorial). BMJ. 2020;371:m4425. https://doi.org/10.1136/bmj.m4425.

    Article  Google Scholar 

  23. 23.

    Becker C. Relationships between academic medicine leaders and industry – time for another look? JAMA. 2020;324(18):1833–4. https://doi.org/10.1001/jama.2020.21021. PMID: 33170245.

    Article  Google Scholar 

  24. 24.

    Miller BL. Science denial and COVID conspiracy theories: potential neurological mechanisms and possible responses. JAMA. 2020;324(22):2255–6. https://doi.org/10.1001/jama.2020.21332.

    CAS  Article  Google Scholar 

  25. 25.

    McHenry LB. The Monsanto Papers: Poisoning the scientific well. Int J Risk Saf Med. 2018;29(3–4):193–205. https://doi.org/10.3233/JRS-180028. PMID: 29843257.

    Article  Google Scholar 

  26. 26.

    Boffetta P, Adami HO, Cole P, Trichopoulos D, Mandel JS. Epidemiologic studies of styrene and cancer: a review of the literature. J Occup Environ Med. 2009;51(11):1275–87. https://doi.org/10.1097/JOM.0b013e3181ad49b2.

    CAS  Article  Google Scholar 

  27. 27.

    La Vecchia C, Boffetta P. Role of stopping exposure and recent exposure to asbestos in the risk of mesothelioma. Eur J Cancer Prev. 2012;21:227–30. https://doi.org/10.1097/CEJ.0b013e32834dbc56.

    CAS  Article  Google Scholar 

  28. 28.

    Acquavella J, Garabrant D, Marsh G, Sorahan T, Weed DL. Glyphosate epidemiology expert panel review: a weight of evidence systematic review of the relationship between glyphosate exposure and non-Hodgkin’s lymphoma or multiple myeloma. Crit Rev Toxicol. 2016;46(sup1):28–43. https://doi.org/10.1080/10408444.2016.1214681. Erratum in: Crit Rev Toxicol. 2018;Sep 26:1.

    CAS  Article  Google Scholar 

  29. 29.

    Ciocan C, Franco N, Pira E, Mansour I, Godono A, Boffetta P. Methodological issues in descriptive environmental epidemiology. The example of study Sentieri. Med Lav. 2021;112(1):15–33. https://doi.org/10.23749/mdl.v112i1.10099.

    Article  Google Scholar 

  30. 30.

    Kramer S, Soskolne CL. Ethics guidelines in environmental epidemiology: their development and challenges we face. Curr Environ Health Rep. 2017;4:142. https://doi.org/10.1007/s40572-017-0138-z. See https://rdcu.be/L4cj.

    Article  Google Scholar 

  31. 31.

    Soskolne CL, Caldwell JC, London L, Bero L, Gochfeld M, Cranor CF, Ramos-Bonilla JP, Mandrioli D, Sass, J, Advani S. International Network for Epidemiology in Policy (INEP) position statement series: conflict-of-interest and disclosure in epidemiology. 2020. https://epidemiologyinpolicy.org/coi-d-position-statement (95 pages). Released to the public on January 5, 2021. Accessed 26 July 2021.

  32. 32.

    Denison R. Trump EPA, ACC and industry law firms colluded to weaken EPA new chemical safety reviews. Environmental Defense Fund (EDF); 2021. http://blogs.edf.org/health/2021/03/11/trump-epa-acc-and-industry-law-firms-colluded-to-weaken-epa-new-chemical-safety-reviews/?utm_source=expert&utm_campaign=edf-health_none_upd_hlth&utm_medium=email&utm_id=1615484423. Accessed 26 July 2021.

  33. 33.

    Cook J, Supran G, Lewandowsky S, Oreskes N, Maibach E. How fossil fuel industry misled Americans deliberately about climate change. The Print; 2019. https://theprint.in/opinion/how-fossil-fuel-industry-misled-americans-deliberately-about-climate-change/315179/. Accessed 26 July 2021.

  34. 34.

    The Climate Reality Project. The climate denial machine: how the fossil fuel industry blocks climate action. 2019. https://www.climaterealityproject.org/blog/climate-denial-machine-how-fossil-fuel-industry-blocks-climate-action.

    Google Scholar 

  35. 35.

    Noor D. Trump’s EPA now says oil and gas production are ‘insignificant’ sources of pollution. January 13, 2021. https://gizmodo.com/trumps-epa-now-says-oil-and-gas-production-are-insignif-1846049568. Accessed 26 July 2021.

  36. 36.

    Milman O. Oil firms knew decades ago fossil fuels posed grave health risks, files reveal. The Guardian; 2021. https://www.theguardian.com/environment/2021/mar/18/oil-industry-fossil-fuels-air-pollution-documents. Accessed 26 July 2021.

  37. 37.

    European Environment Agency (EEA). Late lessons from early warnings: the precautionary principle 1896–2000. Environmental issue report no 22. Luxembourg: Office for Official Publications of the European Communities; 2001. https://www.eea.europa.eu/publications/environmental_issue_report_2001_22/Issue_Report_No_22.pdf/view.

    Google Scholar 

  38. 38.

    European Environment Agency (EEA). Late lessons from early warnings: science, precaution, innovation. Summary. EEA report no 1/2013. Denmark: Rosendahls‑Schultz Grafisk; 2013. https://www.eea.europa.eu/publications/late-lessons-2. Accessed 26 July 2021.

  39. 39.

    Nieuwenhuijsen M, Fletcher T, de Nazelle A, Etzel RA. Re: Sponsorship by Big Oil, like the tobacco industry, should be banned by the research community. Epidemiology. 2021; Volume Publish Ahead of Print - Issue - https://doi.org/10.1097/EDE.0000000000001325. https://journals.lww.com/epidem/Citation/9000/Re__Re__Sponsorship_by_Big_Oil,_Like_the_Tobacco.98302.aspx. Accessed 26 July 2021.

  40. 40.

    Global Policy Forum, New York, N.Y. June 12, 2013. UN NEWS CENTRE: Head of WHO criticizes “big business” and its role in public health. https://archive.globalpolicy.org/component/content/article/221-transnational-corporations/52420-who-criticizes-qbig-businessq-and-its-role-in-public-health.html. Accessed 26 July 2021.

  41. 41.

    Cranor CF. Toxic torts: science, law and the possibility of justice. New York: Cambridge University Press; 2017.

    Google Scholar 

  42. 42.

    Cranor CF. Legally poisoned: how the law puts us at risk from toxicants. Boston: Harvard University Press; 2013.

    Google Scholar 

  43. 43.

    Soskolne CL. Global, regional and local ecological change: ethical aspects of public health research and practice. Part1, Chapter 1. In: Zölzer F, Meskens G, editors. Ethics of environmental health. Routledge studies in environment and health. London and New York: Routledge, Taylor & Francis; 2017. p. 3–16.

    Google Scholar 

  44. 44.

    Soskolne CL. The role of vested interests and dominant narratives in science, risk management and risk communication. Chapter 8. In: Zölzer F, Meskens G, editors. Environmental health risks: ethical aspects. Routledge studies in environment and health. London and New York: Routledge, Taylor & Francis; 2019. p. 123–34.

    Google Scholar 

  45. 45.

    Soskolne CL. Public health and environmental health risk assessment: which paradigm and in whose best interests? In: Westra L, Gray J, Karageorgou V, editors. Ecological systems integrity: governance, law and human rights. Chapter 16. London: Earthscan; 2015. p. 191–200.

    Chapter  Google Scholar 

  46. 46.

    Congressional Research Service. Federal scientific integrity policies: a primer. 2020. p. R46614. https://crsreports.congress.gov.

    Google Scholar 

  47. 47.

    Baur X, Budnik LT, Ruff K, Egilman DS, Lemen RA, Soskolne CL. Ethics, morality, and conflicting interests: how questionable professional integrity in some scientists supports global corporate influence in public health. Int J Occup Environ Health. 2015;21:172–5.

    Article  Google Scholar 

  48. 48.

    Soskolne C, Baur X. How corporate influence continues to undermine the public’s health. Commentary on the Collegium Ramazzini symposium held in Carpi, Italy. J Sci Pract Integr. 2018;1(1). https://www.jospi.org/article/9747-how-corporate-influence-continues-to-undermine-the-public-s-health. Accessed 26 July 2021.

  49. 49.

    Baur X, Soskolne CL, Bero LA. Commentary. How can the integrity of occupational and environmental health research be maintained in the presence of conflicting interests? Environ Health. 2019;18:93. https://ehjournal.biomedcentral.com/articles/10.1186/s12940-019-0527-x.

    Article  Google Scholar 

  50. 50.

    Schlesselman JJ. Case-control studies: design, conduct, analysis. New York: Oxford University Press; 1982.

    Google Scholar 

  51. 51.

    Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3rd ed. Philadelphia: Lippincott Williams & Wilkins; 2012.

    Google Scholar 

  52. 52.

    Breslow NE, Day NE. Statistical methods in cancer research. Volume I: the analysis of case-control studies. Lyon: International Agency for Research on Cancer (IARC); 1980.

    Google Scholar 

  53. 53.

    Breslow NE, Day NE. Statistical methods in cancer research. Volume II: the design and analysis of cohort studies. Lyon: International Agency for Research on Cancer (IARC); 1987.

    Google Scholar 

  54. 54.

    Checkoway H, Pearce N, Kriebel D. Research methods in occupational epidemiology. 2nd ed. New York: Oxford University Press; 2004.

    Book  Google Scholar 

  55. 55.

    Gordis L. Epidemiology. 2nd ed. Philadelphia: Saunders; 2000.

    Google Scholar 

  56. 56.

    Hennekens CH, Buring JE. Epidemiology in medicine. Boston: Little Brown; 1987.

    Google Scholar 

  57. 57.

    Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic research: principles and quantitative methods. Belmont: Lifetime Learning Publications; 1982.

    Google Scholar 

  58. 58.

    MacMahon B, Trichopolous D. Epidemiology: principles & methods. 2nd ed. Boston: Little, Brown; 1996.

    Google Scholar 

  59. 59.

    Mausner JS, Kramer S. Mausner & Bahn epidemiology: an introductory text. Philadelphia: W.B. Saunders; 1985.

    Google Scholar 

  60. 60.

    Szklo M, Nieto FJ. Epidemiology: beyond the basics. 4th ed. Burlington: Jones & Bartlett Learning; 2019.

    Google Scholar 

  61. 61.

    Haynes RB, Sackett DL, Guyatt GH, Tugwell P. Clinical epidemiology: how to do clinical practice research. 3rd ed. Philadelphia: Lippincott Williams & Wilkins; 2005.

    Google Scholar 

  62. 62.

    Smith RA, Levine TR, Lachlan KA, Fediuk TA. The high cost of complexity in experimental design and data analysis. Type I and type II error rates in multiway ANOVA. Hum Commun Res. 2002;28(4):515–30. https://academic.oup.com/hcr/article-abstract/28/4/515/4331132?redirectedFrom=fulltext.

    Article  Google Scholar 

  63. 63.

    Makin TR, Orban de Xivry J-J. Ten common statistical mistakes to watch out for when writing or reviewing a manuscript. eLife. 2019;8:1–13. https://doi.org/10.7554/eLife.48175.

    Article  Google Scholar 

  64. 64.

    Neutra RR, Cranor CF, Gee D. The use and misuse of Bradford Hill in U.S. Tort Law. Jurimetrics J. 2018;58:127–62.

    Google Scholar 

  65. 65.

    Lieber RL. Statistical significance and statistical power in hypothesis testing. J Orthop Res. 1990;8(2):304–9. https://onlinelibrary.wiley.com/doi/abs/10.1002/jor.1100080221.

    CAS  Article  Google Scholar 

  66. 66.

    Altman DG, Bland JM. Absence of evidence is not evidence of absence. BMJ. 1995;311:485. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2550545/.

    CAS  Article  Google Scholar 

  67. 67.

    Alderson P. Absence of evidence is not evidence of absence. BMJ (Clinical research ed). 2004;328(7438):476–7. https://doi.org/10.1136/bmj.328.7438.476.

    Article  Google Scholar 

  68. 68.

    Gee D. Establishing evidence for early action: the prevention of reproductive and developmental harm. Basic Clin Pharmacol Toxicol. 2008;102(2):257–66. https://doi.org/10.1111/j.1742-7843.2008.00207.x. PMID: 18226081.

    CAS  Article  Google Scholar 

  69. 69.

    Huo D, Anderson D, Palmer JR, Herbst AL. Incidence rates and risks of diethylstilbestrol-related clear-cell adenocarcinoma of the vagina and cervix: Update after 40-year follow-up. Gynecol Oncol. 2017;146(3):566–71. https://doi.org/10.1016/j.ygyno.2017.06.028. Epub 2017 Jul 6. PMID: 28689666.

    CAS  Article  Google Scholar 

  70. 70.

    Alpert N, van Gerwen M, Taioli E. Epidemiology of mesothelioma in the 21st century in Europe and the United States, 40 years after restricted/banned asbestos use. Transl Lung Cancer Res. 2020;9(Suppl 1):S28–38. https://doi.org/10.21037/tlcr.2019.11.11. PMID: 32206568; PMCID: PMC7082259.

    Article  Google Scholar 

  71. 71.

    International Agency for Research on Cancer. World Health Organization. IARC monographs on the identification of carcinogenic hazards to humans. Preamble. Lyon; 2019. https://monographs.iarc.who.int/wp-content/uploads/2019/07/Preamble-2019.pdf. Accessed 26 July 2021.

  72. 72.

    Sass J. MacLennan et al. report on an elevated incidence of prostate cancer among workers in a triazine manufacturing plant. J Occup Environ Med. 2003;45(4):343–4. https://doi.org/10.1097/01.jom.0000063624.37065.7a. author reply 344. PMID: 12708135.

  73. 73.

    Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ. 2001;322(February):355–7. https://pubmed.ncbi.nlm.nih.gov/11159665/.

    CAS  Article  Google Scholar 

  74. 74.

    Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM, eds. Developing a protocol for observational comparative effectiveness research: a user’s guide. AHRQ publication no. 12(13)-EHC099. Rockville: Agency for Healthcare Research and Quality; 2013. www.effectivehealthcare.ahrq.gov/Methods-OCER.cfm. https://www.ncbi.nlm.nih.gov/books/NBK126191/. Accessed 26 July 2021.

  75. 75.

    Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ, Egger M. Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4(10): e297. https://doi.org/10.1371/journal.pmed.0040297.

    Article  Google Scholar 

  76. 76.

    Hauptmann M, Lubin JH, Stewart PA, Hayes RB, Blair A. Mortality from lymphohematopoietic malignancies among workers in formaldehyde industries. J Natl Cancer Inst. 2003;95(21):1615–23.

    CAS  Article  Google Scholar 

  77. 77.

    Mao Q, Manservisi F, Panzacchi S, Mandrioli D, Menghetti I, Vornoli A, Bua L, Falcioni L, Lesseur C, Chen J, Belpoggi F, Hu J. The Ramazzini Institute 13-week pilot study on glyphosate and Roundup administered at human-equivalent dose to Sprague Dawley rats: effects on the microbiome. Environ Health. 2018;17(1):50. https://doi.org/10.1186/s12940-018-0394-x. PMID: 29843725; PMCID: PMC5972442.

    CAS  Article  Google Scholar 

  78. 78.

    Manservisi F, Lesseur C, Panzacchi S, Mandrioli D, Falcioni L, Bua L, Manservigi M, Spinaci M, Galeati G, Mantovani A, Lorenzetti S, Miglio R, Andrade AM, Kristensen DM, Perry MJ, Swan SH, Chen J, Belpoggi F. The Ramazzini Institute 13-week pilot study glyphosate-based herbicides administered at human-equivalent dose to Sprague Dawley rats: effects on development and endocrine system. Environ Health. 2019;18(1):15. https://doi.org/10.1186/s12940-019-0453-y. PMID: 30857531; PMCID: PMC6413565.

    Article  Google Scholar 

  79. 79.

    Kavlock RJ, Daston GR, DeRosa, Fenner-Crisp P, Gray LE, Kaattari S, Lucier G, Luster M, Mac MJ, Maczka C, Miller R, Moore J, Rolland R, Scott G, Sheehan DM, Sinks T, Tilson HA. Research needs for the risk assessment of health and environmental effects of endocrine disruptors: a report of the U.S. EPA-sponsored workshop. Environ Health Perspect. 1996;104(Suppl 4):715–40. https://doi.org/10.1289/ehp.96104s4715. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1469675/.

    Article  Google Scholar 

  80. 80.

    Rose G, Barker DJ. Epidemiology for the uninitiated. Chapter 4: measurement bias and error. Br Med J. 1978;2(6149). Available at https://www.bmj.com/about-bmj/resources-readers/publications/epidemiology-uninitiated. Accessed 26 July 2021.

  81. 81.

    Eick SM, Goin DE, Chartres N, Lam J, Woodruff TJ. Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools. Syst Rev. 2020;9(1):249. https://doi.org/10.1186/s13643-020-01490-8. https://pubmed.ncbi.nlm.nih.gov/33121530/.

    Article  Google Scholar 

  82. 82.

    Portier CJ, Armstrong BK, Baguley BC, Baur X, Belyaev I, Bellé R, Belpoggi F, Biggeri A, Bosland MC, Bruzzi P, Budnik LT, Bugge MD, Burns K, Calaf GM, Carpenter DO, Carpenter HM, López-Carrillo L, Clapp R, Cocco P, Consonni D, Comba P, Craft E, Dalvie MA, Davis D, Demers PA, De RoosDeWitt AJJ, Forastiere F, Freedman JH, Fritschi L, Gaus C, Gohlke JM, Goldberg M, Greiser E, Hansen J, Hardell L, Hauptmann M, Huang W, Huff J, James MO, Jameson CW, Kortenkamp A, Kopp-Schneider A, Kromhout H, Larramendy ML, Landrigan PJ, Lash LH, Leszczynski D, Lynch CF, Magnani C, Mandrioli D, Martin FL, Merler E, Michelozzi P, Miligi L, Miller AB, Mirabelli D, Mirer FE, Naidoo S, Perry MJ, Petronio MG, Pirastu R, Portier RJ, Ramos KS, Robertson LW, Rodriguez T, Röösli M, Ross MK, Roy D, Rusyn I, Saldiva P, Sass J, Savolainen K, Scheepers PT, Sergi C, Silbergeld EK, Smith MT, Stewart BW, Sutton P, Tateo F, Terracini B, Thielmann HW, Thomas DB, Vainio H, Vena JE, Vineis P, Weiderpass E, Weisenburger DD, Woodruff TJ, Yorifuji T, Yu IJ, Zambon P, Zeeb H, Zhou SF. Differences in the carcinogenic evaluation of glyphosate between the International Agency for Research on Cancer (IARC) and the European Food Safety Authority (EFSA). J Epidemiol Community Health. 2016;70(8):741–5. https://doi.org/10.1136/jech-2015-207005. Epub 2016 Mar 3. PMID: 26941213; PMCID: PMC4975799.

    Article  Google Scholar 

  83. 83.

    Frakt AB, Bagley N. Protection or harm? Suppressing substance-use data. N Engl J Med. 2015;372(20):1879–81. https://doi.org/10.1056/NEJMp1501362. https://pubmed.ncbi.nlm.nih.gov/25875196/.

    CAS  Article  Google Scholar 

  84. 84.

    Tran TH, Steffen JE, Clancy KM, Bird T, Egilman DS. Talc, asbestos, and epidemiology: corporate influence and scientific incognizance. Epidemiology. 2019;30(6):783–8. https://doi.org/10.1097/EDE.0000000000001091.

    Article  Google Scholar 

  85. 85.

    Suter G, Nichols J, Lavoie E, Cormier S. Systematic review and weight of evidence are integral to ecological and human health assessments: they need an integrated framework. Integr Environ Assess Manag. 2020;16:718–28. https://doi.org/10.1002/ieam.4271.

    Article  Google Scholar 

  86. 86.

    Jinot J, Fritz JM, Vulimiri SV, Keshava N. Carcinogenicity of ethylene oxide: key findings and scientific issues. Toxicol Mech Methods. 2018;28(5):386–96. https://doi.org/10.1080/15376516.2017.1414343.

    CAS  Article  Google Scholar 

  87. 87.

    Risk Assessment Forum. Guidelines for carcinogen risk assessment. Washington, DC: U.S. Environmental Protection Agency; 2005. 166 pages. EPA/630/P-03/001B. https://www3.epa.gov/airtoxics/cancer_guidelines_final_3-25-05.pdf.

    Google Scholar 

  88. 88.

    Pearce N, Vandenbroucke JP, Lawlor DA. Causal inference in environmental epidemiology: old and new approaches. Epidemiology. 2019;30(3):311–6. https://doi.org/10.1097/EDE.0000000000000987. https://journals.lww.com/epidem/Fulltext/2019/05000/Causal_Inference_in_Environmental_Epidemiology_.1.aspx.

    Article  Google Scholar 

  89. 89.

    Smith MT, Guyton KZ, Gibbons CF, Fritz JM, Portier CJ, Rusyn I, DeMarini DM, Caldwell JC, Kavlock RJ, Lambert PF, Hecht SS, Bucher JR, Stewart BW, Baan RA, Cogliano VJ, Straif K. Key characteristics of carcinogens as a basis for organizing data on mechanisms of carcinogenesis. Environ Health Perspect. 2016;124(6):713–21. https://doi.org/10.1289/ehp.1509912. Epub 2015 Nov 24. PMID: 26600562; PMCID: PMC4892922.

    CAS  Article  Google Scholar 

  90. 90.

    National Academies of Sciences, Engineering, and Medicine. The use of systematic review in EPA’s toxic substances control act risk evaluations. Washington, DC: The National Academies Press; 2021. https://doi.org/10.17226/25952.

    Book  Google Scholar 

  91. 91.

    Grandjean P. Delayed discovery, dissemination, and decisions on intervention in environmental health: a case study on immunotoxicity of perfluorinated alkylate substances. Environ Health. 2018;17:62. https://doi.org/10.1186/s12940-018-0405-y.

    Article  Google Scholar 

  92. 92.

    Temkin AM, Hocevar BA, Andrews DQ, Naidenko OV, Kamendulis LM. Application of the key characteristics of carcinogens to per and polyfluoroalkyl substances. Int J Environ Res Public Health. 2020;17(5):1668. https://doi.org/10.3390/ijerph17051668. PMID: 32143379; PMCID: PMC7084585.

    CAS  Article  Google Scholar 

  93. 93.

    Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305–7. https://doi.org/10.1038/d41586-019-00857-9. PMID: 30894741.

    CAS  Article  Google Scholar 

  94. 94.

    Francis JA, Shea AK, Samet JM. Challenging the epidemiologic evidence on passive smoking: tactics of tobacco industry expert witnesses. Tob Control. 2006;15(Suppl 4):iv68–76. https://doi.org/10.1136/tc.2005.014241. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2563583/.

    Article  Google Scholar 

  95. 95.

    Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, Altman DG. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337–50. https://doi.org/10.1007/s10654-016-0149-3. Epub 2016 May 21. PMID: 27209009; PMCID: PMC4877414.

    Article  Google Scholar 

  96. 96.

    Cranor CF. Milward v. Acuity Specialty Products: advances in general causation testimony in toxic tort litigation . Wake Forest J Law Policy. 2013;3:105–37.

  97. 97.

    Cogliano VJ, Baan RA, Straif K, Grosse Y, Secretan B, El Ghissassi F. Use of mechanistic data in IARC evaluations. Environ Mol Mutagen. 2008;49:100–9.

  98. 98.

    Elliott KC, Resnik DB. Science, policy, and the transparency of values. Environ Health Perspect. 2014;122:647–50. https://doi.org/10.1289/ehp.1408107.

    Article  Google Scholar 

  99. 99.

    Traversy G, Barnieh L, Akl EA, Allan GM, Brouwers M, Ganache I, Grundy Q, Guyatt GH, Kelsall D, Leng G, Moore A, Persaud N, Schünemann HJ, Straus S, Thombs BD, Rodin R, Tonelli M. Managing conflicts of interest in the development of health guidelines. CMAJ. 2021;193:E49-54. https://doi.org/10.1503/cmaj.200651.

    Article  Google Scholar 

  100. 100.

    Neltner TG, Alger HM, O’Reilly JT, Krimsky S, Bero LA, Maffini MV. Conflicts of interest in approvals of additives to food determined to be generally recognized as safe: out of balance. JAMA Intern Med. 2013;173(22):2032–6. https://doi.org/10.1001/jamainternmed.2013.10559. PMID: 23925593.

    Article  Google Scholar 

  101. 101.

    Fabbri A, Lai A, Grundy Q, Bero LA. The influence of industry sponsorship on the research agenda: a scoping review. Am J Public Health. 2018;108(11):e9–16. https://doi.org/10.2105/AJPH.2018.304677.

    Article  Google Scholar 

  102. 102.

    Steenland K, Schubauer-Berigan MK, Vermeulen R, Lunn RM, Straif K, Zahm S, Stewart P, Arroyave WD, Mehta SS, Pearce N. Risk of bias assessments and evidence syntheses for observational epidemiologic studies of environmental and occupational exposures: strengths and limitations. Environ Health Perspect. 2020;128(9):10. https://doi.org/10.1289/EHP6980.

    Article  Google Scholar 

  103. 103.

    Woodruff TJ, Sutton P. The Navigation Guide systematic review methodology: a rigorous and transparent method for translating environmental health science into better health outcomes. Environ Health Perspect. 2014;122(10):1007–14. https://doi.org/10.1289/ehp.1307175. Epub 2014 Jun 25. PMID: 24968373; PMCID: PMC4181919.

    Article  Google Scholar 

  104. 104.

    Pega F, Norris SL, Backes C, Bero LA, Descatha A, Gagliardi D, Godderis L, Loney T, Modenese A, Morgan RL, Pachito D, Paulo MBS, Scheepers PTJ, Schlünssen V, Sgargi D, Silbergeld EK, Sørensen K, Sutton P, Tenkate T, Torreão Corrêa da Silva D, Ujita Y, van Deventer E, Woodruff TJ, Mandrioli D. RoB-SPEO: a tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury. Environ Int. 2020;135:105039. https://doi.org/10.1016/j.envint.2019.105039.

    Article  Google Scholar 

  105. 105.

    Rimmer A. Nine in 10 professional organisations say doctors should have to register their financial interests. BMJ. 2021;373: n933. https://doi.org/10.1136/bmj.n933.

    Article  Google Scholar 

  106. 106.

    Ruff K. Commentary: scientific journals and conflict of interest disclosure: what progress has been made? Environ Health. 2015;14:45. https://doi.org/10.1186/s12940-015-0035-6 (8pages).

    Article  Google Scholar 

  107. 107.

    Ruff K. Serving industry, promoting skepticism, discrediting epidemiology. Chapter 7. In: Walker MJ, editor. Corporate ties that bind: an examination of corporate manipulation and vested interest in public health. New York: Skyhorse Publishing; 2017. p. 119–35; 482–5.

    Google Scholar 

Download references

Acknowledgements

This commentary draws heavily on the INEP Position Statement Conflict-of-Interest and Disclosure in Epidemiology, approved by the INEP Board on 16 September 2020, and which exceeded its endorsement threshold for public release by 24 December 2020 [31]. Mark J.J. McCormack reviewed the manuscript; he advanced some concepts and the idea of developing an app. Michael Power made the connection, through clinical epidemiology, with critical appraisal methods in evidence-based medicine and provided the summary/checklist of the Toolkit Table. Dany Gagnon provided technical and editing support throughout. Lastly, independent constructive reviews, as well as editor-suggested improvements and changes, helped to both refine and focus the manuscript. 

Funding

Not applicable.

Author information

Affiliations

Authors

Contributions

Under the leadership of CLS, all authors contributed to the compilation of the International Network for Epidemiology in Policy (INEP) Position Statement on Conflict-of-Interest and Disclosure in Epidemiology (see https://epidemiologyinpolicy.org/coi-d-position-statement). Thereafter, those included here have expanded in this commentary on the “Summary of techniques used to manipulate epidemiological findings” appearing on pages 34–37 of the INEP Position Statement at the above link. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Colin L. Soskolne.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Soskolne, C.L., Kramer, S., Ramos-Bonilla, J.P. et al. Toolkit for detecting misused epidemiological methods. Environ Health 20, 90 (2021). https://doi.org/10.1186/s12940-021-00771-6

Download citation

Keywords

  • Disinformation
  • Ethics
  • Flawed science
  • Manufactured scientific controversy
  • Obfuscation
  • Partiality
  • Public policy
  • Research integrity
  • Scientific misconduct
  • Undue influence