Table 2 Previous risk of bias assessment tools for prevalence studies and their limitations

Hoy et al. 2012 [2]

A checklist intended to facilitate assessment of risk of bias in prevalence studies included in a systematic review, covering internal and external validity via 11 questions with yes/no answers.

• The tool appropriately focuses on internal validity. However, it is not clear how questions relate to risk of bias (e.g., “Were data collected directly from the subjects?”), and therefore whether the tool is sufficiently extensive in coverage of bias issues.

• Only having yes/no answers may not be nuanced enough for capturing a range of risk of bias, especially for questions that may not have a clear yes/no answer (e.g., “Was the length of the shortest prevalence period for the parameter of interest appropriate?”).

• It is also not clear how a range of yes/no answers aggregates into an overall risk of bias rating.

• The tool is designed for studies of prevalence of disease rather than prevalence of exposures.

Munn et al. 2014 [6]

A questionnaire intended to facilitate critical appraisal of prevalence studies in a systematic review. Provides 10 questions with “yes”, “no”, “unclear”, or “not applicable” as answers.

• The tool focuses on a range of important characteristics of study quality; however, some of these characteristics do not relate to internal validity (e.g., “Was the sample size adequate?”). It is not therefore clear how the tool can provide an accurate account of vulnerability of studies to systematic error for the purposes of a systematic review.

• The tool omits important bias issues such as selective reporting, and lacks transparency, as providing justification for judgements does not seem to be required.

• The tool is designed for studies of prevalence of disease rather than prevalence of exposures.