Skip to main content

Measuring the impact of manganese exposure on children’s neurodevelopment: advances and research gaps in biomarker-based approaches

Abstract

Background

Children’s exposure to manganese (Mn) is a public health concern and consistent policy guidelines for safe levels of Mn exposure is lacking. The complexity of establishing exposure thresholds for Mn partially relates to its dual role as an essential micronutrient with low levels required for good health, but also as a neurotoxin at high levels. Questions exist about the age-related susceptibility to excess Mn, particularly for children, and how best to measure chronic exposures. To address this concern we conducted a systematic review of studies examining children’s exposure to Mn and neurodevelopmental outcomes focused on selection of biomarker-based and environmental measurements of Mn exposure to identify the scientific advances and research gaps.

Methods

PubMed and EMBASE databases were searched through March 2016 for studies that were published in English, used a biomarker-based or environmental measurement of Mn exposure, and measured at least one neurological outcome for children aged 0–18 years. Ultimately, thirty-six papers from 13 countries were selected. Study designs were cross-sectional (24), prospective cohorts (9), and case control (3). Neurodevelopmental outcomes were first assessed for Mn exposure in infants (6 papers), toddlers or preschoolers (3 papers) and school-age children (27 papers).

Results

Studies of school-aged children most frequently measured Intelligence Quotient (IQ) scores using Mn biomarkers of hair or blood. Higher hair concentrations of Mn were consistently associated with lower IQ scores while studies of blood biomarkers and IQ scores had inconsistent findings. Studies of infants and toddlers most frequently measured mental and psychomotor development; findings were inconsistent across biomarkers of Mn (hair, cord blood, tooth enamel, maternal or child blood and dentin).

Although few studies measured environmental sources of Mn, hair biomarkers were associated with Mn in drinking water and infant formula. Only one paper quantified the associations between environmental sources of Mn and blood concentrations.

Conclusion

Hair-Mn was the more consistent and valid biomarker of Mn exposure in school-aged children. Accurate measurement of children’s exposure to Mn is crucial for addressing these knowledge gaps in future studies. However, research on biomarkers feasible for fetuses and infants is urgently needed given their unique vulnerability to excessive Mn.

Peer Review reports

Background

Neurodevelopmental disabilities exact a significant toll on children. The global burden of attention deficit hyperactivity disorder (ADHD)/hyperkinetic disorder (HD) was estimated at 5.3 % in 2006 with acknowledgement of the limitations of pooled national estimates [1]. Recent prevalence estimates from the United States (U.S.) identify 4.9 million (8 %) children are learning disabled and another 5.9 million (9.5 %) have attention deficit disorder [2]. The number of U.S. children diagnosed with learning and behavioral problems has increased with time. From 1998–2000 through 2007–2009, the prevalence of ADHD among children aged 5–17 years grew from 6.9 % to 9.0 % [3]. Smaller, subclinical decrements in brain function are more common than diagnosed disorders and such conditions may decrease children’s academic success, disturb behavior, and diminish quality of life [4]. These conditions are associated with a growing list of potential neurotoxicants including manganese (Mn). As with most divalent metals (e.g., iron, lead and cadmium), excessive environmental exposure to Mn adversely affects the brain function in adult humans and pre-clinical (animal) models of maternal-fetal dyads. The health implications for fetuses and infants are a concern given the propensity for Mn accumulation in tissue is higher during development [5], and their rapidly developing brain may be at risk of injury at lower levels of Mn exposure, relative to older children and adults [6, 7]. However, the potential adverse effects of excessive levels of Mn on the infant brain are poorly understood. Manganese is an essential micronutrient that plays a critical role in normal growth and development, particularly for brain development [8]. Humans need Mn in their daily diet because it is required for normal amino acid, lipid, protein, and carbohydrate metabolism [6]. Mn deficiencies are considered rare because Mn is present in numerous commonly consumed food items such as seafood, nuts, spinach, and tea. However, overexposure to Mn is also detrimental to health. Accumulation of Mn in the brain results in neurotoxic effects. Neurons in their early developmental stage are especially sensitive to the neurotoxic effects of Mn [9]. Animal studies demonstrate that Mn uptake by the brain is higher in the pre-weaning period, relative to later ages. Exposure to excess Mn in the prenatal and postnatal periods leads to tissue Mn deposition in the striatum and hippocampus [5, 7], brain regions that are important for cognitive function. Increased startle, hyperactivity, and learning and memory deficits are the functional consequences of exposure to excess Mn during development in rats [7, 10]. Some of these effects are long-term and persist into adulthood, despite the cessation of exposure to excess Mn [11]. Mn neurotoxicity is greater with combined prenatal and postnatal exposures than with exposure limited to either prenatal or postnatal period, and is mediated by altered neurotransmission, neuronal apoptosis and mismigration, excitotoxicity and oxidative stress [5]. In addition, Mn may indirectly affect brain function by altering tissue homeostasis of other divalent metals that are important for normal neurodevelopment, such as iron, by altering the expression of transporters that are common to all divalent metals [12].

In adult humans, excess Mn may result in anxiety, learning and memory deficits, and motor impairment [13, 14]. Inhalation of Mn is a long-standing concern for workers in the ferromanganese, iron and steel mining, welding and battery assembly industries that contain extremely high levels of Mn (>1–5 milligrams Mn/meter,3 or mg Mn/m3) [6]. Community exposures to Mn also exist and include air contaminants from industrial activities [15], residential proximity to hazardous waste [16] or ingestion of water with naturally occurring Mn [17, 18]. Mn inhalation may bypass the biliary excretion mechanism and enter the brain through facilitated diffusion and active transport across the blood-brain barrier [19], or be passively transported from the olfactory bulb to the cerebral cortex [20], Mn has been considered to be less toxic when ingested than inhaled because adult humans regulate Mn absorption in the gastrointestinal tract and usually excrete excess Mn taken orally [21]. However, infants’ regulatory system is immature thus the risk of tissue Mn accumulation is greater for fetuses and infants [6, 22] raising questions about a World Health Organization’s (WHO) [23] decision to suspend guidelines addressing Mn concentrations in water [24].

The former WHO drinking water guideline of 400 micrograms/L for Mn was withdrawn in 2011 as unnecessary with an assertion that this health-based level was well above Mn concentrations normally found in drinking water [23]. However, Frisbie and colleagues report that over 50 countries have drinking water or potential drinking-water supplies that contain a Mn concentration greater than 400 micrograms /L and argue that protective policy guidance is needed [24]. The US Environmental Protection Agency (EPA) provides Health Advisory (HA) values for unregulated contaminants that may cause non-cancerous health effects. EPA has identified that a lifetime HA at 0.3 mg/L Mn in water is not expected to cause adverse neurological effects [25]. While age-specific exposure limits are not available, for infants younger than 6 months, the lifetime HA of 0.3 mg/L Mn in water is recommended for acute exposures of 10 days, given concerns for differences in Mn content in human milk and formula and the possibility of a higher absorption and lower excretion in young infants [25].

A rapidly growing body of literature reveals the complexity of the association between exposure to Mn and children’s adverse neurodevelopmental outcomes given a child’s age, developmental and nutritional status (e.g., hemoglobin levels). However, the levels, timing and duration of exposure at which these outcomes may occur, and the potential effect of various routes of exposure to Mn (e.g., drinking water, dietary practices and contaminated air or soil), are not well established. Furthermore, the mechanisms of Mn toxicity are poorly understood and are complicated by interactions with other toxic metals such as lead (Pb) [2628] and arsenic (As) [16, 29, 30] and limited and inconsistent evidence of gender-specific neurological effects (generally greater effects in girls [31, 32], but also found in boys) [33]. Accurate measurement of children’s exposure to Mn is critical to address these knowledge gaps in future studies. Our paper examines the evidence for the association of Mn exposures to children’s neurodevelopmental outcomes, focused on the contribution of biomarkers and environmental measures for elucidating the exposure-outcome relationship.

Methods

We identified studies using PubMed and EMBASE search engines in March of 2016. The searches were conducted by combining the results from a search on ‘manganese’ combined with the results from a strategy that used the concept of neurological outcomes including the following keywords: ‘neurobehavioral manifestations’ or ‘intelligence’ or ‘child behavior’ or ‘child development’ or ‘psychomotor performance’ or ‘neuropsychological tests’ or ‘psychomotor disorders’ or ‘cognition’ or ‘intelligence test’ or ‘intelligence quotient.’ The inclusion criteria were that the article was published in English and reported a study that measured both Mn exposure and a neurological outcome in humans aged 0–18 years. Any study that met the selection criteria, regardless of the publication date, was included in an initial phase of review. Measurements of Mn exposure varied including biomarkers and environmental sources; both types of exposure measurements were included. While various neurological outcomes were assessed, no limits were placed on the types of neurological outcomes examined.

Results

The searches returned 132 unique references. Fifty-six papers were outside the scope of this review because they were published in languages other than English, were review articles or meeting abstracts, had animal subjects, or did not include both a measure of Mn exposure and a neurological outcome. Abstracts were reviewed for the remaining 76 articles; ultimately 36 papers met all selection criteria and were included in this paper (Table 1).

Table 1 Summary of study characteristics

Thirty six studies were conducted in thirteen countries investigating populations from the U.S. (six papers), Bangladesh and Mexico (five papers each), Brazil and South Korea (four papers each), Canada (three papers), Italy and China (two papers each), and France, Sweden, Taiwan, the United Kingdom and Uruguay (one paper each). Study designs were primarily cross-sectional (N = 24), and less frequently, prospective cohorts (N = 9), and case control (N = 3), although Collipp et al. [34] augmented the primary cross-sectional study with a secondary case control study (which is not included in the count of study designs). Sample sizes ranged from 16 (cases only) to 1,588 with approximately 7,639 children in total (except for children classified as controls in the case-control studies). Ten studies enrolled newborns. The ages at which neurodevelopmental outcomes were first assessed in relation to Mn exposure included infants (6 papers), toddlers or preschoolers (3 papers) and school-age children (27 papers). Exposure was more frequently measured with biomarkers (33 papers) than environmental samples (13 papers), (Table 1).

Neurodevelopmental outcomes

Studies examining the potential for the adverse impact of Mn on neurological outcomes most frequently assessed measures of IQ [1518, 26, 3032, 3541], infant and toddler development [27, 28, 4246], motor skills [33, 39, 4749], attention deficit and hyperactivity disorder [3941, 50, 51], attention [36, 43, 52, 53] memory [15, 16, 30, 31, 36, 39, 43, 54] and behavioral problems [16, 17, 29, 39, 40, 52, 53].

IQ was most frequently studied among children ages 7–14 years [15, 16, 18, 26, 3032, 35, 36, 3841] with the Wechsler Intelligence Scale for Children (WISC), consistent with its design for children ages 6 to 16 years and 11 months, using both full-scale IQ (global intelligence) and specific (verbal or performance) scores [55]. Study findings varied across study designs. Lower full-scale IQ scores were associated with increased concentrations of Mn in six studies investigating IQ as the only neurodevelopmental outcome [15, 26, 30, 32, 35, 37] and in three additional studies evaluating several neurological outcomes [16, 31, 36]. However, six studies did not report a significant association between IQ and Mn including studies only measuring IQ [18, 31, 35, 38] and one measuring additional outcomes [41]. In contrast to the former studies, Mora et al. examined several neurological outcomes and reported a positive association between postnatal Mn concentrations and IQ only for boys [39].

Motor function was measured in children 7 to 14 years and measures included grooved pegboard (a manipulative dexterity test), finger tapping [33, 39, 46, 47], the Santa Ana test which assesses manual dexterity and motor coordination [47, 48], the Bruininks – Oseretsky test which evaluates gross and fine motor functioning [49], the Aiming Pursuit test of hand dexterity [33] and subtests of motor coordination from the Luria Nebraska Battery [33]. Findings varied by study design and measures of outcomes. Mora et al. found higher concentrations of prenatal and postnatal Mn was associated with improved motor outcomes, but only in boys [39]. Oulhote et al. reported a significant association between intake of water-Mn and poorer motor function [47]. Hernández-Bonilla et al. reported a subtle, negative association of Mn with specific areas of motor speed and coordination [48]. Lucchini et al. reported higher Mn levels associated with poorer motor coordination and hand dexterity, and increased tremor intensity [33]. In contrast to the preceding studies, Parvez et al. did not find associations between Mn and motor function [49].

Among toddlers and infants ages 1 to 42 months the Bayley Scales of Infant Development (BSID-II) [56] were most frequently used to measure mental and psychomotor development [27, 42, 44, 46, 47]. A significant, inverted U-shaped association between Mn and development scores was reported in two studies. Chung et al. reported a dose-response relationship with both lower and higher concentrations of Mn associated with poorer Mental Development Index (MDI) and Psychomotor Development Index (PDI) scores in 6 month old infants [42]. Claus Henn et al. found an association between concurrent MDI scores at 12 (but not 24) months of age, but no association for PDI scores at either time period [44]. Two additional studies reported significant interactions of Mn and development scores by sex. A significant interaction of postnatal Mn exposures and poorer 6 month MDI and PDI scores and sex was reported by Gunier et al.; a significant inverse, linear relationship was seen only for girls [45]. A significant, positive interaction between postnatal Mn and sex was also seen at 24 months, but only for boys who had better MDI scores [45]. Rink et al. also reported a positive association between Mn and MDI scores only in boys, on average 29 months of age [46].

Biomarkers

Studies generally used biomarkers of children’s hair or blood to assess Mn, but a few investigators measured fetal cord blood or serum, maternal blood or children’s enamel or dentin from shed teeth; one study measured urine, (Tables 2, 3 and 4). Hair-Mn was the biomarker most consistently associated with a range of neurodevelopmental deficits. Higher levels of hair-Mn in school-aged children were significantly, and inversely associated with IQ scores [15, 16, 31, 32, 3537], learning [16, 54], memory [16, 36, 54], perceptual reasoning [15] and positively related with greater hyperactive and oppositional behavior [17, 34].

Table 2 Summary of results from studies examining manganese concentrations in hair (Hair-Mn or H-Mn)
Table 3 Summary of results from studies examining manganese concentrations in blood (blood-Mn)
Table 4 Summary of results from studies examining manganese in teeth

IQ was the most frequently identified neurodevelopmental deficit associated with hair-Mn. Lower IQ scores were associated with increased concentrations of hair-Mn in four studies investigating IQ as the only neurocognitive outcome [15, 32, 35, 37], but IQ was also determined to be the only significant association with hair-Mn in a fifth study which measured several neurological outcomes [31]. Only one study found no significant association between IQ and Mn concentrations in hair [41]. In this study the mean Mn concentration in the hair was low, perhaps because Mn exposure in this study was from historical ferroalloy emissions.

Estimates of the effect size of hair-Mn on the average, full scale IQ scores of children, (mean age 9 years), were reported by Bouchard and colleagues to decline slightly (from 106 to 104) with hair-Mn values less than 1.5 micrograms/g, but significantly so for IQ scores of 101 with mean hair-Mn values of 3.2 micrograms/g, suggestive of biological significance [32]. Evidence of a U-shaped relationship with both high and low concentrations of hair-Mn associated with lower full scale IQ scores in children, on average, 8 years old were reported by Haynes et al. [15], suggestive of Mn as both a neurotoxicant and a micronutrient. Study findings revealed a significant, negative association between the highest quartile versus middle two quartiles of hair-Mn (ß -3.66; 95 % CI: -6.9, -0.43) and full scale IQ [15].

Blood-Mn levels were associated with a neurodevelopmental outcome in nine of sixteen papers reviewed. Most studies reporting associations between blood-Mn and neurological outcomes measured several outcomes. However, seven studies only examined IQ as the primary outcome and the findings were inconsistent. Four investigations did not find an association between IQ and blood-Mn [18, 31, 35, 38], but studies by Haynes et al. [15], Kim et al. [26], Wasserman et al. [30], showed a significant, inverse association between blood-Mn and IQ scores for children, on average, 8–9 years of age with mean concentrations of blood-Mn at 9.7 micrograms/L, 14.3 micrograms/L and 14.8 micrograms/L, respectively. Evidence of an inverse, U-shaped association between low and high levels of blood-Mn and low IQ scores was seen in three studies with children [15, 42, 44], two of which used the same outcome measure. Claus Henn et al. reported a significant association between concurrent MDI scores and blood- Mn in 12 month old infants comparing the middle three Mn quintiles with the lowest Mn quintile (ß -3.3, 95%CI: -6.0, -0.7) and the highest Mn quintile (ß -2.8, 95%CI: -5.5, -0.2) [44]. Chung and colleagues also reported a significant, inverse U-shaped association between maternal blood-Mn with infant PDI scores at 6 months. Increasing maternal blood-Mn levels up to 24–28 micrograms/L were positively associated with PDI scores while higher blood-Mn concentrations were associated with decreased PDI scores suggesting adverse effects of both low (<20 micrograms/L) and (high ≥ 30 micrograms/L) maternal blood-Mn levels [42].

Evidence for the usefulness of other Mn biomarkers included three papers that reported significant associations between Mn in cord blood or serum and early life neurodevelopment indicative of the importance of prenatal Mn exposure. Takser et al. reported an inverse association between cord blood-Mn at birth (Geometric Mean: 38.5 micrograms/L) and attention and non-verbal memory in three year olds and a significant, negative association with hand skills, significantly poorer scores in boys [43]. Lin et al. found cord blood-Mn (mean 50.7 micrograms/L; SD: 16.7 micrograms/L) and blood-lead (13.0 micrograms/L; SD: 7.51 micrograms/L) levels above the 75th percentile had a significant association with overall (ß -7.03; SE = 2.56; p = 0.009), cognitive (ß -8.19, SE = 3.17; p = 0.012) and language scores (ß -6.81, SE = 2.73, p = 0.013) [28]. Yang et al. found that a high cord serum-Mn (≥75th percentile, median: 4.0 micrograms/L) was associated with significantly lower scores on a Neonatal Behavioral Neurological Assessment (NBBA) at 3 days of age [57]. An interactive, protective effect was seen with prenatal selenium (Se); as the Mn/Se ratio increased, NBNA scores decreased while high levels of Se had a protective effect in the high Mn group (Mn ≥ 9.1 micrograms/L; Se ≥ 63.1 micrograms/L).

Teeth-Mn levels were analyzed in three studies suggestive of their potential value as biomarkers of early life exposures providing insight on the timing of Mn exposure and developmental windows of susceptibility. Ericson et al. measured tooth enamel in shed molars and found significant associations between Mn levels in enamel formed during the first 20 weeks of gestation and increased childhood behavioral inhibition at 36 months [52]. Studies from the Center for the Health Assessment of Mothers and Children of Salinas (CHAMCOS) birth cohort provided findings on the timing of early life Mn exposures. Gunier et al. reported small decreases in mental and motor development among 6 month old infants in association with prenatal dentin-Mn concentrations, but only for girls whose mothers had lower hemoglobin levels [45]. Additionally, a two-fold increase of postnatal dentin-Mn, reflecting exposures from birth to 2.5 months, was associated with a small, but significant decrease for infants’ mental development scores at 6 and 12 months. A significant interaction between postnatal dentin-Mn concentrations and sex for MDI (-1.5 points; 95 % CI: -2.4, -0.6) and PDI (-1.8 points; 95 % CI: -3.3, -0.3) scores at 6 months was reported, but only for girls; it was no longer evident by 24 months. Mora et al. reported increased Mn levels in pre-and postnatal dentin adversely associated with behavior problems in school aged children [39]. In contrast, the authors also reported positive effects of pre- and postnatal dentin-Mn specific to boys including better cognition, memory and motor function.

Environmental sources of manganese

Levels of Mn in environmental sources were less frequently quantified than biomarkers of Mn. Collipp et al. found higher levels of hair-Mn in infants fed formula relative to breastfed infants [34]. Since the study’s publication, levels of Mn in infant formula have declined. This study was one of the first published papers to show an association between ingestion of dietary Mn (formula) and hair biomarkers. However, it is unclear if water containing Mn was used to reconstitute the formula which may have influenced levels of Mn in hair.

Findings supporting the exposure – outcome relationship between Mn concentrations in water, hair and child neurodevelopment were reported in three papers. Bouchard et al. reported higher levels of hair-Mn in children whose well water had higher Mn levels [32], and higher levels of Mn in water and hair were significantly associated with lower IQ scores. A 10-fold increase of Mn intake from water consumption was associated with a decrease of 2.5 IQ points (95 % CI:-3.9, -0.9; p < 0.01) among 9 year olds [32]. Oulhote et al. reported higher concentrations of Mn in hair and water were associated with poorer scores on memory, attention and motor function from the same population [47]. Average Mn water levels in this study were lower than the earlier study (20 micrograms/L. vs 300 micrograms/L.). do Nascimento et al. also reported higher levels of Mn in hair and household tap water were associated with poorer IQ scores in children 6–12 years [37].

Only one study reported blood levels of Mn associated with both a measured environmental source and neurodevelopmental outcomes. Lucchini et al. reported levels of Mn in blood and hair were both positively associated with tremor intensity in the dominant hand; the authors also found a borderline association between soil-Mn and tremor intensity [33]. Comparisons between the exposed and reference communities revealed average concentrations of Mn in soil (958 ppm versus 427 ppm), respectively. The authors describe metals in soil as good indicators of general environmental insult given their stability over time in the environment reflecting both background soil deposition and cumulative inputs from atmospheric deposition of historical industrial emissions.

Two additional studies reported higher levels of Mn in both the hair and the blood of children who lived near an industrial source of Mn [15, 31]. Haynes et al. reported low and high Mn levels in blood and hair were associated with lower full IQ and subscale scores, with significant negative associations between the highest versus middle two quartiles of blood-Mn (ß -3.51; 95 % CI: -6.64, -0.38) and hair-Mn (ß -3.66; 95 % CI: -6.9, -0.43) and full scale IQ in children ages 7–8 years [15]. Riojas-Rodriguez et al. found hair-Mn was inversely associated with verbal IQ (ß -0.29; 95%CI: -0.51, -0.08), performance IQ (ß -0.08; 95%CI: -0.32, -0.16), and total IQ (ß -0.20; 95%CI: -0.42, 0.02), in children ages 7–11 years [31]. The authors reported the 24 h median Mn in PM10 in exposed communities (0.13 micrograms/m3) was higher than the exposed communities (0.02 micrograms/m3).

Finally, Kahn et al. reported an inverse association between Mn in drinking water and children’s annual test scores in mathematics [58]. Levels of Mn in water above 400 micrograms/L (the former WHO standard) was associated with a 6.4 percentage score loss (95 % CI = 0.5, 12.3) in test scores. This study did not test any Mn biomarkers, but a prior paper showed a lack of association between blood-Mn and water-Mn [29].

Discussion

A growing body of literature has examined the association of increased levels of Mn with neurodevelopmental effects in children from across the world. The evidence is most consistent in studies reporting decrements in IQ scores among primary school-aged children exposed to excessive levels of Mn. However, the inconsistency of findings in other studies reflects, in part, the considerable variation in study design including the source of Mn (water, air, or soil), exposure pathway (ingestion or inhalation), biomarkers measured (blood, hair, teeth, urine), study population (age, sex, and developmental and nutritional status) and neurological outcomes examined (IQ, motor skills, infant or early childhood development).

A recent pilot study tested the use of fMRI to reveal specific brain changes associated with Mn exposure. The findings revealed long-term exposure to Mn in the first stage of life can decrease olfactory function. There was also evidence that Mn exposure can adversely affect the functionality of the limbic system which the authors describe as suggestive of an alteration of the brain network in addressing emotional responses [59]. While scientifically promising, this approach may be less feasible for large, population studies of infants and young children given the expense and potential resistance of parents to having their children scanned for research in the absence of disease. However, with further testing in larger samples this approach could complement the use of biomarkers in studies of Mn exposure.

While relatively few studies investigated Mn exposures with biomarkers and neurodevelopment outcomes in infants, those studies using prospective study designs provided compelling evidence of the adverse effect of Mn. Biomakers of Mn using cord blood or serum provided a temporal association between fetal Mn exposures and later outcomes including cognitive and language development scores in 2 year olds [28], attention and nonverbal memory and hand skills in 3 year olds [43], and behavioral neurological development in newborns [57].

Measurement of Mn deposits in shed teeth provided insights more precise than those of cord blood or serum into the timing of early life exposures. While the CHAMACOS study is a large and comprehensive study of potential neurodevelopmental effects from pre-and postnatal dentin-Mn exposure in school-aged children [39, 45], the findings raise questions as the direction of the effects observed with higher levels pre- and postnatal Mn included both adverse effects with behavioral outcomes and positive effects with better memory abilities [39] inconsistent with other studies of school age children reporting higher Mn levels associated with poorer memory [4, 16, 54] and cognitive outcomes [15, 16, 31, 32, 3537]. These authors posit the inconsistent findings may be due to differences in the exposure matrix used to quantify Mn levels or Mn exposure pathways or possibly that the levels of Mn in their sample could be within the range at which Mn acts as a beneficial nutrient rather a than a neurotoxicant suggesting a need for additional research [39].

Based on the studies reviewed here, hair-Mn was the most frequently examined biomarker, and it was consistently associated with lower child IQ scores suggesting hair may be the most consistent and valid biomarker for Mn to date for children in population studies. While blood-Mn was associated with a range of neurodevelopmental outcomes, the findings across studies were inconsistent.

Bouchard and colleagues acknowledged the lack of consensus on an optimal biomarker of exposure to Mn and blood-Mn levels can vary widely in the short-term and likely does not reflect long-term exposure [32]. Oulhote et al. reported that blood and urine are poor measures of Mn exposure [47].

In contrast, hair-Mn is posited by these investigators as a more consistent and valid biomarker of Mn. Bouchard et al. reported that hair-Mn will reflect the metal uptake averaged over the duration of the follicle formation although the mechanism of Mn uptake into hair is not well understood [32]. Hair typically grows 1 cm per month thereby providing an exposure estimate of 1–6 months [15]. Lucchini et al.’s preliminary analysis of hair biomarkers of Mn suggests it may be a better measure of integrated exposure and body burden over the prolonged period of hair growth, relative to biomarkers of blood or urine Mn, due to its rapid homeostatic control [33]. However, variability in hair-Mn concentrations may be related to various factors including difference in exposure, pharmacokinetics, hair pigmentation and issues of sample collection and cleaning [15]. Hair analysis for Mn requires rigorous cleaning procedures to minimize contribution of external Mn contamination without comprising endogenously incorporated Mn [33, 60].

Interpretation of Mn levels in hair must be carefully evaluated because Mn levels may be higher in some hair types than others (i.e., in darker hair), and because dye, bleach or other topical treatment may either contaminate hair or effect Mn incorporation into its structure [61, 62], although topical hair treatment is less relevant for studies of children. Additionally, in a pilot study in progress we have found some infants lack sufficient hair to analyze.

The literature also lacks sufficient analyses of the connections between the environmental source, the internal dose and the associated neurodevelopmental and cognitive outcomes. Studies reported findings supporting the exposure-outcome relationship between Mn concentrations in water, hair and adverse outcomes in child neurodevelopment [32, 37, 47]. In contrast, investigators who collected data on well water-Mn, blood-Mn and neurological outcomes failed to demonstrate an association between Mn concentrations in water and blood [18, 29, 30]. However, a statistically significant and dose-dependent association between water-Mn concentrations and IQ scores (Full Scale, Performance and Verbal) was reported [18]. This result is important as it provides strong evidence that ingestion of drinking water is a major source of environmental Mn potentially related to adverse neurodevelopment.

Few studies provided evidence of the association of environmental sources, biomarkers of Mn and neurodevelopment outcomes. Torres-Agustín et al. reported significantly higher Mn in blood and hair in an exposed (versus control) group with respiratory exposure to fine particulate matter of 2.5 microns or less in width, although only hair-Mn was significantly associated with poorer neurological outcomes [54]. This is important, though they did not report how increases in air Mn content affected Mn biomarker levels. However, Lucchini et al. reported evidence of both blood-Mn and hair-Mn being associated with increased tremor intensity in the dominant hand, and a borderline association between soil-Mn and tremor intensity. These authors report that the soil-Mn reflects past or cumulative exposures [33]. Future studies also need to quantify the association between environmental sources and selected biomarkers.

Finally, it is important that the continuum of exposure is carefully measured given the possibility of an inverted U-shaped association between Mn exposure and children’s health, neurodevelopment and cognitive outcomes. Ultimately, if public health programs are to provide prevention guidance for specific exposure sources such as drinking water, PM10 and soil regarding over-exposure to Mn, the threshold of beneficial Mn exposure must also be identified to ensure children receive the optimal benefit and the safe limit relative to their age and duration of exposure.

Conclusion

With evidence mounting for the negative impact of Mn on children, research is needed to address the gaps in the literature that would help elucidate safe levels of Mn exposure for fetuses, infants and children. There is a particular need for a consistent measurement approach to biomarkers of Mn, as well as for environmental exposure sources and neurological outcomes, to make research findings comparable across studies. Additionally, feasibility issues are important when selecting biomarkers of exposure. The most promising Mn biomarker to date for the study of children is hair, but hair collection is not feasible for all infants and cleaning exogenous contamination of hair requires particular attention to evidence-based procedures. While cord blood appears an effective biomarker for measuring fetal exposure, it is logistically challenging and expensive to collect if study participants give birth at multiple hospitals. The use of teeth as a biomarker of Mn is intriguing, but it requires a minimum of 8 years from enrollment of pregnant women before children start to shed teeth that can be analyzed for Mn concentrations. The scientific and practical challenges of selecting the best biomarkers of Mn in children suggests the need for novel applications of additional biomarkers of chronic exposure to Mn. to help inform the science and ultimately determine public health prevention policies particularly for fetuses and infants given their heightened vulnerability to excessive Mn.

Abbreviations

ADHD:

Attention deficit hyperactivity disorder

As:

Arsenic

B-Mn:

Blood biomarker of manganese

B-Pb:

Blood biomarker of lead

CBC:

Children’s behavioral checklist

Cd:

Cadmium

CHAMCOS:

Center for Health Assessment of Mothers and Children of Salinas

CI:

Confidence interval

cm:

Centimeter

Cu:

Copper

Fe:

Iron

FeS:

Iron sulfide

g:

Gram

GM:

Geometric mean

H.A.:

Health advisory

Hair-Fe:

Hair biomarker of iron

Hair-Mn:

Hair biomarker of manganese

Hair-Pb:

Hair biomarker of lead

HD:

Hyperkinetic disorder

Hg:

Mercury

I.Q.:

Intelligence quotient

L:

Liter

m3:

Cubic meter

MDI:

Mental Development Index, Bayley Scales of Infant Development

Mg:

Magnesium

mg:

Milligram

Mn:

Manganese

NS:

A statistically nonsignificant association

Pb:

Lead

PDI:

Psychomotor Development Index, Bayley Scales of Infant Development

PM10:

Particulate matter with a diameter of 10 micrometers or less

PM2.5:

Particulate matter less than 2.5 microns in width

SD:

Standard deviation

SES:

Socioeconomic status

WASI:

Wechsler Abbreviated Intelligence Test

Water-Mn:

Water as an environmental measure of manganese

WISC:

Wechsler Intelligence Scale for Children

Zn:

Zinc

References

  1. Polanczyk G, de Lima MS, Horta BL, Biederman J, Rohde LA. The worldwide prevalence of ADHD: a systematic review and metaregression analysis. Am J Psychiatry. 2007;164(6):942–8.

    Article  Google Scholar 

  2. Bloom B, Jones LI, Freeman G. Summary health statistics for U.S. children: National Health Interview Survey, 2012. Vital Health Stat. 2013;10(258):1–81.

    Google Scholar 

  3. Akinbami LJ, Liu X, Pastor PN, Reuben CA. Attention deficit hyperactivity disorder among children aged 5-17 years in the United States, 1998-2009. NCHS Data Brief. 2011;70:1–8.

    Google Scholar 

  4. Grandjean P, Landrigan PJ. Neurobehavioral effects of developmental toxicity. Lancet Neurol. 2014;13(3):330–8.

    Article  CAS  Google Scholar 

  5. Fechter LD. Distribution of manganese in development. Neurotoxicology. 1999;20(2-3):197–201.

    CAS  Google Scholar 

  6. Erikson KM, Thompson K, Aschner J, Aschner M. Manganese neurotoxicity: a focus on the neonate. Pharmacol Ther. 2007;113(2):369–77.

    Article  CAS  Google Scholar 

  7. Dorman DC, Struve MF, Vitarella D, Byerly FL, Goetz J, Miller R. Neurotoxicity of manganese chloride in neonatal and adult CD rats following subchronic (21-day) high-dose oral exposure. J Appl Toxicol. 2000;20(3):179–87.

    Article  CAS  Google Scholar 

  8. Aschner M. Manganese: brain transport and emerging research needs. Environ Health Perspect. 2000;108 Suppl 3:429–32.

    Article  CAS  Google Scholar 

  9. Hernández RB, Farina M, Espósito BP, Souza-Pinto NC, Barbosa Jr F, Suñol C. Mechanisms of manganese-induced neurotoxicity in primary neuronal cultures: the role of manganese speciation and cell type. Toxicol Sci. 2011;124(2):414–23.

    Article  Google Scholar 

  10. Kern CH, Stanwood GD, Smith DR. Preweaning manganese exposure causes hyperactivity, disinhibition, and spatial learning and memory deficits associated with altered dopamine receptor and transporter levels. Synapse. 2010;64(5):363–78.

    Article  CAS  Google Scholar 

  11. Betharia S, Maher TJ. Neurobehavioral effects of lead and manganese individually and in combination in developmentally exposed rats. Neurotoxicology. 2012;33(5):1117–27.

    Article  CAS  Google Scholar 

  12. Erikson KM, Syversen T, Aschner JL, Aschner M. Interactions between excessive manganese exposures and dietary iron-deficiency in neurodegeneration. Environ Toxicol Pharmacol. 2005;19(3):415–21.

    Article  CAS  Google Scholar 

  13. Aschner M, Guilarte TR, Schneider JS, Zheng W. Manganese: recent advances in understanding its transport and neurotoxicity. Toxicol Appl Pharmacol. 2007;221(2):131–47.

    Article  CAS  Google Scholar 

  14. Zoni S, Lucchini RG. Manganese exposure: cognitive, motor and behavioral effects on children: a review of recent findings. Curr Opin Pediatr. 2013;25(2):255–60.

    Article  CAS  Google Scholar 

  15. Haynes EN, Sucharew H, Kuhnell P, Alden J, Barnas M, Wright RO, et al. Manganese exposure and neurocognitive outcomes in rural school-age children: the communities actively researching exposure study (Ohio, USA). Environ Health Perspect. 2015;123(10):1066–71.

    Article  Google Scholar 

  16. Wright RO, Amarasiriwardena C, Woolf AD, Jim R, Bellinger DC. Neuropsychological correlates of hair arsenic, manganese, and cadmium levels in school-age children residing near a hazardous waste site. Neurotoxicology. 2006;27(2):210–6.

    Article  CAS  Google Scholar 

  17. Bouchard M, Laforest F, Vandelac L, Bellinger D, Mergler D. Hair manganese and hyperactive behaviors: pilot study of school-age children exposed through tap water. Environ Health Perspect. 2007;115(1):122–7.

    Article  CAS  Google Scholar 

  18. Wasserman GA, Liu X, Parvez F, Ahsan H, Levy D, Factor-Litvak P, et al. Water manganese exposure and children’s intellectual function in Araihazar, Bangladesh. Environ Health Perspect. 2006;114(1):124–9.

    CAS  Google Scholar 

  19. Aschner M. The transport of manganese across the blood-brain barrier. Neurotoxicology. 2006;27(3):311–4.

    Article  CAS  Google Scholar 

  20. Dorman DC, Struve MF, Wong BA. Brain manganese concentrations in rats following manganese tetroxide inhalation are unaffected by dietary manganese intake. Neurotoxicology. 2002;23(2):185–95.

    Article  CAS  Google Scholar 

  21. Schwartz R, Apgar BJ, Wien EM. Apparent absorption and retention of Ca, Cu, Mg, Mn, and Zn from a diet containing bran. Am J Clin Nutr. 1986;43(3):444–55.

    CAS  Google Scholar 

  22. Dörner K, Dziadzka S, Höhn A, Sievers E, Oldigs HD, Schulz-Lell G, et al. Longitudinal manganese and copper balances in young infants and preterm infants fed on breast-milk and adapted cow’s milk formulas. Br J Nutr. 1989;61(3):559–72.

    Article  Google Scholar 

  23. World Health Organization (WHO). Guidelines for drinking-water quality. 4th ed. Geneva: WHO; 2011. p. 177–226.

    Google Scholar 

  24. Frisbie SH, Mitchell EJ, Dustin H, Maynard DM, Sarkar B. World Health Organization discontinues its drinking-water guideline for manganese. Environ Health Perspect. 2012;120(6):775–8.

    Article  CAS  Google Scholar 

  25. United States Environmental Protection Agency (EPA). Drinking Water Health Advisory for Manganese. (EPA-822-R-04-003) 2004. http://www.epa.gov/safewater/Accessed 12 Dec 2015.

  26. Kim Y, Kim BN, Hong YC, Shin MS, Yoo HJ, Kim JW, et al. Co-exposure to environmental lead and manganese affects the intelligence of school-aged children. Neurotoxicology. 2009;30(4):564–71.

    Article  CAS  Google Scholar 

  27. Claus Henn B, Schnaas L, Ettinger AS, Schwartz J, Lamadrid-Figueroa H, Hernández-Avila M, et al. Associations of early childhood manganese and lead coexposure with neurodevelopment. Environ Health Perspect. 2012;120(1):126–31.

    Article  Google Scholar 

  28. Lin CC, Chen YC, Su FC, Lin CM, Liao HF, Hwang YH, et al. In utero exposure to environmental lead and manganese and neurodevelopment at 2 years of age. Environ Res. 2013;123:52–7.

    Article  CAS  Google Scholar 

  29. Khan K, Factor-Litvak P, Wasserman GA, Liu X, Ahmed E, Parvez F, et al. Manganese exposure from drinking water and children’s classroom behavior in Bangladesh. Environ Health Perspect. 2011;119(10):1501–6.

    Article  CAS  Google Scholar 

  30. Wasserman GA, Liu X, Parvez F, Factor-Litvak P, Ahsan H, Levy D, et al. Arsenic and manganese exposure and children’s intellectual function. Neurotoxicology. 2011;32(4):450–7.

    Article  CAS  Google Scholar 

  31. Riojas-Rodríguez H, Solís-Vivanco R, Schilmann A, Montes S, Rodríguez S, Ríos C, et al. Intellectual function in Mexican children living in a mining area and environmentally exposed to manganese. Environ Health Perspect. 2010;118(10):1465–70.

    Article  Google Scholar 

  32. Bouchard MF, Sauvé S, Barbeau B, Legrand M, Brodeur MÈ, Bouffard T, et al. Intellectual impairment in school-age children exposed to manganese from drinking water. Environ Health Perspect. 2011;119(1):138–43.

    Article  CAS  Google Scholar 

  33. Lucchini RG, Guazzetti S, Zoni S, Donna F, Peter S, Zacco A, et al. Tremor, olfactory and motor changes in Italian adolescents exposed to historical ferro-manganese emission. Neurotoxicology. 2012;33(4):687–96.

    Article  CAS  Google Scholar 

  34. Collipp PJ, Chen SY, Maitinsky S. Manganese in infant formulas and learning disability. Ann Nutr Metab. 1983;27(6):488–94.

    Article  CAS  Google Scholar 

  35. Menezes-Filho JA, Novaes Cde O, Moreira JC, Sarcinelli PN, Mergler D. Elevated manganese and cognitive performance in school-aged children and their mothers. Environ Res. 2011;111(1):156–63.

    Article  CAS  Google Scholar 

  36. Carvalho CF, Menezes-Filho JA, de Matos VP, Bessa JR, Coelho-Santos J, Viana GF, et al. Elevated airborne manganese and low executive function in school-aged children in Brazil. Neurotoxicology. 2014;45:301–8.

    Article  CAS  Google Scholar 

  37. do Nascimento SN, Barth A, Göethel G, Baierle M, Charão MF, Brucker N, et al. Cognitive deficits and ALA-D-inhibition in children exposed to multiple metals. Environ Res. 2015;136:387–95.

    Article  CAS  Google Scholar 

  38. Sun H, Chen W, Wang D, Jin Y, Chen X, Xu Y, et al. Inverse association between intelligence quotient and urinary retinol binding protein in Chinese school-age children with low blood lead levels: results from a cross-sectional investigation. Chemosphere. 2015;128:155–60.

    Article  CAS  Google Scholar 

  39. Mora AM, Arora M, Harley KG, Kogut K, Parra K, Hernández-Bonilla D, et al. Prenatal and postnatal manganese teeth levels and neurodevelopment at 7, 9, and 10.5 years in the CHAMACOS cohort. Environ Int. 2015;84:39–54.

    Article  CAS  Google Scholar 

  40. Bhang SY, Cho SC, Kim JW, Hong YC, Shin MS, Yoo HJ, et al. Relationship between blood manganese levels and children’s attention, cognition, behavior, and academic performance--a nationwide cross-sectional study. Environ Res. 2013;126:9–16.

    Article  CAS  Google Scholar 

  41. Lucchini RG, Zoni S, Guazzetti S, Bontempi E, Micheletti S, Broberg K, et al. Inverse association of intellectual function with very low blood lead but not with manganese exposure in Italian adolescents. Environ Res. 2012;118:65–71.

    Article  CAS  Google Scholar 

  42. Chung SE, Cheong HK, Ha EH, Kim BN, Ha M, Kim Y, et al. Maternal blood manganese and early neurodevelopment: The Mothers and Children’s Environmental Health (MOCEH) Study. Environ Health Perspect. 2015;123(7):717–22.

    Google Scholar 

  43. Takser L, Mergler D, Hellier G, Sahuquillo J, Huel G. Manganese, monoamine metabolite levels at birth, and child psychomotor development. Neurotoxicology. 2003;24(4-5):667–74.

    Article  CAS  Google Scholar 

  44. Claus Henn B, Ettinger AS, Schwartz J, Téllez-Rojo MM, Lamadrid-Figueroa H, Hernández-Avila M, et al. Early postnatal blood manganese levels and children’s neurodevelopment. Epidemiology. 2010;21(4):433–9.

    Article  Google Scholar 

  45. Gunier RB, Arora M, Jerrett M, Bradman A, Harley KG, Mora AM, et al. Manganese in teeth and neurodevelopment in young Mexican-American children. Environ Res. 2015;142:688–95.

    Article  CAS  Google Scholar 

  46. Rink SM, Ardoino G, Queirolo EI, Cicariello D, Mañay N, Kordas K. Associations between hair manganese levels and cognitive, language, and motor development in preschool children from Montevideo, Uruguay. Arch Environ Occup Health. 2014;69(1):46–54.

    Article  CAS  Google Scholar 

  47. Oulhote Y, Mergler D, Barbeau B, Bellinger DC, Bouffard T, Brodeur MÈ, et al. Neurobehavioral function in school-age children exposed to manganese in drinking water. Environ Health Perspect. 2014;122(12):1343–50.

    CAS  Google Scholar 

  48. Hernández-Bonilla D, Schilmann A, Montes S, Rodríguez-Agudelo Y, Rodríguez-Dozal S, Solís-Vivanco R, et al. Environmental exposure to manganese and motor function of children in Mexico. Neurotoxicology. 2011;32(5):615–21.

    Article  Google Scholar 

  49. Parvez F, Wasserman GA, Factor-Litvak P, Liu X, Slavkovich V, Siddique AB, et al. Arsenic exposure and motor function among children in Bangladesh. Environ Health Perspect. 2011;119(11):1665–70.

    Article  CAS  Google Scholar 

  50. Ode A, Rylander L, Gustafsson P, Lundh T, Källén K, Olofsson P, et al. Manganese and selenium concentrations in umbilical cord serum and attention deficit hyperactivity disorder in childhood. Environ Res. 2015;137:373–81.

    Article  CAS  Google Scholar 

  51. Shin DW, Kim EJ, Lim SW, Shin YC, Oh KS, Kim EJ. Association of hair manganese level with symptoms in attention-deficit/hyperactivity disorder. Psychiatry Investig. 2015;12(1):66–72.

    Article  CAS  Google Scholar 

  52. Ericson JE, Crinella FM, Clarke-Stewart KA, Allhusen VD, Chan T, Robertson RT. Prenatal manganese levels linked to childhood behavioral disinhibition. Neurotoxicol Teratol. 2007;29(2):181–7.

    Article  CAS  Google Scholar 

  53. Menezes-Filho JA, de Carvalho-Vivas CF, Viana GF, Ferreira JR, Nunes LS, Mergler D, et al. Elevated manganese exposure and school-aged children’s behavior: a gender-stratified analysis. Neurotoxicology. 2014;45:293–300.

    Article  CAS  Google Scholar 

  54. Torres-Agustín R, Rodríguez-Agudelo Y, Schilmann A, Solís-Vivanco R, Montes S, Riojas-Rodríguez H, et al. Effect of environmental manganese exposure on verbal learning and memory in Mexican children. Environ Res. 2013;121:39–44.

    Article  Google Scholar 

  55. Wechsler D. Manual for the WISC-III. San Antonio: Psychological Corporation; 1991.

    Google Scholar 

  56. Bayley N. Bayley Scales of Infant Development. San Antonio: Psychological Corporation; 1993.

    Google Scholar 

  57. Yang X, Bao Y, Fu H, Li L, Ren T, Yu X. Selenium protects neonates against neurotoxicity from prenatal exposure to manganese. PLoS One. 2014;9(1):e86611.

    Article  Google Scholar 

  58. Khan K, Wasserman GA, Liu X, Ahmed E, Parvez F, Slavkovich V, et al. Manganese exposure from drinking water and children’s academic achievement. Neurotoxicology. 2012;33(1):91–7.

    Article  CAS  Google Scholar 

  59. Iannilli E, Gasparotti R, Hummel T, Zoni S, Benedetti C, Fedrighi C, et al. Effects of manganese exposure on olfactory functions in teenagers: a pilot study. PLoS One. 2016;11(1):e0144783.

    Article  Google Scholar 

  60. Eastman RR, Jursa TP, Benedetti C, Lucchini RG, Smith DR. Hair as a biomarker of environmental manganese exposure. Environ Sci Technol. 2013;47(3):1629–37.

    CAS  Google Scholar 

  61. Aschner M, Erikson KM, Dorman DC. Manganese dosimetry: species differences and implications for neurotoxicity. Crit Rev Toxicol. 2005;35(1):1–32.

    Article  CAS  Google Scholar 

  62. Barlow PJ. A pilot study on the metal levels in the hair of hyperactive children. Med Hypotheses. 1983;11(3):309–18.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study was funded through the Consortium on Law and Values in Health, Environment and the Life Sciences, University of Minnesota. The Consortium was not involved in any phase of the research for or writing of this paper.

Availability of data and materials

Not applicable (i.e., this paper is a review of the literature).

Authors’ contributions

DC conducted the original literature search, critically reviewed the human studies, drafted the paper and developed the tables. PM conceived of the paper, critically reviewed the human studies, revised drafts and supervised DC’s work. RR drafted manuscript sections on the preclinical studies on the effects of excess Mn exposure on neurodevelopment and the pathways involved and reviewed all drafts of the paper. LH critically reviewed studies assessing Mn intake via diet, water and infant feeding practices and reviewed all drafts of the paper. MG helped conceptualize the paper and reviewed all drafts of the paper. IS helped conceptualize the paper, reviewed all drafts of the paper and oversaw review of the scientific content. All authors read and approved the final manuscript.

Authors’ information

1 DC is a student at the School of Medicine, University of Minnesota.

2 PM is the Bond Professor of Environmental and Occupational Health Policy in the School of Public Health, University of Minnesota.

3 RR is an Associate Professor in Pediatrics, at the School of Medicine and a clinical neonatologist at the Neonatal Intensive Care Unit of the University of Minnesota Masonic Children’s Hospital.

4 LH is a Professor and Director for the Nutrition Coordinating Center in the Division of Epidemiology and Community Health, School of Public Health, University of Minnesota.

5 MG is the Martin Lenz Harrison Land Grant Chair, Professor in Pediatrics and Institute of Child Development, Director of the Division of Neonatology and Director of the Center for Neurobehavioral Development in the School of Medicine, University of Minnesota.

6 IS an Associate Professor in the Division of Environmental Sciences, School of Public Health and the Masonic Cancer Center at the University of Minnesota.

Competing interests

The authors declare they have no competing interests.

Consent for publication

Not Applicable.

Ethics approval and consent to participate

Not Applicable.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Patricia M. McGovern.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Coetzee, D.J., McGovern, P.M., Rao, R. et al. Measuring the impact of manganese exposure on children’s neurodevelopment: advances and research gaps in biomarker-based approaches. Environ Health 15, 91 (2016). https://doi.org/10.1186/s12940-016-0174-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12940-016-0174-4

Keywords