Impact of exposure measurement error in air pollution epidemiology: effect of error type in time-series studies
© Goldman et al; licensee BioMed Central Ltd. 2011
Received: 3 January 2011
Accepted: 22 June 2011
Published: 22 June 2011
Two distinctly different types of measurement error are Berkson and classical. Impacts of measurement error in epidemiologic studies of ambient air pollution are expected to depend on error type. We characterize measurement error due to instrument imprecision and spatial variability as multiplicative (i.e. additive on the log scale) and model it over a range of error types to assess impacts on risk ratio estimates both on a per measurement unit basis and on a per interquartile range (IQR) basis in a time-series study in Atlanta.
Daily measures of twelve ambient air pollutants were analyzed: NO2, NOx, O3, SO2, CO, PM10 mass, PM2.5 mass, and PM2.5 components sulfate, nitrate, ammonium, elemental carbon and organic carbon. Semivariogram analysis was applied to assess spatial variability. Error due to this spatial variability was added to a reference pollutant time-series on the log scale using Monte Carlo simulations. Each of these time-series was exponentiated and introduced to a Poisson generalized linear model of cardiovascular disease emergency department visits.
Measurement error resulted in reduced statistical significance for the risk ratio estimates for all amounts (corresponding to different pollutants) and types of error. When modelled as classical-type error, risk ratios were attenuated, particularly for primary air pollutants, with average attenuation in risk ratios on a per unit of measurement basis ranging from 18% to 92% and on an IQR basis ranging from 18% to 86%. When modelled as Berkson-type error, risk ratios per unit of measurement were biased away from the null hypothesis by 2% to 31%, whereas risk ratios per IQR were attenuated (i.e. biased toward the null) by 5% to 34%. For CO modelled error amount, a range of error types were simulated and effects on risk ratio bias and significance were observed.
For multiplicative error, both the amount and type of measurement error impact health effect estimates in air pollution epidemiology. By modelling instrument imprecision and spatial variability as different error types, we estimate direction and magnitude of the effects of error over a range of error types.
The issue of measurement error is unavoidable in epidemiologic studies of air pollution . Although methods for dealing with this measurement error have been proposed [2, 3] and applied to air pollution epidemiology specifically [4, 5], the issue remains a central concern in the field . Because large-scale time-series studies often use single central monitoring sites to characterize community exposure to ambient concentrations , uncertainties arise regarding the extent to which these monitors are representative of exposure. Zeger et al.  identify three components of measurement error: (1) the difference between individual exposures and average personal exposure, (2) the difference between average personal exposure and ambient levels, and (3) the difference between measured and true ambient concentrations. While the former two components of error can have a sizeable impact on epidemiologic findings that address etiologic questions of health effects and personal exposure, it is the third component that is particularly relevant in time-series studies that address questions of the health benefits of ambient regulation .
Prior studies have suggested that the impact of measurement error on time-series health studies differs depending upon the type of error introduced [8, 10, 11]. Two distinctly different types of error have been identified. One type is classical error, in which measurements, Z t , vary randomly about true concentrations, ; this can be considered the case for instrument error associated with ambient monitors. That is, instrument error is independent of the true ambient level, such that . Moreover, the variation in the measurements, Z t , is expected to be greater than the variation in the true values, . Therefore, classical error is expected to attenuate the effect estimate in time-series epidemiologic studies. In contrast, under a Berkson error framework, the true ambient, , varies randomly about the measurement, Z t . This might be the case, for example, of a measured population average over the study area with true individual ambient levels varying randomly about this population average measurement. In this case, measurement error is independent of the measured population average ambient; that is, . Furthermore, the measurement, Z t , is less variable than the true ambient level, . A purely Berkson error is expected to yield an unbiased effect estimate, provided that the true dose-response is linear .
Several studies have investigated the impact of error type on regression models. The simultaneous impact of classical and Berkson errors in a parametric regression estimating radon exposure has been investigated  and error type has been assessed in a semiparametric Bayesian setting looking at exposure to radiation from nuclear testing [13, 14]; however, no study to date has comprehensively assessed the impact of error type across multiple pollutants for instrument imprecision and spatial variability in a time-series context.
Error type depends on the relationship between the distribution of measurements and the distribution of true values. Because true relevant exposure in environmental epidemiologic studies is not known exactly, determination of error type is challenging; thus, here we examine the impact of error modelled as two distinctly different types: classical and Berkson. First, we examine monitor data to assess whether error is better modelled on a logged or unlogged basis. Typically, researchers investigating error type have added error on an unlogged basis (e.g. [8, 11]); however, air pollution data are more often lognormal due to atmospheric dynamics and concentration levels that are never less than zero. It is plausible that true ambient exposures are distributed lognormally about a population average as well; therefore, measurement error may be best described as additive error on the log scale. We investigate the combined error from two sources that have been previously identified as relevant in time-series studies: (1) instrument precision error and (2) error due to spatial variability . We limit our scope to ambient levels of pollutants measured in accordance with regulatory specifications, disregarding spatial microscale variability, such as near roadway concentrations, as well as temporal microscale variability, such as that associated with meteorological events on sub-hour time scales. Here, building on a previously developed model for the amount of error associated with selected ambient air pollutants , we quantitatively assess the effect of error type on the impacts of measurement error on epidemiologic results from an ongoing study of air pollution and emergency department visits in Atlanta.
Air Pollutant Data
To assess error due to instrument imprecision and spatial variability of ambient concentrations, 1999-2004 datasets were used for the 12 pollutants with data completeness for this time period (2,192 days) ranging from 82% to 97%. Data from collocated instruments were used to characterize instrument precision error. Measurement methods and data quality are discussed in detail in our prior work . Distributions of all air pollutant measures more closely approximate lognormal distributions than normal distributions (, see Additional file 1, Table S1); therefore, additive error was characterized and modeled on a log concentration basis so that simulations with error added to a base case time-series would also have lognormal distributions.
Measurement Error Model
Here, ε χt is the modeled error in for day t, N t is a random number with distribution ~N(0,1) and σ err is the standard deviation of error added, a parameter derived from the population-weighted semivariance to capture the amount of error present for each pollutant, as described in the next subsection. Short-term temporal autocorrelation observed in the differences between measurements was modeled using a three-day running average of random numbers for Nt .
Here, χ t is the standardized simulated time-series (on the log scale) with type C error added and normal distribution . In this case of type C error, ε χt and are independent (i.e. ). For type B error, ε χt and χ t are independent (i.e. E[R(ε χt , χ t )] = 0) and . It can be shown (see Additional file 2, eqs. S1-S6) that simulations with type B error can be generated from the true time-series by eq. 4.
For both error types, the simulated time-series (Z t ) and true time-series ( ) have the same log means (μ InZ = μ InZ* ). For classical-like error (type C), the log standard deviation is greater for the simulated time-series than the true time-series (σ InZ > σ InZ* ) because the simulated values are scattered about the true values. For Berkson-like error (type B), the log standard deviation is less for the simulated time-series than the true time-series (σ InZ < σ InZ* ) because the true values are scattered about the simulated values.
Thus, γ' represents the spatial semivariance scaled to a quantity indicative of the range of exposures over which health risk is being assessed; it is unitless and allows for comparison across pollutants. A scaled semivariance value of 0 corresponds to perfectly correlated observations (R = 1) and a value of 1 corresponds to perfectly uncorrelated observations (R = 0).
Population-weighted scaled semivariances, , Pearson correlation coefficients, , and model parameters used in the Monte Carlo simulations to simulate amount of error (σ err ) and error type (σ InZ /σ InZ* )
σ InZ /σ InZ*
σ InZ /σ InZ*
1-hr max NO2
1-hr max NOx
8-hr max O3
1-hr max SO2
1-hr max CO
Values of σ err and σ InZ /σ InZ* used here can be found in Table 1.
Sets of 1000 simulated time-series with instrument and spatial error added for each pollutant for the scenarios of C and B error types were produced for the six-year period 1999-2004. In addition, simulations of CO measurement error only were generated for a range of error types with σ InZ /σ InZ* values between error types C and B. Scatterplots demonstrate that C and B error types defined on a log basis (i.e. InZ - InZ*) are independent of InZ* and InZ, respectively (see Additional file 3, Figure S1).
Distribution of Measurement Error Simulations
Impact of Error on Health Risk Assessment
Summarized epidemiologic model results with the magnitude of error representative of error associated with using a population-weighted average for each pollutant added to the base case (RR* = 1.0139, 95% CI = 1.0078-1.0201, p-value = 0.000009, IQR = 1.00 ppm)
RR per ppm (95% CI)
RR per IQR (95% CI)
Error Type C simulations
1-hr max NO2
1-hr max NOx
8-hr max O3
1-hr max SO2
1-hr max CO
Error Type B simulations
1-hr max NO2
1-hr max NOx
8-hr max O3
1-hr max SO2
1-hr max CO
In this study, in which quantification of error is based on the variability between monitors, error due to spatial variation is much greater than error due to instrument imprecision, particularly for primary air pollutants . Conceptually, therefore, we speculate that this error is more likely of the Berkson type, with true values varying randomly about a population-weighted average represented by the base case. If spatial error is best described by the Berkson-like type defined on a log basis (our error type B) and the mean of the measurements is the same mean as the true values, we estimate there to be a 24% to 34% attenuation in RR per IQR estimates (Figure 4, right panel), and a 19% to 31% bias away from the null in RR estimates on a per unit of measurement basis (Figure 4, left panel), for the primary pollutants studied (SO2, NO2/NOx, CO, and EC) when using a population-weighted average as the exposure metric. For the secondary pollutants and pollutants of mixed origin (O3, SO4, NO3, NH4, PM2.5, OC, and PM10), we estimate a 5% to 15% attenuation in RR per IQR estimates and a 2% to 9% bias away from the null in RR estimates on a per unit of measurement basis. We are currently investigating different methods for estimating actual error type based on simulated pollutant fields trained to have all of the characteristics, including the pattern of spatial autocorrelation, expected of true pollutant fields.
This study addresses error between measured and true ambient concentrations. Our results are consistent with previous finding that suggest that Berkson error, as defined on an unlogged scale (additive), produces no bias in the effect estimate [8, 11] as shown in Figure 5; however, Berkson-like error defined on a log basis (multiplicative) can lead to risk ratio estimates per unit increase that are biased away from the null (although with a reduction in significance). Thus, the direction and magnitude of the bias are functions of error type. With the multiplicative error structure used here in conjunction with a linear dose response, large "true" values of air pollution would likely be underestimated, resulting in an overestimation of pollution health effects. We have shown how multiple air pollution measurements over space can be used to quantify the amount of error and provide a strategy for evaluating impacts of different types of this error. The results suggest that estimating impacts of measurement error on health risk assessment are particularly important when comparing results across primary and secondary pollutants as the corresponding error will vary widely in both amount and type depending on the degree of spatial variability. These results are suggestive of error impacts one would have from time-series studies in which a single measure, such as the population-weighted average, is used to characterize an urban or regional population exposure. The methodology used here can be applied to other study areas to quantify this type of measurement error and quantify its impacts on health risk estimates.
Health risk estimates of exposure to ambient air pollution are impacted by both the amount and the type of measurement error present, and these impacts vary substantially across pollutants. By modeling combined instrument imprecision and spatial variability over a range of error types, we are able to estimate a range of effects of these sources of measurement error, which are likely a mixture of both classical and Berkson error types. This study demonstrates the potential impact of measurement error in an air pollution epidemiology time-series study and how this impact depends on error type and amount.
List of Abbreviations
US EPA's Air Quality System
the Southeastern Aerosol Research and Characterization Study
Assessment of Spatial Aerosol Composition in Atlanta
The authors acknowledge financial support from the following grants: NIEHS R01ES111294, NIEHS K01ES019877, EPRI EP-P277231/C13172, EPA STAR R89291301, EPA STAR R83362601, EPA STAR R83386601, and EPA STAR RD83479901. The contents of this publication are solely the responsibility of the grantee and do not necessarily represent the official views of the USEPA. Further, USEPA does not endorse the purchase of any commercial products or services mentioned in the publication.
- Sarnat JA, Wilson WE, Strand M, Brook J, Wyzga R, Lumley T: Panel discussion review: session one - exposure assessment and related errors in air pollution epidemiologic studies. Journal of Exposure Science and Environmental Epidemiology. 2007, 17: S75-S82.View ArticleGoogle Scholar
- Carroll RJ, Ruppert D, Stefanski L: Measurement Error in Nonlinear Models. 1995, London: Chapman & HallView ArticleGoogle Scholar
- Fuller WA: Measurement Error Models. 1987, Chichester: WileyView ArticleGoogle Scholar
- Dominici F, Zeger SL, Samet JM: A measurement error model for time-series studies of air pollution and mortality. Biostat. 2000, 1: 157-175. 10.1093/biostatistics/1.2.157.View ArticleGoogle Scholar
- Strand M, Vedal S, Rodes C, Dutton SJ, Gelfand EW, Rabinovitch N: Estimating effects of ambient PM2.5 exposure on health using PM2.5 component measurements and regression calibration. Journal of Exposure Science and Environmental Epidemiology. 2006, 16: 30-38. 10.1038/sj.jea.7500434.View ArticleGoogle Scholar
- Ren C, Tong S: Health effects of ambient air pollution - recent research development and contemporary methodological challenges. Environmental Health. 2008, 7-Google Scholar
- Wilson JG, Kingham S, Pearce J, Sturman AP: A review of intraurban variations in particulate air pollution: Implications for epidemiological research. Atmospheric Environment. 2005, 39: 6444-6462. 10.1016/j.atmosenv.2005.07.030.View ArticleGoogle Scholar
- Zeger SL, Thomas D, Dominici F, Samet JM, Schwartz J, Dockery D, Cohen A: Exposure measurement error in time-series studies of air pollution: concepts and consequences. Environmental Health Perspectives. 2000, 108: 419-426. 10.1289/ehp.00108419.View ArticleGoogle Scholar
- Carrothers TJ, Evans JS: Assessing the impact of differential measurement error on estimates of fine particle mortality. Journal of the Air & Waste Management Association. 2000, 50: 65-74.View ArticleGoogle Scholar
- Sheppard L, Slaughter JC, Schildcrout J, Liu LJS, Lumley T: Exposure and measurement contributions to estimates of acute air pollution effects. Journal of Exposure Analysis and Environmental Epidemiology. 2005, 15: 366-376. 10.1038/sj.jea.7500413.View ArticleGoogle Scholar
- Armstrong BG: Effect of measurement error on epidemiological studies of environmental and occupational exposures. Occupational and Environmental Medicine. 1998, 55: 651-656. 10.1136/oem.55.10.651.View ArticleGoogle Scholar
- Reeves GK, Cox DR, Darby SC, Whitley E: Some aspects of measurement error in explanatory variables for continuous and binary regression models. Statistics in Medicine. 1998, 17: 2157-2177. 10.1002/(SICI)1097-0258(19981015)17:19<2157::AID-SIM916>3.0.CO;2-F.View ArticleGoogle Scholar
- Li YH, Guolo A, Hoffman FO, Carroll RJ: Shared uncertainty in measurement error problems, with application to Nevada test site fallout data. Biometrics. 2007, 63: 1226-1236.View ArticleGoogle Scholar
- Mallick B, Hoffman FO, Carroll RJ: Semiparametric regression modeling with mixtures of Berkson and classical error, with application to fallout from the Nevada test site. Biometrics. 2002, 58: 13-20. 10.1111/j.0006-341X.2002.00013.x.View ArticleGoogle Scholar
- Goldman GT, Mulholland JA, Russell AG, Srivastava A, Strickland MJ, Klein M, Waller LA, Tolbert PE, Edgerton ES: Ambient Air Pollutant Measurement Error: Characterization and Impacts in a Time-Series Epidemiologic Study in Atlanta. Environmental Science & Technology. 2010, 44: 7692-7698. 10.1021/es101386r.View ArticleGoogle Scholar
- Hansen DA, Edgerton ES, Hartsell BE, Jansen JJ, Kandasamy N, Hidy GM, Blanchard CL: The southeastern aerosol research and characterization study: Part 1-overview. Journal of the Air & Waste Management Association. 2003, 53: 1460-1471.View ArticleGoogle Scholar
- Solomon PA, Chameides W, Weber R, Middlebrook A, Kiang CS, Russell AG, Butler A, Turpin B, Mikel D, Scheffe R, Cowling E, Edgerton E, St John J, Jansen J, McMurry P, Hering S, Bahadori T: Overview of the 1999 Atlanta Supersite Project. Journal of Geophysical Research-Atmospheres. 2003, 108-Google Scholar
- Butler AJ, Andrew MS, Russell AG: Daily sampling of PM2.5 in Atlanta: results of the first year of the assessment of spatial aerosol composition in Atlanta study. Journal of Geophysical Research-Atmospheres. 2003, 108-Google Scholar
- Hinkley D: On quick choice of power transformation. Applied Statistics. 1977, 26: 67-69. 10.2307/2346869.View ArticleGoogle Scholar
- Casado LS, Rouhani S, Cardelino CA, Ferrier AJ: Geostatistical Analysis and Visualization of Hourly Ozone Data. Atmospheric Environment. 1994, 28: 2105-2118. 10.1016/1352-2310(94)90477-4.View ArticleGoogle Scholar
- Wade KS, Mulholland JA, Marmur A, Russell AG, Hartsell B, Edgerton E, Klein M, Waller L, Peel JL, Tolbert PE: Effects of instrument precision and spatial variability on the assessment of the temporal variation of ambient air pollution in Atlanta, Georgia. Journal of the Air & Waste Management Association. 2006, 56: 876-888.View ArticleGoogle Scholar
- Metzger KB, Tolbert PE, Klein M, Peel JL, Flanders WD, Todd K, Mulholland JA, Ryan PB, Frumkin H: Ambient air pollution and cardiovascular emergency department visits. Epidemiology. 2004, 15: 46-56. 10.1097/01.EDE.0000101748.28283.97.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.