An assessment of survey measures used across key epidemiologic studies of United States Gulf War I Era Veterans

  • Rebecca B McNeil1Email author,

    Affiliated with

    • Catherine M Thomas1,

      Affiliated with

      • Steven S Coughlin2,

        Affiliated with

        • Elizabeth Hauser1, 3,

          Affiliated with

          • Grant D Huang4,

            Affiliated with

            • Karen M Goldstein5,

              Affiliated with

              • Marcus R Johnson6,

                Affiliated with

                • Tyra Dunn-Thomas7 and

                  Affiliated with

                  • Dawn T Provenzale1, 5, 3

                    Affiliated with

                    Environmental Health201312:4

                    DOI: 10.1186/1476-069X-12-4

                    Received: 21 May 2012

                    Accepted: 24 December 2012

                    Published: 9 January 2013

                    Abstract

                    Over the past two decades, 12 large epidemiologic studies and 2 registries have focused on U.S. veterans of the 1990–1991 Gulf War Era. We conducted a review of these studies’ research tools to identify existing gaps and overlaps of efforts to date, and to advance development of the next generation of Gulf War Era survey tools. Overall, we found that many of the studies used similar instruments. Questions regarding exposures were more similar across studies than other domains, while neurocognitive and psychological tools were the most variable. Many studies focused on self-reported survey results, with a range of validation practices. However, physical exams, biomedical assessments, and specimen storage were not common. This review suggests that while research may be able to pool data from past surveys, future surveys need to consider how their design can yield data comparable with previous surveys. Additionally, data that incorporate recent technologies in specimen and genetic analyses would greatly enhance such survey data. When combined with existing data on deployment-related exposures and post-deployment health conditions, longitudinal follow-up of existing studies within this collaborative framework could represent an important step toward improving the health of veterans.

                    Keywords

                    Persian Gulf War 1991 Survey methods Blood banks

                    Background

                    Approximately 697,000 members of the U.S. Armed Forces were deployed to Southwest Asia in support of the 1990–1991 Gulf War. More than 4 million remaining troops were deployed elsewhere or not deployed. Elevated rates of measurable symptomatology and illness in Gulf War Era veterans have been well documented [14], with the excess burden of symptom prevalence estimated to be as high as 25-30% among deployed Gulf War troops, while their disease-related mortality remains equivalent to, or even lower than, that of the general population [16]. Many epidemiologic studies have been performed to understand the physical and psychological symptoms observed in and reported by veterans who served during this conflict era [1, 2, 4, 717]. However, a clear pathologic explanation or overarching diagnosis has remained elusive. While studies initially focused on post-deployment mental health, the number of reports of medically unexplained chronic multisymptom illness in this veteran population increased during the 1990s [18, 19]. This attracted the attention of researchers in several disciplines, including immunology, toxicology, neurology, occupational (military) health, epidemiology, and environmental health. Consequently, assessments in research expanded to address physical health, symptoms, and deployment-related exposures. Relative to epidemiological research in most other clinical areas, Gulf War-related studies have involved a wide diversity of outcome measures, risk factors, cohort characteristics, and methods. Novel strategies will be required to unify and harmonize the approach to both clinical care and research for this population in the setting of such broad clinical outcomes and diverse risk factors.

                    A deeper understanding of the tools used by past studies will enable critical evaluation and direction for future efforts. The Institute of Medicine and the Research Advisory Committee on Gulf War Veterans’ Illnesses have both completed comprehensive reviews of Gulf War research [2023]. However, these focused primarily on the findings of the many studies conducted thus far, and not the research tools used to gather the studies’ data. There has not been a recent systematic review of the broad range of assessments used in Gulf War research. In this manuscript, we describe and review the research tools used by the fourteen major Gulf War Era studies to date. It is our intent that this information will assist ongoing research efforts through the following two focused purposes: to provide an overview of the assessment tools used in past studies, and to discuss how future Gulf War research may benefit from consideration of commonalities and gaps in the epidemiologic research.

                    Inclusion and exclusion criteria

                    We used three primary criteria to select candidate studies for inclusion in this review. First, we included epidemiologic studies of U.S. veterans that were considered “major cohorts” by the Institute of Medicine’s (IOM) 2009 Gulf War and Health review [22], according to their specified criteria for methodologic rigor, health outcomes assessment, medical evaluations, and use of laboratory testing. Second, we included two Gulf War veteran registries established by the Departments of Veterans Affairs and Defense; while these necessarily suffer from the limitations common to registries, such as non-generalizability and absence of a control group, they jointly comprise a vast repository of standardized data that may be useful in future carefully constructed research efforts. Finally, we considered studies reported since the IOM’s 2009 review, and ongoing studies.

                    We excluded studies of non-U.S. veterans and clinical studies or trials related to specific disease entities. Substudies of primary studies were included under the umbrella of their original study.

                    These criteria resulted in a total of twelve epidemiologic studies and two large government registries. The specific studies are: Devens Cohort Study (Devens, originally the Ft. Devens Operation Desert Storm Reunion Survey) [7, 24], New Orleans study [8], National Survey of Gulf War Era Veterans and Their Families (National Health Survey) [10], Pennsylvania-Hawaii study [17], Iowa study [2], Air National Guard study [4], Navy Seabees study [12], Oregon-Washington study [13], cross-sectional Kansas study (Kansas I) [1], case–control Kansas study (Kansas II) [14], Millennium Cohort Study [15], Military Health Survey [16], and the VA Persian Gulf War Health Examination Registry (VA Registry) [9] and DoD Comprehensive Clinical Evaluation Program Registry (DoD Registry) [11]. A tabulation of study characteristics is contained in Table 1. Demographic characteristics of the study participants are summarized in Table 2.
                    Table 1

                    Design characteristics of major studies of U.S. veterans of the first Gulf War

                    Study

                    Inception

                    Design

                    Administration

                    Sample

                    Focus

                    Survey Pilot Testing

                    References

                    Devens Cohort Study

                    1991

                    Longitudinal cohort, substudies

                    In-person surveys and evaluations, mailed surveys

                    84 units returning from GW through Ft. Devens, MA

                    Psychological health, domestic and military exposures

                    Unable to determinea

                    [7, 2527]

                    New Orleans

                    1991

                    Longitudinal cohort, substudies

                    In-person surveys and evaluations, mailed surveys, phone

                    Units based in Louisiana

                    Psychological and physical health, stressors

                    Unable to determinea

                    [8]

                    VA Registry

                    1992

                    Registry, substudies

                    In-person survey and two-stage health evaluation

                    Self-nominated veterans of 1st GW and OIF

                    Physical health, military exposures

                    Unable to determinea

                    [28]

                    Nat’l Health Survey

                    1992

                    Longitudinal cohort, substudies

                    In-person evaluations, mailed surveys, phone surveys

                    Stratified random sample of GW-deployed and non-deployed

                    Physical and psychological health, military exposures

                    Reviewed by OMB

                    [10, 29]

                    PA & HI

                    1993

                    Cross-sectional

                    In-person distribution

                    All active duty, National Guard, and reserve units in PA and HI

                    Physical and psychological health

                    Unable to determinea; similar survey used in previous studies

                    [17]

                    DoD Registry

                    1994

                    Registry, substudies

                    In-person survey and two-stage health evaluation

                    Self-nominated veterans of 1st GW and OIF

                    Physical health, military exposures

                    Unable to determinea

                    [30]

                    Iowa

                    1995

                    Cross-sectional, substudies

                    In-person health evaluation, phone survey

                    Stratified random sample of GW regular military and NG/Reserve from Iowa

                    Physical and psychological health, functional status, military exposures

                    Tested in 24 veterans from the random sample, and 3 non-sampled members of the military

                    [2]

                    Seabees

                    1997

                    Cross-sectional

                    Mailed survey, phone survey

                    Construction Battalion (CB) members who served for 30+ days active duty during GW

                    Current and past health issues, exposures, behavioral risk factors

                    Tested in Navy personnel, reviewed by OMB and DoD, test-retest reliability on earlier survey

                    [12, 31]

                    OR & WA

                    1995

                    Cross-sectional, case–control

                    In-person health evaluation, mailed survey, phone survey

                    Random sample of veterans from OR or WA who were deployed to SW Asia between 8/1/90 and 7/31/91

                    Physical and psychological health, military exposures

                    Test-retest reliability survey (exposures) in 305 case–control study participants

                    [32]

                    Air Nat’l Guard

                    1994

                    Cross-sectional, substudies

                    In-person health evaluation and surveys

                    GW veterans from a PA-based Air National Guard unit, two AF reserve units (PA, FL), and an active duty AF unit (FL)

                    Physical health, risk factors for illness

                    Unable to determinea

                    [4, 33]

                    Kansas I

                    1998

                    Cross-sectional

                    Phone survey

                    KS veterans or reserve members who served on active duty between 8/90 and 7/91

                    Physical health, military exposures

                    Health questions tested in MO veterans

                    [1]

                    Kansas II

                    2000

                    Case–control

                    In-person survey and blood draw, phone

                    Veteran KS/MO residents of Kansas City metropolis, who deployed to GW between 8/1/1990 and 7/31/1991, with Gulf War Illness (cases) or not ill (controls)

                    Gulf War Illness, military exposures

                    Tested in veterans residing outside of sampling frame

                    [14]

                    Millennium Cohort

                    2000

                    Longitudinal cohort

                    Mailed survey, web survey

                    Stratified random sample of regular active duty, NG, and reserve

                    Physical health (chronic illness), risk factors

                    Focus groups and 1000-participant pilot survey (total 2564 tests)

                    [34]

                    Military Health Survey

                    2007

                    Cross-sectional, substudies

                    In-person blood sample, phone survey

                    Stratified random sample of deployed and deployable nondeployed between 8/2/90 and 7/31/91

                    Physical health, military exposures, behavioral risk factors

                    200-veteran sample pilot tested survey content, interview procedures

                    [16, 35, 36]

                    Abbreviations: AF Air Force, DoD Department of Defense, FL Florida, GW Gulf War, HI Hawaii, KS Kansas, MA Massachusetts, MO Missouri, NG National Guard, OIF Operation Iraqi Freedom, OMB Office of Management and Budget, OR Oregon, PA Pennsylvania, SW southwest, VA Department of Veterans Affairs, WA Washington.

                    aWe were unable to determine from published sources whether the investigators conducted pilot testing or early validation testing of their survey instruments; however, this was likely done in many instances.

                    Table 2

                    Characteristics of participants in major studies of U.S. veterans of the first Gulf War a

                     

                    Devens Cohort Study

                    New Orleans

                    VA Registry

                    Nat’l Health Survey

                    PA & HI

                    DoD Registry

                    Iowa

                    Seabees

                    OR & WA

                    Air Nat’l Guard

                    Kansas I

                    Kansas II

                    Millennium Cohortc

                    Military Health Survey

                    Total

                    Sample size

                     

                    2,949b

                    1,520d

                    66,227

                    20,917

                    4,344

                    54,244

                    3,695

                    11,868

                    1,119

                    3,723

                    2,030

                    304

                    45,372

                    8,020

                    226,332

                    Deployment status

                    GW

                    2,345

                    1,520

                    66,227

                    11,441

                    1,524

                    46,625

                    1,896

                    3,831

                    1,119

                    1,154

                    1,548

                    304

                    9,248

                    5,699

                    154,481

                     

                    Non-GW

                       

                    9,476

                    2,727

                    4,888

                    1,799

                    8,037

                     

                    2,569

                    482

                     

                    36,124

                    1, 192

                    67,294

                    Age, years

                    <25

                          

                    1,895

                           

                    1,895

                     

                    17-21

                      

                    15,008

                           

                    386

                       

                    15,394

                     

                    22-25

                      

                    14,887

                           

                    426

                       

                    15,313

                     

                    >25

                          

                    1,800

                           

                    1,800

                     

                    26-31

                      

                    14,390

                             

                    5,761

                     

                    20,151

                     

                    >31

                      

                    21,947

                             

                    41,471

                     

                    63,418

                     

                    26-33

                              

                    528

                     

                    10,118

                     

                    10,646

                     

                    >34

                              

                    690

                     

                    35,254

                     

                    35,944

                     

                    Mean

                    30.2

                    29

                     

                    31

                     

                    32.5

                     

                    30.6

                    26.4

                    35

                      

                    39.1

                      

                    Date of age

                     

                    1991

                    deploy

                    deploy

                    1991

                      

                    8/1990

                    1990

                    deploy

                    1995

                    1/1991

                     

                    2000

                      

                    Gender

                    Male

                    2,137

                    1,307

                    59,697

                    16,715

                     

                    47,301

                    3,360

                    11,334

                    962

                    3,202

                    1,766

                    282

                    35,460

                     

                    183,523

                     

                    Female

                    208

                    213

                    6,530

                    4,202

                     

                    6,943

                    335

                    534

                    157

                    521

                    264

                    22

                    9,895

                     

                    29,824

                    Race

                    White

                    1,975

                    867

                    43,182

                    15,550

                    3,078

                    27,936

                    3,543

                    10,432

                    1,041

                    3,202

                    1,786

                    265

                    31,988

                     

                    144,845

                     

                    Black

                    176

                    456

                    16,527

                    3,828

                     

                    15,893

                     

                    843

                      

                    162

                    26

                    6,501

                     

                    44,412

                     

                    Hispanic

                    84

                    182

                    1,752

                      

                    2,712

                        

                    61

                    15

                    2,319

                     

                    7,125

                     

                    Other

                    110

                    15

                    4,766

                    1,539

                    977

                    7,648

                    152

                    593

                    78

                    521

                    41

                    8

                    4,537

                     

                    20,985

                    Service

                    Active

                    265

                     

                    42,590

                    8,082

                    2,291

                    44,697

                    1,953

                    10,266

                    772

                    1,680

                    914

                     

                    21,516

                     

                    135,026

                     

                    NG/Res

                      

                    23,637

                       

                    1,742

                     

                    347

                     

                    1,116

                       

                    26,842

                     

                    NG

                    1,494

                      

                    5,759

                         

                    1,205

                      

                    10,863

                     

                    19,321

                     

                    Reserve

                    586

                      

                    7,076

                    1,714

                    6,455

                     

                    1,602

                     

                    838

                      

                    12,954

                     

                    31,225

                    Branch

                    Army

                    2,345

                     

                    47,850

                    13,238d

                     

                    43,504

                    2,083

                     

                    526

                     

                    1,238

                    168

                    20,325

                     

                    131,277

                     

                    Air Force

                      

                    4,148

                    2,693

                     

                    3,634

                    532

                     

                    67

                    3,723

                    447

                    28

                    14,945

                     

                    30,217

                     

                    Marines

                      

                    8,977

                    2,304

                     

                    2,658

                    503

                     

                    213

                     

                    142

                    58

                    1,569

                     

                    16,424

                     

                    Navy

                      

                    5,252

                    2,682

                     

                    2,712

                     

                    11,868

                    313

                     

                    223

                    49

                    7,986

                     

                    31,085

                     

                    Other

                         

                    1,682

                    577

                         

                    547

                     

                    2,806

                    Rank

                    Enlisted

                    982

                    1,398

                    61,797

                    17,553

                    3,581

                     

                    3,298

                    10,244

                     

                    3,202

                    1,705

                    241

                    326

                     

                    104,327

                     

                    Officer

                    155

                    122

                     

                    3,060

                      

                    397

                    1,624

                      

                    325

                    63

                      

                    5,746

                     

                    Noncom

                    1,208

                               

                    32,219

                     

                    33,427

                     

                    Com

                      

                    3,686

                             

                    11,537

                     

                    15,223

                     

                    Warrant

                      

                    744

                    288

                            

                    1,290

                     

                    2,322

                    Reference

                     

                    [7]

                    [8]

                    [37]

                    [10]

                    [17]

                    [11]

                    [38]

                    [12]

                    [13]

                    [4]

                    [1]

                    [14]

                    [15]

                    [16]

                     

                    Abbreviations: Com Commissioned Officer, DoD Department of Defense, GW Gulf War, HI Hawaii, NG National Guard, Noncom Noncommissioned Officer, OR Oregon, PA Pennsylvania, Res Reserve Service, VA Department of Veterans Affairs, WA Washington.

                    aSome cells may not total to the full study sample size due to rounding imprecision in published percentages, or missing data or errors in the original publication.

                    bFigures presented are based on 2,345 participants of Devens Cohort Study, as reported in [7]. Figures for the full cohort of 2,949 are not available in published literature.

                    cFigures presented are based on the subset of Millennium Cohort participants who served during the era of the first Gulf War. These are unpublished data that were provided to us in a personal communication by Dr. Nancy Crum-Cianflone of the Naval Health Research Center.

                    dDenotes that the figure is reported variably in multiple publications, or is otherwise uncertain.

                    Survey instruments

                    The major domains surveyed by the studies and registries are summarized in Tables 3, 4, and 5. These comprise the domains of mental and physical health (Tables 3 and 4) and deployment-related exposures (Table 5). These areas are commonly assessed during epidemiological research on military cohorts.
                    Table 3

                    Health domains surveyed by major studies of U.S. veterans of the first Gulf War

                    Domain

                    Devens Cohort Study

                    New Orleans

                    VA Registry

                    Nat’l Health Survey

                    DoD Registry

                    PA & HI

                    Iowa

                    Seabees

                    OR & WA

                    Air Nat’l Guard

                    Kansas I

                    Kansas II

                    Millennium Cohort

                    Military Health Survey

                    Medical history and clinical diagnoses

                    X, XS

                    XS

                    X, XS

                    X

                    X

                     

                    X, XS

                    X, XS

                    X

                    X

                    X

                    X

                    X

                    X

                    Symptoms

                    X, XS

                    X, XS

                    X, XS

                    X

                    X

                    X

                    X

                    X, XS

                    X

                    X

                    X

                    X

                    X

                    X

                    Functional and health status

                    X, XS

                    XS

                    X, XS

                    X, XS

                      

                    X

                    X

                    XS

                    XS

                    X

                    X

                    X

                    X

                    Fatigue, CFS, fibromyalgia, MCS, MSI

                    X, XS

                     

                    XS

                    X, XS

                     

                    X

                    X

                    X, XS

                    XS

                    XS

                    X

                    X

                    X, XS

                    X

                    PTSD

                    X, XS

                    X, XS

                    XS

                    X, XS

                    Xs

                    X

                    X, XS

                    X, XS

                    XS

                    XS

                    X

                    X

                    X

                    X

                    Clinical evaluation and validation

                    Xs

                     

                    X, XS

                    XS

                    X

                     

                    X, XS

                    XS

                    X, XS

                    XS

                     

                    X

                    X

                    XS

                    Family and social support

                    X

                    XS

                        

                    XS

                           

                    Intelligence

                    XS

                    XS

                     

                    XS

                      

                    XS

                           

                    Personality

                    X

                    XS

                    XS

                               

                    Psychiatric status

                    X, XS

                    X, XS

                    XS

                    XS

                    XS

                    X

                    X

                    XS

                    XS

                       

                    X

                    X

                    Depression

                     

                    X, XS

                     

                    X, XS

                    XS

                     

                    X

                     

                    XS

                    XS

                      

                    X

                    X

                    Anxiety

                     

                    X, XS

                     

                    XS

                      

                    X

                     

                    XS

                       

                    X

                    X

                    Memory

                    XS

                    XS

                     

                    XS

                      

                    XS

                     

                    XS

                        

                    X

                    Executive function

                    XS

                    XS

                    XS

                    XS

                      

                    X, XS

                    X

                    XS

                         

                    Psychomotor function

                    XS

                    XS

                     

                    XS

                      

                    XS

                    XS

                     

                    XS

                        

                    Quality of life

                       

                    XS

                            

                    X

                     

                    References

                    [3, 7, 25, 26, 3943]

                    [3, 8, 42, 4447]

                    [9, 24, 4852]

                    [10, 29, 5356]

                    [30, 57]

                    [17, 58, 59]

                    [2, 60, 61]

                    [12, 31]

                    [32, 62, 63]

                    [4]

                    [1]

                    [14]

                    [34, 64, 65]a

                    [16, 35]

                    Abbreviations: CFS chronic fatigue syndrome, DoD Department of Defense, HI Hawaii, MCS multiple chemical sensitivity, MSI multisymptom illness, OR Oregon, PA Pennsylvania, PTSD post-traumatic stress disorder, VA Department of Veterans Affairs, WA Washington.

                    Note: subscripted “S” designates that some data for that domain are only available for a subset of the study sample.

                    a Additional reference: personal communication by Dr. Nancy Crum-Cianflone of the Naval Health Research Center.

                    Table 4

                    Psychological status evaluations used in major studies of U.S. veterans of the first Gulf War

                    Psychological Domain

                    Instrument

                    Acronym

                    Reference

                    Family and social support

                    Social Provision Scale

                     

                    [66, 67]

                     

                    Family Relationship Index

                    FRI

                    [68]

                     

                    Social Support Questionnaire

                     

                    [69]

                     

                    Social Support

                     

                    [70]

                     

                    Family Stress

                     

                    [71]

                     

                    Family Adaptability and Cohesion Evaluation Scale

                    FACES-II

                    [72]

                     

                    Relationship Quality

                     

                    [73]

                    Intelligence

                    National Adult Reading Test

                    NART

                    [74]

                     

                    Wechsler Adult Intelligence Scale

                    WAIS

                    [75]

                     

                    Shipley Institute of Living Scale

                    SILS

                    [76, 77]

                    Personality

                    Neuroticism, Extroversion, Openness Personality Inventory

                    NEO-PI

                    [78]

                     

                    Dispositional Resilience Scale

                    DRS

                    [79]

                     

                    Eysenck Personality Questionnaire

                     

                    [80]

                    Psychiatric status

                    Hopkins Symptom Checklist

                    HSCL

                    [81, 82]

                     

                    Minnesota Multiphasic Personality Inventory-2

                    MMPI-2

                    [83]

                     

                    Personality Assessment Inventory

                     

                    [84]

                     

                    Composite International Diagnostic Interview

                    CIDI

                    [85]

                     

                    Symptom Checklist-90-Revised

                    SCL-90R

                    [86]

                     

                    Diagnostic Interview Schedule

                    DIS

                    [87]

                     

                    Brief Symptom Inventory

                    BSI

                    [88, 89]

                     

                    Structured Clinical Interview for DSM-III-R

                    SCID

                    [90]

                     

                    Health Screening System

                    HSS

                    [91]

                    Depression

                    Prime-MD/Patient Health Questionnaire

                    PHQ

                    [92, 93]

                     

                    Beck Depression Inventory

                    BDI

                    [94]

                    Anxiety

                    Anxiety Sensitivity Index

                    ASI

                    [95]

                     

                    Beck Anxiety Inventory

                    BAI

                    [96, 97]

                     

                    State-Trait Anxiety Inventory

                     

                    [98]

                    Cognitive Functioning

                    Digit Span

                     

                    [75]

                     

                    California Verbal Learning Test

                    CVLT

                    [99]

                     

                    Rey Auditory Verbal Learning Test

                     

                    [100, 101]

                     

                    Test of Memory Malingering

                    TOMM

                    [102]

                     

                    Wechsler Memory Scale

                    WMS

                    [103, 104]

                     

                    Recognition Memory Test

                     

                    [105]

                     

                    Heaton Memory Test

                     

                    [106]

                     

                    Continuous Visual Memory Test

                     

                    [107]

                     

                    Auditory Verbal Learning Test

                    AVLT

                    [108]

                     

                    Trail-Making Tests A & B

                     

                    [109, 110]

                     

                    Continuous Performance Test

                     

                    [111, 112]

                     

                    Cognitive Failure Questionnaire

                     

                    [113]

                     

                    Wisconsin Card-Sorting Test

                     

                    [114]

                     

                    Stroop Test

                     

                    [115]

                     

                    Performance On-line

                     

                    [116]

                     

                    Rey-Osterrieth Complex Figure Test

                     

                    [117, 118]

                     

                    Paced Auditory Serial Addition Test

                    PASAT

                    [119]

                     

                    Controlled Oral Word Association Test

                    COWAT

                    [120]

                     

                    Neurobehavioral Evaluation System Continuous Performance Test

                     

                    [121]

                     

                    Behavioral Assessment and Research System

                    BARS

                    [122]

                     

                    Symbol Digit

                     

                    [123]

                     

                    Simple Reaction Time

                     

                    [124]

                     

                    Oregon Dual-Task Procedure

                     

                    [125]

                    Psychomotor function

                    Grip Strength Test

                      
                     

                    Symbol Digit

                     

                    [123]

                     

                    Grooved Pegboard

                     

                    [126]

                     

                    Purdue Pegboard

                     

                    [127]

                     

                    Finger Tapping Test

                     

                    [109, 128]

                    Health perception

                    Sickness Impact Profile

                     

                    [129, 130]

                     

                    Barsky Amplification Scale

                     

                    [131]

                     

                    Illness Behavior Questionnaire

                     

                    [132]

                    Quality of Life

                    Quality of Life Inventory

                     

                    [133]

                    Table 5

                    Deployment-related exposure domains surveyed by major studies of U.S. veterans of the first Gulf War

                    Exposure

                    Devens Cohort Study

                    New Orleans

                    VA Registry

                    Nat’l Health Survey

                    PA & HI

                    DoD Registry

                    Iowa

                    Seabees

                    OR & WA

                    Air Nat’l Guard

                    Kansas I

                    Kansas II

                    Millennium Cohort

                    Military Health Survey

                    Vaccinations

                    Any vaccine

                    X

                         

                    X

                      

                    X

                    X

                    X

                     

                    X

                    Anthrax

                    X

                     

                    X, XS

                    X

                     

                    X

                     

                    X

                    X

                       

                    X

                    X

                    Typhoid

                      

                    XS

                    X

                       

                    X

                          

                    Botulism

                      

                    X, XS

                    X

                     

                    X

                     

                    X

                    X

                        

                    X

                    Immune globulin

                      

                    XS

                    X

                       

                    X

                         

                    X

                    Plague

                      

                    XS

                    X

                       

                    X

                         

                    X

                    Meningococcus

                      

                    XS

                    X

                       

                    X

                          

                    Medications

                    Malaria pills

                      

                    XS

                    X

                     

                    X

                     

                    XS

                          

                    Pyridostigmine bromide

                    X

                    XS

                    X, XS

                    X

                     

                    X

                    X

                    X, XS

                    X

                    X

                     

                    X

                     

                    X

                    Ciprofloxacin & antibiotics

                    X

                     

                    XS

                    X

                       

                    X, XS

                          

                    Airborne exposures

                    Wore chemical protective gear

                    X

                     

                    XS

                    X

                    X

                     

                    X

                    X

                        

                    X

                    X

                    Petrol fuels/solvents or fumes

                      

                    X

                    X

                     

                    X

                    X

                    XS

                    X

                        

                    X

                    Smoke, oil fires, combustion products

                    X

                    XS

                    X, XS

                    X

                    X

                    X

                    X

                    X, XS

                    X

                    X

                     

                    X

                      

                    Smoke from tent heaters

                    X

                    XS

                    X, XS

                    X

                     

                    X

                    X

                    X, XS

                    X

                        

                    X

                    Vehicle exhaust

                    X

                    XS

                     

                    X

                              

                    Chemical or biological warfare agents

                    X

                    XS

                    XS

                        

                    XS

                    X

                    X

                      

                    X

                    X

                    Nerve gas

                      

                    X

                    X

                     

                    X

                    X

                           

                    Mustard gas

                      

                    X

                    X

                     

                    X

                    X

                           

                    SCUD missile or artillery explosions nearby; debris contact

                    X

                    XS

                    XS

                    X

                     

                    X

                    X

                     

                    X

                    X

                     

                    X

                     

                    X

                    Burning trash or feces

                    X

                    XS

                    X, XS

                    X

                      

                    X

                     

                    X

                       

                    XS

                     

                    CARC paint

                      

                    X

                    X

                     

                    X

                    X

                     

                    X

                      

                    X

                     

                    X

                    Radiation sources

                    Depleted uranium

                      

                    X

                    X

                     

                    X

                    X

                    XS

                    X

                      

                    X

                    X

                    X

                    Microwaves

                      

                    X

                    X

                     

                    X

                    X

                         

                    X

                     

                    Sources of infection or contaminants

                    Contaminated or local food

                    X

                     

                    X

                    X

                     

                    X

                    X

                    X

                    X

                    X

                        

                    Contaminated or local water

                    X

                    XS

                    X

                    X

                     

                    X

                    X

                    X

                    X

                    X

                        

                    Pesticides

                    X

                    XS

                    X

                    X

                     

                    X

                    X

                    X

                    X

                    X

                     

                    X

                    X

                    X

                    Live/dead animals or insects

                    X

                     

                    XS

                    X

                      

                    X

                    X

                    X

                      

                    X

                      

                    Psychological trauma

                    Life events

                    X

                     

                    XS

                    X

                    X

                     

                    X

                     

                    XS

                    X

                      

                    X

                     

                    Combat-related stressors

                    X

                    X

                    XS

                    XS

                     

                    X

                    X

                     

                    XS

                       

                    X

                    X

                    Direct combat duty

                    X

                     

                    X, XS

                    X

                    X

                    X

                    X

                    X

                     

                    X

                     

                    X

                    X

                    X

                    Witnessed casualties

                    X

                     

                    XS

                    X

                    X

                    X

                    X

                    X

                     

                    X

                     

                    X

                    X

                    X

                    Contact with POW

                      

                    XS

                    X

                      

                    X

                    X

                       

                    X

                    X

                    X

                    Physical injury

                    X

                     

                    XS

                     

                    X

                    X

                    X

                      

                    X

                      

                    XS

                    X

                    Suffered forced sexual relations, assault, or sexual harassment

                      

                    XS

                    X

                      

                    X

                     

                    XS

                       

                    X

                    X

                    Behavioral risk factors

                    Alcohol use

                    X

                    XS

                    XS

                    X

                    X

                     

                    X

                    X

                    X

                    X

                      

                    X

                    X

                    Tobacco use

                    X

                    XS

                    X

                    X

                    X

                    X

                    X

                    X

                    X

                    X

                     

                    X

                    X

                    X

                    References

                    [3, 2527, 40, 134]

                    [3, 8, 45, 135]

                    [48, 51]

                    [10, 136, 137]

                    [17, 58, 59]

                    [30, 138140]

                    [2, 38]

                    [12, 31]

                    [13, 32, 141]

                    [4, 33, 142]

                    [1]

                    [14]

                    [34, 143145]a

                    [35]

                    Abbreviations: CARC Chemical agent resistant coating, DoD Department of Defense, HI Hawaii, OR Oregon, PA Pennsylvania, POW Prisoner of war, VA Department of Veterans Affairs, WA Washington.

                    Note: subscripted “S” designates that some data for that domain are only available for a subset of the study sample.

                    aAdditional reference: personal communication by Dr. Nancy Crum-Cianflone of the Naval Health Research Center.

                    As described in further detail below, the domains of mental and physical health include health status, functional status, symptoms, fatigue complex (fatigue, chronic fatigue syndrome (CFS), fibromyalgia, and multiple chemical sensitivities), post-traumatic stress disorder (PTSD), neurocognitive and/or psychological status, and clinical evaluations and validations. The domains of environmental exposures include vaccinations, medications, airborne exposures, radiation sources, sources of infection or contaminants, psychological trauma, and behavioral risk factors. Each of these exposure domains has multiple subdomains. This review is not a comprehensive representation of all areas of research in these studies, but includes the domains most frequently studied thus far, that are arguably perceived to be the most critical.

                    Mental and physical health domains

                    Medical history and clinical diagnoses

                    Comparable survey measures were used by multiple studies. Eight studies (Devens, Kansas I, Kansas II, National Health Survey, Millennium Cohort, Seabees, Military Health Study, and a VA Registry sub-study) asked whether participants had been diagnosed or treated by a clinician for any of several medical conditions. There was substantial overlap in the conditions queried, including diabetes, depression, asthma, bronchitis, and chronic fatigue syndrome [1, 9, 12, 14, 24, 25, 29, 34, 35, 146]. It was also common to ask the approximate onset date for each condition. In contrast, Oregon-Washington and the VA and DoD registries had open-ended questions about health history, which resulted in a broad range of responses that are not easily comparable [9, 30, 48, 62].

                    Some instruments were used by only one study. For example, Iowa alone drew questions from instruments such as the National Health Interview Survey [147], the Behavioral Risk Factor Surveillance Survey [148], and the Agricultural Health Study [2, 149, 150].

                    Symptoms

                    Symptoms were assessed by all of the studies. They were typically queried by asking if the veteran had experienced persistent or recurring symptoms during the 12 months prior to the survey, using a symptom checklist. The number of symptoms surveyed varied by study (8 to 78 symptoms). In addition, some studies collected information about symptom severity and date of onset [1, 3, 4, 10, 12, 14, 16, 17, 24, 29, 30, 34, 35, 48, 60, 62, 136, 151153]. New Orleans [8] and Devens [25] evaluated symptoms using the Health Symptom Checklist [81, 82]. Devens also used a variant of the Psychosomatic Complaint Scale (Psychological Well-Being Scale) [26, 79, 154, 155]. In later survey versions, Devens used the Expanded Health Symptom Checklist, which included additional questions on symptom presence during the past 30 days, duration, trajectory, and frequency [3, 25, 26]. National Health Survey used the 15-item Patient Health Questionnaire to measure the occurrence of somatic symptoms during the 4 weeks prior to survey administration [29, 156].

                    In this subdomain, the registries and Iowa differed the most from the rest of the studies. The registries both used open-ended questions to assess symptoms, instead of a binary format [9, 30, 48]. Iowa asked additional questions about symptoms of asthma and bronchitis from the American Thoracic Society Questionnaire [157, 158].

                    Functional status and health status

                    Of the eight studies that evaluated functional status, all used the SF-12, SF-36, or Veterans SF-36, either alone or in combination [24, 16, 29, 34, 39, 53, 63, 159163]. The only study that asked questions to further evaluate functional status was the Iowa study, which administered the Health Utilities Index-Mark 3 [164, 165].

                    Health status was one of the most commonly assessed domains. All of the studies, with the exception of Pennsylvania-Hawaii, administered a health status evaluation. However, few of the instruments were directly comparable because their response scales varied, making it difficult to compare the responses. For instance, the Military Health, National Health Survey, Air National Guard, Kansas II, VA Registry, and Seabees all asked a general health rating question [4, 9, 12, 14, 24, 29, 35, 48, 153]. However, National Health Survey, Air National Guard, and the VA Registry used a five-category response scale, while Kansas II used a four-category response scale, and the Seabees used a three-category response scale. In addition, the response scale anchor points used by Air National Guard and National Health Survey differed from that used by the VA Registry, and the responses cannot be directly compared.

                    Chronic multisymptom illness and related diagnoses

                    It is estimated that chronic multisymptom illness and other symptom-based diagnoses affect up to 25 to 32% of the Gulf War veteran population [23]. Although several instruments have been developed to diagnose these conditions, they are still challenging to distinguish due to the number of their symptoms that overlap with other illnesses. Eleven of the studies collected data on at least one of these illnesses: fatigue, chronic fatigue syndrome, fibromyalgia, multiple chemical sensitivity, and multi-symptom illness.

                    Fatigue was assessed by Iowa, Devens, and a VA Registry substudy using the Chalder Fatigue Scale [2, 60, 166], the Fatigue Severity Scale [40, 167], and the Multidimensional Fatigue Inventory [49, 168], respectively. Chronic fatigue syndrome was most commonly identified in National Health Survey, Iowa, VA Registry, Air National Guard, Kansas II, Military Health Study, a Devens substudy, and Seabees [2, 4, 12, 14, 24, 29, 31, 35, 41, 53, 169] using the Centers for Disease Control definition developed by Fukuda [169]. In addition, a National Health Survey substudy validated self-reported chronic fatigue syndrome, fibromyalgia, and other conditions by physician examination [53]. Fibromyalgia was evaluated by four studies (Iowa, a National Health Survey substudy, Military Health Study, and Oregon-Washington) using the American College of Rheumatology criteria [2, 35, 53, 62, 170, 171]. Multiple chemical sensitivity was queried in five studies, but they each used different measures. These measures included the Chemical Odor Intolerance Index and a scale developed by the Agency for Toxic Substances and Disease Registry [35, 49, 50, 172, 173] and multiple internally-developed measures [12, 25, 40, 41, 174]. Several studies inferred the presence of medically unexplained multisymptom illness based on the symptoms and medical conditions that they surveyed [1, 4, 12, 14, 17, 24, 35, 41, 175, 176]; however, the 2005 National Health Survey asked questions specifically about participants’ experiences with unexplained multisymptom illness. These included years of first and most recent experience, activities and treatments that improve or aggravate the condition, and status relative to initial diagnosis [29, 153]. In addition to questions about chronic fatigue syndrome, fibromyalgia, and multiple chemical sensitivity, the Military Health Study contained a large number of self-constructed questions about illness symptoms in support of their goal of validating a case definition for Gulf War Illness [16, 35].

                    Post-traumatic stress disorder (PTSD)

                    Combat-related stress was initially hypothesized to be one of the factors responsible for the illnesses afflicting Gulf War veterans [177]; as a result, it was one of the few factors surveyed by all of the studies. The most frequently used instruments included the Clinician Administered PTSD Scale (4 studies) [30, 42, 48, 54, 178, 179], PTSD Checklist Military and Civilian versions (6 studies) [2, 8, 25, 26, 29, 40, 60, 63, 64, 180, 181], Mississippi Scale for Combat-Related PTSD (3 studies) [61, 63, 182], Mississippi Scale for Desert Storm War Zone Personnel (3 studies) [4, 8, 42, 183], the Structured Clinical Interview for DSM-III-R - Patient Edition (3 studies) [30, 44, 48, 90, 184], and the Impact of Events Scale (2 studies) [51, 58, 185]. Additional PTSD instruments, including the Penn Inventory for PTSD, were used by individual studies [63].

                    Psychological status

                    Psychological status is a very broad subdomain and is by far one of the most discordant areas in assessment across studies. It includes several fields, such as family and social support, intelligence, personality, psychiatric status, depression, anxiety, memory, executive function, psychomotor function, health perception, and quality of life. There were a total of 59 tests used in this domain, and all of the studies except Kansas I and Kansas II evaluated at least some of the fields. Many of the survey instruments and tests were only used by one study, but the most frequently used instruments were the Wechsler Adult Intelligence Scale and the Trail-Making Tests, which were each used by 5 studies to measure intelligence and executive function, respectively. A summary of the instruments used to evaluate psychological status can be found in Table 4.

                    Clinical evaluations and stored biospecimens

                    Several of the studies have performed detailed clinical evaluations, including complete physicals with standard laboratory tests (VA and DoD Registries, Iowa, Air National Guard, Oregon-Washington, and a National Health Survey substudy) [4, 9, 13, 24, 30, 53, 165]. Both registries have implemented a battery of additional laboratory tests for veterans requiring further evaluation [9, 30, 52]. The Devens and Military Health studies have performed MRIs [16, 35, 43]. Many of the studies have performed analyses using blood specimens, particularly for clinical laboratory measures; however, for most, it is not clear whether sufficient samples remain to support additional biomarker research. To our knowledge, only Seabees, Military Health Study, and Kansas II have collected biospecimens for future studies [14, 16, 31, 35, 186].

                    Healthcare expenditures and utilization

                    Veterans of this era varied in their approaches to utilizing healthcare services, and this variability was noted both within and between deployed and non-deployed veterans. In particular, participation in health registries that involved a clinical evaluation, such as the VA and DOD registries, accounted for a significant number of medical encounters. Health expenditures were directly addressed only by Iowa, using the National Health Interview Survey and the National Medical Expenditure Survey [2, 187189]. Iowa also administered a telephone survey on health utilization to a subsample of their population who suffered from multiple chemical sensitivity [190]. Devens and National Health Survey used rough measures of health utilization, asking the number of clinic visits or hospitalizations [10, 26].

                    Deployment-related exposure domains

                    Vaccinations

                    Receipt of vaccinations prior to or during deployment was assessed by all but two studies (New Orleans and Pennsylvania-Hawaii). The vaccinations queried included anthrax, botulinum, typhoid, meningococcus, plague, a generic “any vaccinations,” and receipt of immune globulin. The form and scope of the questions can be divided roughly into two categories: those that asked whether any vaccines had been received (Iowa, Kansas I, Kansas II, Air National Guard, and Military Health Study) [1, 2, 14, 33, 35], and those that asked if specific vaccines had been received (Devens, VA and DoD Registries, National Health Survey, Seabees, Oregon-Washington, Millennium Cohort, and Military Health Study) [10, 12, 15, 25, 30, 32, 35, 40, 48, 51, 136].

                    Medications

                    Three medications were commonly assessed: malaria prophylaxis (any), ciprofloxacin and/or other antibiotics, and pyridostigmine bromide [2, 3, 10, 1214, 31, 35, 48, 51, 134, 138, 142]. Use of these medications was queried using a binary format, or a categorical format that could be collapsed to binary. However, it was common to attempt to elicit additional details of pyridostigmine bromide use, in the form of the total number of pills used, frequency of use, number of days on which a certain dose was exceeded, and the occurrence of specified side effects or feelings of illness after using the pills [2, 13, 35, 51, 134, 136]. Some studies also asked open-ended questions about the use of over-the-counter and prescription medications [13, 35].

                    Airborne exposures

                    Many of the Gulf War studies assessed exposure to airborne toxins. These contaminants included petroleum fuels, solvents, fumes, smoke, combustion products from oil fires or incinerated trash/feces, tent heater smoke, vehicle exhaust, chemical agent resistant compound paint, debris from missile or artillery explosions, and chemical or biological warfare agents.

                    All of the studies except Kansas I asked questions regarding airborne exposures. These exposures were most commonly assessed using a binary response or a categorical format that could be collapsed to binary. In addition, several studies asked additional questions about the number of days or times exposed to airborne toxins; these types of questions had either open-ended or categorical response types [3, 10, 1214, 17, 2527, 31, 35, 38, 48, 51, 58, 136, 138, 142, 143]. In addition, Iowa asked whether the exposure was temporally associated with feelings of illness [38].

                    Radiation sources

                    The most commonly surveyed sources of radiation were depleted uranium and microwaves. Most of the studies queried these exposures using a binary or a categorical format that could be collapsed to binary [10, 13, 31, 38, 48, 138, 144]. In addition, National Health Survey and Iowa both asked about time period of exposure [38, 136].

                    Military Health Study and Kansas II evaluated depleted uranium exposure using questions regarding contact with destroyed enemy vehicles (or inhalation of smoke from vehicles) that may have been struck with artillery rounds containing depleted uranium [14], and exposure to the Camp Doha fire. They also asked whether participants had undergone a urine test for the presence of depleted uranium, the timing and provider of the test, and the results [35].

                    Sources of infection or contaminants

                    Many of the studies surveyed their participants regarding exposure to potential sources of infection or contamination, which included food, water, pesticides, and local fauna. The response formats for most of the questions were at least binary [10, 1214, 25, 32, 33, 35, 38, 48, 51, 136, 138, 142]. Similar question phrasing was used for National Health Survey, Iowa, and the VA Registry [38, 48, 136] and furthermore, National Health Survey and Iowa used the same three-point scale to describe the length of exposure [38, 136]. Devens and a New Orleans subset also used identical tools to evaluate exposures [3, 25, 45]. Additional exposure information was gathered in three studies. A VA registry subset was asked about the frequency of each exposure’s occurrence [51]. Devens inquired about water and pesticide use, including source of drinking water, unusual smell or tastes in the water, any illnesses or health problems caused by the drinking water, pesticide name, who sprayed the pesticide, presence of any acute symptoms, and exposure frequency [25]. Military Health Study also asked questions about frequency and quantity of application [35].

                    Psychological trauma

                    The experiencing of psychologically stressful events was one of the primary factors surveyed in the initial studies of veterans of the Gulf War. The perceived importance of this factor is demonstrated by its inclusion in all of the studies except Kansas I. The most commonly studied items were life events and combat-related stressors, the latter of which included direct combat duty, witnessing of casualties, POW contact, physical injury, and the experience of sexual harassment and/or sexual assault or rape.

                    Because combat-related traumatic events have long been recognized for their effects on psychological health, some tools for their assessment existed at the initiation of the Gulf War studies. Instruments used in these studies included the Combat Exposure Scale [191, 192], the Mississippi Scale (Operation Desert Storm version) [183], the Keane Combat Scale [193], the Operation Desert Storm Stress Exposure Scale [194], and the Haley Gulf War Combat Exposure Index [35]. With the exception of the Keane Combat Scale (used by Oregon-Washington, Military Health Study, and VA Registry), each instrument was used by only one study. In addition, many studies incorporated self-constructed questions regarding lifetime and combat-related stressors. Thus, creating a framework of direct comparability for this domain may be challenging.

                    Behavioral risk factors

                    All but Kansas I evaluated behavioral risk factors. The most common of these were alcohol and tobacco use. Alcohol use was primarily evaluated using self-constructed questions about current drinking status, alcohol abuse, and number of drinks consumed [2, 4, 10, 12, 13, 17, 25, 35, 45, 51, 195, 196]. However, Iowa, National Health Survey, and Millennium Cohort used validated instruments. Iowa used the CAGE (Cut back, Annoyance, Guilt, Eye-openers) questionnaire [2, 197], and both National Health Survey and Millennium Cohor used the alcohol component of the Patient Health Questionnaire [92].Tobacco use was most commonly queried by asking participants about their smoking status (never, past, or current smoker) [24, 10, 1214, 17, 25, 35, 45, 48, 139, 198]. Additional information obtained varied by study, but included increases or decreases in daily smoking habits [198] and whether participants had smoked at least 100 cigarettes [51, 198].

                    Validation studies

                    The studies reviewed herein all gathered self-reported data, which has well-documented limitations, including recall bias and lack of correlation with objective measures. In light of these challenges to data quality, several studies performed reliability and validation analyses, using test-retest methods and comparing self-reported data to clinical evaluations and medical records.

                    Seabees, Iowa, Millennium Cohort, and Oregon-Washington measured the internal reliability of self-reported survey data, including demographics, physical and functional status, symptoms, health histories, vaccinations, and exposures, using test-retest methods. The results of these studies were mixed, particularly with respect to self-reported exposures, diagnosed health conditions, and symptoms. The Seabees study [31] reported high reliability of demographic attributes (kappas 0.89-1.00), and moderate reliability of exposures (0.60-0.70) and “other” survey items (0.51-0.67). However, self-reported diagnoses and symptoms had widely ranging kappa coefficients of −0.01-1.00 and −0.01-0.86, respectively [31]. The Iowa cohort reported test-retest agreement percentages of 89.6-97.0% and kappa coefficients of 0.39-0.70 for self-reported medical and psychiatric concerns [2]. In the Oregon-Washington evaluation of self-reported exposure reliability, only eight of thirteen exposures had a kappa coefficient statistically greater than 0.4, reflecting poor agreement on the majority of measures [32]. The Oregon-Washington investigators also assessed self-reported exposure misclassification by comparing reported exposure to anthrax and botulinum toxoid vaccines, chemical warfare agents, and pyridostigmine bromide to reported deployment periods, as each exposure was only possible during known periods of time. The results of this comparison suggested that these exposures may be overreported among certain subgroups of veterans [32]. Similarly, an analysis of National Health Survey data found evidence of reporting bias in self-reported anthrax vaccination data [199]. We note that it has been difficult to validate reported exposures due to the absence of objective documentation.

                    Clinical evaluations and medical records were used to externally validate self-reported information by several studies. As was the case for test-retest reliability analyses, the results were mixed. A New Orleans substudy examined the validity of survey-based PTSD assessments by calculating the agreement between the Structured Clinical Interview for DSM-III-R (SCID) and other PTSD diagnostic measures (agreement 82-100%) [44]. The Seabees validation study, which compared personnel and medical records to survey responses, found kappa coefficients of −0.02-0.48 for self-reported diagnosed illness, 0.41-0.76 for demographic traits, and 0.59-0.66 for “other” characteristics [31]. In the National Health Survey, comparison of reported reasons for clinic visits and hospitalizations to medical records found agreement rates greater than 90% [10, 29], and clinical evaluations confirmed an increased risk of certain conditions among deployed veterans [53]. The Iowa study drew on state registries to validate reported birth defects and cancers [60]. In addition, a case validation study in a subset of Iowa cohort members found that only 32% of those who originally reported depressive symptoms met criteria for lifetime depression after a later SCID-IV interview; however, multiple factors may have contributed to this difference [200]. The Millennium Cohort found excellent negative agreement (generally > 95%) and moderate positive agreement when comparing self-reported medical conditions to Department of Defense electronic medical records, suggesting that self-reported data may be useful in excluding the presence of conditions [146]. Finally, in a small-sample comparison of thirteen self-reported medical conditions to medical records, the Devens Cohort Study observed low-moderate kappa coefficients of 0.35-0.64 for most conditions [39]. These heterogeneous findings regarding reliability and validity lend support to concerns about the utility of self-reported data, and emphasize the importance of evaluating reliability and validity early in the study implementation period, and including objective data sources, when resources permit.

                    Conclusions

                    We compared the survey tools used in keystone epidemiologic studies and registries of Gulf War Era veterans, with the intent of highlighting commonalities and differences in efforts aimed at understanding health and risk factors. It is apparent that there are many areas of at least minimal concordance with respect to question and response format. We note that some investigators intentionally drew on the materials used by prior studies, establishing areas of commonality [1, 12, 60]. Among mental and physical health domains, there was moderate concordance among the measures used to assess medical history, symptoms, functional status, and the diagnoses of chronic fatigue syndrome and fibromyalgia. Most of the exposure assessment instruments also have response formats that can be reduced to the same binary framework. These similarities suggest that meta-analyses of study-level or individual-level data could be performed on several of the subdomains, with varying degrees of loss of detail.

                    In contrast, there is substantial variation in survey instruments for the subdomains of health status, PTSD, psychological status, psychological trauma, and the diagnoses of fatigue, multiple chemical sensitivity, and multisymptom illness. The nature of this variability differs by subdomain. While this does not preclude the use of meta-analytic techniques, it requires the analyst to carefully consider issues of heterogeneity and whether the instruments demonstrate convergent validity.

                    We cannot overemphasize the importance of considering the impact of heterogeneity in study design, population, sampling methods, quality, and generalizability to the validity of meta-analyses. While it is possible to carefully combine data from similarly-conducted epidemiologic studies [201], the practical reality is that explicitly accommodating design and quality differences to yield valid inference in analyses of this type remains an extremely difficult (at times impossible) task [202, 203]. In addition, restrictions on the sharing of study data may present a logistical challenge to the performance of individual-level joint analyses. This places additional emphasis on the need for investigators to consent future study participants using language that explicitly permits recontact for future studies or (at minimum) sharing of de-identified data under approved protocols for Gulf War Era related research.

                    Based on our review of the existing studies, we suggest three considerations for future studies of Gulf War Era veterans. First, to engage Gulf War Era research experts during study planning. Second, to gather blood for genetic and proteomic analyses, and link the specimens to survey and medical/administrative records. Third, to carefully consider the strengths and weaknesses of the survey instruments used in the past, and select instruments that are appropriately validated, detailed, and compatible with previous studies. These three activities will support the development of unified data and biospecimen resources with opportunities for analytic collaborations. Through such efforts, epidemiologic research can continue to make important strides that advance our collective ability to enhance the health of these veterans.

                    Disclaimer

                    The contents of this manuscript do not represent the views of the Department of Veterans Affairs or the United States Government.

                    Abbreviations

                    AF: 

                    Air Force

                    CAGE: 

                    Cut back, Annoying, Guilt, Eye-openers

                    CARC: 

                    Chemical agent resistant coating

                    CFS: 

                    Chronic fatigue syndrome

                    Com: 

                    Commissioned officer

                    DoD: 

                    United States Department of Defense

                    DSM-III-R: 

                    Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised

                    FL: 

                    Florida

                    GW: 

                    Gulf War

                    HI: 

                    Hawaii

                    KS: 

                    Kansas

                    MA: 

                    Massachusetts

                    MCS: 

                    Multiple chemical sensitivity

                    MO: 

                    Missouri

                    MRI: 

                    Magnetic resonance imaging

                    MSI: 

                    Multisymptom illness

                    NG: 

                    National Guard

                    Noncom: 

                    Noncommissioned officer

                    OIF: 

                    Operation Iraqi Freedom

                    OMB: 

                    Office of Management and Budget

                    OR: 

                    Oregon

                    PA: 

                    Pennsylvania

                    PTSD: 

                    Post-traumatic stress disorder

                    POW: 

                    Prisoner of war

                    Res: 

                    Reserve service

                    SCID: 

                    Structured clinical interview for DSM-III-R

                    Seabees: 

                    Construction Battalion (CB)

                    SF: 

                    Medical Outcomes Study Short Form Survey

                    SW: 

                    Southwest

                    US: 

                    United States of America

                    VA: 

                    United States Department of Veterans Affairs

                    WA: 

                    Washington.

                    Declarations

                    Acknowledgements

                    We are grateful for the helpful perspectives and materials provided by Drs. Jeannie Beckham, Nancy Crum-Cianflone, Scott McDonald, Susan Proctor, and Robert F. Woolson. Authors RBM, CMT, EH, MJ, and DP were supported by award #585, Cooperative Studies Program (CSP), Office of Research and Development, Veterans Health Administration, Department of Veterans Affairs. GDH manages the program that provides individual study funding, including CSP #585.

                    Authors’ Affiliations

                    (1)
                    Durham Epidemiologic Research and Information Center, Durham Veterans Affairs Medical Center
                    (2)
                    Department of Epidemiology, Rollins School of Public Health, Emory University
                    (3)
                    Department of Medicine, Duke University
                    (4)
                    Office of Research and Development, Department of Veterans Affairs, Cooperative Studies Program
                    (5)
                    Durham Veterans Affairs Medical Center
                    (6)
                    Health Services Research and Development, Durham Veterans Affairs Medical Center
                    (7)
                    Clinical Informatics

                    References

                    1. Steele L: Prevalence and patterns of Gulf War illness in Kansas veterans: association of symptoms with characteristics of person, place, and time of military service. Am J Epidemiol 2000, 152:992–1002.View Article
                    2. The Iowa Persian Gulf Study Group: Self-reported illness and health status among Gulf War veterans: a population-based study. JAMA 1997, 277:238–245.View Article
                    3. Proctor S: Health status of Persian Gulf War veterans: self-reported symptoms, environmental exposures, and the effect of stress. Int J Epidemiol 1998, 27:1000–1010.View Article
                    4. Fukuda K, Nisenbaum R, Stewart G, Thompson W, Robin L, Washko R, Noah D, Barrett D, Randall B, Herwaldt B, et al.: Chronic multisymptom illness affecting Air Force veterans of the Gulf War. JAMA 1998, 280:981–988.View Article
                    5. Kang HK, Bullman TA: Mortality among U.S. veterans of the Persian Gulf War. N Engl J Med 1996, 335:1498–1504.View Article
                    6. Gray GC, Kang HK: Healthcare utilization and mortality among veterans of the Gulf War. Philos Trans R Soc Lond B Biol Sci 2006, 361:553–569.View Article
                    7. Wolfe J, Kelly JM, Bucsela ML: Fort Devens reunion survey: report of phase I. In Returning Persian Gulf Troops: First Year Findings. West Haven: Department of Veterans Affairs; 1992:19–44.
                    8. Brailey K, Vasterling J, Sutker P: Psychological Aftermath of Participation in the Persian Gulf War. In The Environment and Mental Health: A Guide for Clinicians. Edited by: Lundberg A. Mahwah: Lawrence Erlbaum Associates; 1998:83–101.
                    9. Murphy FM, Kang H, Dalager NA, Lee KY, Allen RE, Mather SH, Kizer KW: The health status of Gulf War veterans: lessons learned from the Department of Veterans Affairs Health Registry. Mil Med 1999, 164:327–331.
                    10. Kang H, Mahan CM, Lee KY, Magee CA, Murphy FM: Illnesses among United States veterans of the Gulf War: a population-based survey of 30,000 veterans. J Occup Environ Med 2000, 42:491–501.View Article
                    11. Stuart JA, Murray KM, Ursano RJ, Wright KM: The Department of Defense's Persian Gulf War registry year 2000: an examination of veterans' health status. Mil Med 2002, 167:121–128.
                    12. Gray GC, Reed RJ, Kaiser KS, Smith TC, Gastanaga VM: Self-reported symptoms and medical conditions among 11,868 Gulf War-era veterans: the Seabee Health Study. Am J Epidemiol 2002, 155:1033–1044.View Article
                    13. Spencer PS, McCauley LA, Lapidus JA, Lasarev M, Joos SK, Storzbach D: Self-reported exposures and their association with unexplained illness in a population-based case–control study of Gulf War veterans. J Occup Environ Med 2001, 43:1041–1056.
                    14. Steele L, Sastre A, Gerkovich MM, Cook MR: Complex factors in the etiology of Gulf War Illness: wartime exposures and risk factors in veteran subgroups. Environ Health Perspect 2012, 120:112–118.View Article
                    15. Smith B, Leard CA, Smith TC, Reed RJ, Ryan MA: Anthrax vaccination in the Millennium Cohort: validation and measures of health. Am J Prev Med 2007, 32:347–353.View Article
                    16. Iannacchione VG, Dever JA, Bann CM, Considine KA, Creel D, Carson CP, Best H, Haley RW: Validation of a research case definition of Gulf War Illness in the 1991 US military population. Neuroepidemiology 2011, 37:129–140.View Article
                    17. Stretch RH, Bliese PD, Marlowe DH, Wright KM, Knudson KH, Hoover CH: Physical health symptomatology of Gulf War-era service personnel from the states of Pennsylvania and Hawaii. Mil Med 1995, 160:131–136.
                    18. National Institute of Health: The Persian Gulf experience and health: NIH Technology Assessment Workshop Panel. JAMA 1994, 272:391–396.View Article
                    19. Persian Gulf Veterans Coordinating Board: Unexplained illnesses among Desert Storm veterans: a search for causes, treatment, and cooperation. Arch Intern Med 1995, 155:262–268.View Article
                    20. Institute of Medicine: Health Consequences of Service During the Persian Gulf War: Recommendations for Research and Information Systems. Washington: The National Academies Press; 1996.
                    21. Institute of Medicine: Gulf War and Health. 1–8 volumes. Washington: The National Academies Press; 2000–2010.
                    22. Institute of Medicine: Gulf War and Health, Volume 8: Update of Health Effects of Serving in the Gulf War. Washington: The National Academies Press; 2010.
                    23. Research Advisory Committee on Gulf War Veterans' Illnesses: Gulf War Illness and the Health of Gulf War Veterans: Scientific Findings and Recommendations. Washington: U.S. Department of Veterans Affairs; 2008.
                    24. Hallman WK, Kipen HM, Diefenbach M, Boyd K, Kang H, Leventhal H, Wartenberg D: Symptom patterns among Gulf War registry veterans. Am J Public Health 2003, 93:624–630.View Article
                    25. White R, Wolfe J, Proctor S, Keane TM: Devens Cohort Study: Time 3. Boston: Boston Environmental Hazards Center and National Center for PTSD, Department of Veterans Affairs, VA Boston Healthcare System; 1994–1996.
                    26. Wolfe J, Mark W, Keane T: Ft. Devens ODS Reunion Survey: Time 2. Boston: National Center for PTSD, Behavioral Science Division, Department of Veterans Affairs, VA Boston Healthcare System; 1991–1993.
                    27. Wolfe J, Mark W, Keane T: Ft. Devens ODS Reunion Survey: Time 1. Boston: National Center for PTSD, Behavioral Science Division, Department of Veterans Affairs, VA Boston Healthcare System; 1991–1993.
                    28. Committee on the Evaluation of the Department of Veterans Affairs Uniform Case Assessment Protocol (Institute of Medicine): Adequacy of the VA Persian Gulf Registry and Uniform Case Assessment Protocol. Washington: National Academies Press; 1998.
                    29. Kang H, Li B, Mahan CM, Eisen SA, Engel CJ: Health of US veterans of 1991 Gulf War: a follow-up survey in 10 years. J Occup Environ Med 2009, 51:401–410.View Article
                    30. Joseph SC, Comprehensive Clinical Evaluation Program Evaluation Team: A comprehensive clinical evaluation of 20,000 Persian Gulf War veterans. Mil Med 1997, 162:149–155.
                    31. Gray GC, Kaiser KS, Hawksworth AW, Hall FW, Barrett-Connor E: Increased postwar symptoms and psychological morbidity among U.S. Navy Gulf War veterans. Am J Trop Med Hyg 1999, 60:758–766.
                    32. McCauley LA, Joos SK, Spencer PS, Lasarev M, Shuell T, Members of the Portland Environmental Hazards Research Center: Strategies to assess validity of self-reported exposures during the Persian Gulf War. Environ Res 1999, 81:195–205.View Article
                    33. Centers for Disease Control and Prevention: Unexplained illness among Persian Gulf War veterans in an Air National Guard Unit: preliminary report--August 1990-March 1995. MMWR 1995, 44:443–447.
                    34. Gray GC, Chesbrough KB, Ryan MA, Amoroso P, Boyko EJ, Gackstetter GD, Hooper TI, Riddle JR: The Millennium Cohort Study: a 21-year prospective cohort study of 140,000 military personnel. Mil Med 2002, 167:483–488.
                    35. Department of Veterans Affairs: VA Contract V549P-0027: Gulf War Veterans' Illnesses Research, Task Order 2. Washington: Awarded to University of Texas, Southwestern; 2007–2010.
                    36. Department of Veterans Affairs Office of Inspector General: Review of Contract No. VA549-P-0027 Between the Department of Veterans Affairs and The University of Texas Southwestern Medical Center at Dallas (UTSWMC) for Gulf War Illness Research. Washington: VA Office of Inspector General; 2009.
                    37. Smith TC, Smith B, Ryan MA, Gray GC, Hooper TI, Heller JM, Dalager NA, Kang HK, Gackstetter GD: Ten years and 100,000 participants later: occupational and other factors influencing participation in US Gulf War health registries. J Occup Environ Med 2002, 44:758–768.
                    38. Carney CP, Sampson TR, Voelker M, Woolson R, Thorne P, Doebbeling BN: Women in the Gulf War: combat experience, exposures, and subsequent health care use. Mil Med 2003, 168:654–661.
                    39. Proctor SP, Harley R, Wolfe J, Heeren T, White RF: Health-related quality of life in Persian Gulf War veterans. Mil Med 2001, 166:510–519.
                    40. Wolfe J, Proctor SP, Keane TM: Devens Cohort Study: Time 4. Boston: Boston Environmental Hazards Center and National Center for PTSD, Department of Veterans Affairs, VA Boston Healthcare System; 1997–1998.
                    41. Proctor S, Heaton KJ, White RF, Wolfe J: Chemical sensitivity and chronic fatigue syndrome in Gulf War veterans: a brief report. J Occup Environ Med 2001, 43:359–364.
                    42. Lindem K, Heeren T, White RF, Proctor SP, Krengel M, Vasterling JJ, Sutker PB, Wolfe J, Keane TM: Neuropsychological performance in Gulf War era veterans: traumatic stress symptomatology and exposure to chemical–biological warfare agents. J Psychopathol Behav 2003, 25:105–119.View Article
                    43. Heaton KJ, Palumbo CL, Proctor SP, Killiany RJ, Yurgelun-Todd DA, White RF: Quantitative magnetic resonance brain imaging in US Army veterans of the 1991 Gulf War potentially exposed to sarin and cyclosarin. Neurotoxicology 2007, 28:761–769.View Article
                    44. Sutker PB, Uddo M, Brailey K, Allain AN, Errera P: Psychological symptoms and psychiatric diagnoses in Operation Desert Storm troops serving graves registration duty. J Trauma Stress 1994, 7:159–171.View Article
                    45. Vasterling JJ, Brailey K, Tomlin H, Rice J, Sutker PB: Olfactory functioning in Gulf War-era veterans: relationships to war-zone duty, self-reported hazards exposures, and psychological distress. J Int Neuropsychol Soc 2003, 9:407–418.View Article
                    46. Sutker PB, Davis JM, Uddo M, Ditta SR: War zone stress, personal resources, and PTSD in Persian Gulf War returnees. J Abnorm Psychol 1995, 104:444–452.View Article
                    47. Vasterling JJ, Brailey K, Constans JI, Sutker PB: Attention and memory dysfunction in posttraumatic stress disorder. Neuropsychology 1998, 12:125–133.View Article
                    48. Department of Veterans Affairs. Persian Gulf Registry Code Sheet. Form 10–9009a. http://​www.​gulflink.​org/​gwr/​10-9009a.​pdf
                    49. Fiedler N, Giardino N, Natelson B, Ottenweller JE, Weisel C, Lioy P, Lehrer P, Ohman-Strickland P, Kelly-McNeil K, Kipen H: Responses to controlled diesel vapor exposure among chemically sensitive Gulf War veterans. Psychosom Med 2004, 66:588–598.View Article
                    50. Kipen HM, Hallman W, Kang H, Fiedler N, Natelson BH: Prevalence of chronic fatigue and chemical sensitivities in Gulf Registry veterans. Arch Environ Health 1999, 54:313–318.View Article
                    51. Boyd KC, Hallman WK, Wartenberg D, Fiedler N, Brewer NT, Kipen HM: Reported exposures, stressors, and life events among Gulf War Registry veterans. J Occup Environ Med 2003, 45:1247–1256.View Article
                    52. Koch TR, Emory TS: Evaluation of chronic gastrointestinal symptoms following Persian Gulf War exposure. Mil Med 2005, 170:696–700.
                    53. Eisen SA, Kang H, Murphy FM, Blanchard M, Reda DJ, Henderson W, Toomey R, Jackson L, Alpern R, Parks B, et al.: Gulf War veterans' health: medical evaluation of a U.S. cohort. Ann Intern Med 2005, 142:881–890.
                    54. Toomey R, Kang H, Karlinsky JB, Baker D, Vasterling J, Alpern R, Reda DJ, Henderson W, Murphy FM, Eisen SA: Mental health of US Gulf War veterans 10 years after the war. Brit J Psychiat 2007, 190:385–393.View Article
                    55. Toomey R, Alpern R, Vasterling J, Baker D, Reda DJ, Lyons M, Henderson W, Kang H, Eisen SA, Murphy FM: Neuropsychological functioning of U.S. Gulf War veterans 10 years after the war. J Int Neuropsych Soc 2009, 15:717–729.View Article
                    56. Wallin MT, Wilken J, Alfaro MH, Rogers C, Mahan C, Chapman JC, Fratto T, Sullivan C, Kang H, Kane R: Neuropsychologic assessment of a population-based sample of Gulf War veterans. Cogn Behav Neurol 2009, 22:155–166.View Article
                    57. Roy MJ, Koslowe PA, Kroenke K, Magruder C: Signs, symptoms, and ill-defined conditions in Persian Gulf War veterans: findings from the Comprehensive Clinical Evaluation Program. Psychosom Med 1998, 60:663–668.
                    58. Stretch RH, Marlowe DH, Wright KM, Bliese PD, Knudson KH, Hoover CH: Post-traumatic stress disorder symptoms among Gulf War veterans. Mil Med 1996, 161:407–410.
                    59. Stretch RH, Bliese PD, Marlowe DH, Wright KM, Knudson KH, Hoover CH: Psychological health of Gulf War-era military personnel. Mil Med 1996, 161:257–261.
                    60. Doebbeling BN, Jones MF, Hall DB, Clarke WR, Woolson RF, Torner JC, Burmeister LF, Snyders-Crumley T, Barrett DH, Falter KH, et al.: Methodologic issues in a population-based health survey of Gulf War veterans. J Clin Epidemiol 2002, 55:477–487.View Article
                    61. Barrash J, Denburg NL, Moser DJ, Woolson RF, Schumacher AJ, Doebbeling BN: Credibility of neuropsychological performances of Persian Gulf War veterans and military control subjects participating in clinical epidemiological research. Mil Med 2007, 172:697–707.
                    62. Bourdette DN, McCauley LA, Barkhuizen A, Johnston W, Wynn M, Joos SK, Storzbach D, Shuell T, Sticker D: Symptom factor analysis, clinical findings, and functional status in a population-based case control study of Gulf War unexplained illness. J Occup Environ Med 2001, 43:1026–1040.
                    63. Storzbach D, Rohlman DS, Anger WK, Binder LM, Campbell KA: Neurobehavioral deficits in Persian Gulf veterans: additional evidence from a population-based study. Environ Res 2001, 85:1–13.View Article
                    64. Smith TC, Wingard DL, Ryan MA, Kritz-Silverstein D, Slymen DJ, Sallis JF: PTSD prevalence, associated exposures, and functional health outcomes in a large, population-based military cohort. Public Health Rep 2009, 124:90–102.
                    65. Proctor S, Wells TS, Jones KA, Boyko EJ, Smith TC: Examination of post-service health-related quality of life among rural and urban military members of the Millennium Cohort Study. J Rur Soc Sci 2011, 26:32–56.
                    66. Cutrona CE: Social support and stress in the transition to parenthood. J Abnorm Psychol 1984, 93:378–390.View Article
                    67. Cutrona CE, Russell D: The provisions of social relationships and adaptation to stress. In Advances in Personal Relationships. 1st edition. Edited by: Jones WH, Perlman D. Greenwich: JAI Press; 1987:37–67.
                    68. Holahan CJ, Moos RH: Life stressors, personal and social resources, and depression: a 4-year structural model. J Abnorm Psychol 1991, 100:31–38.View Article
                    69. Sarason IG, Sarason BR, Shearin EN, Pierce GR: A brief measure of social support: practical and theoretical implications. J Soc Pers Relat 1987, 4:497–510.View Article
                    70. Steiner A, Raube K, Stuck AE, Aronow HU, Draper D, Rubenstein LZ, Beck JC: Measuring psychosocial aspects of well-being in older community residents: performance of four short scales. Gerontologist 1996, 36:54–62.View Article
                    71. McCubbin HI, Thompson AI, McCubbin MA: Family Assessment: Resiliency, Coping, and Adaptation-Inventories for Research and Practice. In Family Inventory of Life Events and Changes. Edited by: McCubbin HI, Patterson J, Wilson L. Madison: University of Wisconsin System; 1996:103–178.
                    72. Olson DH, Bell R, Portner J: FACES II: Family Adaptibility and Cohesion Evaluation Scale. In Family Inventories. Edited by: Olson D, McCubbin HI, Barnes LA, Musen M, Wilson M. St. Paul: Department of Family Social Science, University of Minnesota; 1982.
                    73. Kukla R, Schlenger W, Fairbank J, Hough R, Jordan B, Marmar C, Weiss D: Trauma and the Vietnam War Generation: Report of Findings from the National Vietnam Veterans' Readjustment Study (NVVRS). New York: Brunner/Mazel; 1990.
                    74. Nelson H: The National Adult Reading Test (NART): Test Manual. Windsor: NFER; 1982.
                    75. Wechsler D: Manual for the Wechsler Adult Intelligence Scale. New York: Psychological Corporation; 1945.
                    76. Shipley W: A self-administering scale for measuring intellectual impairment and deterioration. J Psychol 1940, 9:371–377.View Article
                    77. Shipley W: Institute of Living Scale. Los Angeles: Western Psychological Services; 1946.
                    78. Costa PT, McCrae RR: The NEO Personality Inventory Manual. Odessa: Psychological Assessment Resources; 1985.
                    79. Bartone PT, Ursano RJ, Wright KM, Ingraham LH: The impact of a military air disaster on the health of assistance workers. A prospective study. J Nerv Ment Dis 1989, 177:317–328.View Article
                    80. Eysenck HJ, Eysenck SBG: Manual of the Eysenck Personality Questionnaire. London: Hodder and Stoughton; 1975.
                    81. Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L: The Hopkins Symptom Checklist (HSCL): a self-report symptom inventory. Behav Sci 1974, 19:1–15.View Article
                    82. Derogatis LR, Rickels K, Uhlenhuth EH, Covi L: The Hopkins Symptom Checklist: A Measure of Primary Symptom Dimensions. In Psychological Measurements in Psychopharmacology. 7th edition. Edited by: Pichot P. Paris: Karger; 1974:79–110.
                    83. Butcher JN, Dahlstrom WG, Tellegen A, Kaemmer B: Minnesota Multiphasic Personality Inventory-2: Manual for Administration and Scoring. Minneapolis: University of Minnesota Press; 1989.
                    84. Morey L: The Personality Assessment Inventory Professional Manual. Lutz: Psychological Assessment Resources; 1991.
                    85. Andrews G, Peters L: The psychometric properties of the Composite International Diagnostic Interview. Soc Psychiatry Psychiatr Epidemiol 1998, 33:80–88.View Article
                    86. Derogatis LR: The Symptoms Checklist-90. Towson: Clinical Psychometric Research; 1983.
                    87. Robins L, Helzer J: Diagnostic interview schedule (DIS), version III-R. St. Louis: Washington University School of Medicine; 1991.
                    88. Derogatis LR, Spencer PM: The Brief Symptom Inventory (BSI): Administration, Scoring and Procedures Manual. Baltimore: Clinical Psychometric Research; 1982.
                    89. Derogatis LR, Melisaratos N: The Brief Symptom Inventory: an introductory report. Psychol Med 1983, 13:595–605.View Article
                    90. Spitzer RL, Williams J, Gibbon M: Structured Clinical Interviews for DSM-III-R, Version NP-V. New York: Biometrics Research Department, New York State Psychiatric Institute; 1987.
                    91. Kovera CA, Anger WK, Campbell KA, Binder LM, Storzbach D, Davis KL, Rohlman DS: Computer-administration of questionnaires: a health screening system (HSS) developed for veterans. Neurotoxicol Teratol 1996, 18:511–518.View Article
                    92. Spitzer RL, Kroenke K, Williams JB, Patient Health Questionnaire Primary Care Study Group: Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. JAMA 1999, 282:1737–1744.View Article
                    93. Spitzer RL, Williams JB, Kroenke K, Linzer M, deGruy FV III, Hahn SR, Brody D, Johnson JG: Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. JAMA 1994, 272:1749–1756.View Article
                    94. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J: An inventory for measuring depression. Arch Gen Psychiatry 1961, 4:561–571.View Article
                    95. Peterson RA, Reiss S: Anxiety Sensitivity Index Manual. Worthington: IDS Publishing; 1987.
                    96. Beck AT, Epstein N, Brown G, Steer RA: An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 1988, 56:893–897.View Article
                    97. Beck AT, Steer RA: Beck Anxiety Inventory Manual. San Antonio: Psychological Corporation Harcourt; 1993.
                    98. Spielberger CD, Sydeman SJ: State-Trait Anxiety Inventory and State-Trait Anger Expression. In The Use of Psychological Testing for Treatment Planning and Outcome Assessment. Edited by: Maruish ME. Hillsdale: L. Erlbaum Associates; 1994:292–321.
                    99. Delis DC, Kramer JH, Kaplan E, Ober B: California Verbal Learning Test Manual. New York: Psychological Corporation; 1987.
                    100. Rey A: L'examen clinique en psychologic. [The clinical examination in psychology]. Paris: Presses Universitaires de France; 1964.
                    101. Schmidt M: Rey Auditory Verbal Learning Test: A Handbook. Los Angeles: Western Psychological Services; 1996.
                    102. Tombaugh TN: Test of Memory and Malingering. Toronto: Multi-Health Systems, Inc; 1996.
                    103. Wechsler D: A standardized memory scale for clinical use. J Psychol 1945, 19:87–95.View Article
                    104. Wechsler D: Wechsler Memory Scale, Third Edition Manual. San Antonio: Psychological Corporation; 1997.
                    105. Warrington E: The Recognition Memory Test: Manual. London: NFER-Nelson; 1984.
                    106. Heaton R, Grant I, Matthews C: Comprehensive Norms for an Expanded Halstead-Reitan Battery: Demographic Corrections, Research Findings, and Clinical Applications. Odessa: Psychological Assessment Resources Inc; 1991.
                    107. Trahan DE, Larrabee GJ: Continuous Visual Memory Test Professional Manual. Odessa: Psychological Assessment Resources; 1988.
                    108. Barrash J, Suhr J, Manzel K: Detecting poor effort and malingering with an expanded version of the Auditory Verbal Learning Test (AVLTX): validation with clinical samples. J Clin Exp Neuropsychol 2004, 26:125–140.View Article
                    109. Reitan R, Wolfson D: The Halstead-Reitan Neuropsychological Test Battery: Theory and Clinical Interpretation. Tucson: Neuropsychology Press; 1985.
                    110. Army Individual Test Battery: Manual of Directions and Scoring. Washington: War Department, Adjutant General's Office; 1944.
                    111. Conners CK: Conners' Continuous Performance Test Computer Program User's Guide. Toronto: Multi-Health Systems; 1992.
                    112. Kay GG, Berman B, Mockoviak SH, Morris CE, Reeves D, Starbuck V, Sukenik E, Harris AG: Initial and steady-state effects of diphenhydramine and loratadine on sedation, cognition, mood, and psychomotor performance. Arch Intern Med 1997, 157:2350–2356.View Article
                    113. Broadbent DE, Cooper PF, FitzGerald P, Parkes KR: The Cognitive Failures Questionnaire (CFQ) and its correlates. Br J Clin Psychol 1982, 21:1–16.View Article
                    114. Berg EA: A simple objective technique for measuring flexibility in thinking. J Gen Psychol 1948, 39:15–22.View Article
                    115. Stroop JR: Studies of interference in serial verbal reactions. J Exp Psychol 1935, 18:643–661.View Article
                    116. Mills KC, Parkman KM, Spruill SE: A PC-based software test for measuring alcohol and drug effects in human subjects. Alcohol Clin Exp Res 1996, 20:1582–1591.View Article
                    117. Shin MS, Park SY, Park SR, Seol SH, Kwon JS: Clinical and empirical applications of the Rey-Osterrieth Complex Figure Test. Nat Protoc 2006, 1:892–899.View Article
                    118. Rey A: L’examen psychologique dans les cas d’encephalopathie traumatique [The psychological examination in cases of traumatic encephalopathy]. Arch Psychol 1941, 28:286–340.
                    119. Gronwall DM: Paced auditory serial-addition task: a measure of recovery from concussion. Percept Mot Skills 1977, 44:367–373.View Article
                    120. Benton AL, Hamsher KDS: Multilingual Aphasia Examination. 2nd edition. Iowa City: AJA Associates; 1976.
                    121. Letz R: NES2 User's Manual (version 4.4). Winchester: Neurobehavioral Systems, Inc; 1991.
                    122. Anger WK, Rohlman DS, Sizemore OJ, Kovera CA, Gibertini M, Ger J: Human behavioral assessment in neurotoxicology: producing appropriate test performance with written and shaping instructions. Neurotoxicol Teratol 1996, 18:371–379.View Article
                    123. Smith A: SDMT: A neuropsychological test for economic screening. Hear Disord 1968, 3:83–91.
                    124. Posner MI: Chronometric Explorations of Mind. Hillside: Erlbaum; 1978.
                    125. Binder LM, Willis SC: Assessment of motivation after financially compensable minor head trauma. Psychol Assess 1991, 3:175–181.View Article
                    126. Klove H: Clinical neuropsychology. Med Clin North Am 1963, 47:1647–1658.
                    127. Tiffin J: Purdue Pegboard Examiner's Manual. Rosemont: London House; 1968.
                    128. Halstead WC: Brain and Intelligence. Chicago: University of Chicago Press; 1947.
                    129. Bergner M, Bobbitt RA, Kressel S, Pollard WE, Gilson BS, Morris JR: The sickness impact profile: conceptual formulation and methodology for the development of a health status measure. Int J Health Serv 1976, 6:393–415.View Article
                    130. Gilson BS, Gilson JS, Bergner M, Bobbit RA, Kressel S, Pollard WE, Vesselago M: The sickness impact profile: development of an outcome measure of health care. Am J Public Health 1975, 65:1304–1310.View Article
                    131. Barsky AJ, Wyshak G, Klerman GL: The somatosensory amplification scale and its relationship to hypochondriasis. J Psychiatr Res 1990, 24:323–334.View Article
                    132. Pilowsky I, Spence N: Manual for the Illness Behavior Questionnaire. 3rd edition. South Australia: Department of Psychiatry, University of Adelaide; 1994.
                    133. Frisch MB: Manual and Treatment Guide for the Quality of Life Inventory. Minneapolis: National Computer Systems; 1994.
                    134. Wolfe J, Proctor SP, Erickson DJ, Hu H: Risk factors for multisymptom illness in US Army veterans of the Gulf War. J Occup Environ Med 2002, 44:271–281.View Article
                    135. Follow-up of Psychological and Neurocognitive Gulf War Outcome: Relation to Stress. http://​webcache.​googleuserconten​t.​com/​search?​q=​cache:​crtS-W6PyecJ:​http://​fhp.​osd.​mil/​deploymed/​projectDetail.​jsp%3FprojectId%3D527%26region%3D0%26researchTopic%3D2%26majorDeploymen​t%3D0%26researchSubTop​ic%3D10%2Bdeploymed+proj​ect+detail+527&​rls=​com.​microsoft%3A*&​oe=​UTF-8&​startIndex=​&​startPage=​1&​gs_​l=​heirloom-serp.​3.​.​5516.​5704.​0.​6704.​2.​2.​0.​0.​0.​0.​0.​0.​0.​0.​.​0.​0.​.​1c.​1.​Asten6c5jYk&​hl=​en&​nfpr=​1&​spell=​&​ct=​clnk
                    136. Department of Veterans Affairs: National Health Survey of Persian Gulf War Era Veterans Questionnaire. Washington: Department of Veterans Affairs; 1995.
                    137. Blanchard M, Eisen SA, Alpern R, Karlinsky JB, Toomey R, Reda DJ, Murphy FM, Jackson L, Kang H: Chronic multisymptom illness complex in Gulf War I veterans 10 years later. Am J Epidemiol 2006, 163:66–75.View Article
                    138. Kroenke K, Koslowe P, Roy M: Symptoms in 18,495 Persian Gulf War veterans: latency of onset and lack of association with self-reported exposures. J Occup Environ Med 1998, 40:520–528.View Article
                    139. Cowan DN, Lange JL, Heller J, Kirkpatrick J, DeBakey S: A case–control study of asthma among U.S. Army Gulf War veterans and modeled exposure to oil well fire smoke. Mil Med 2002, 167:777–782.
                    140. Engel CC Jr, Ursano R, Magruder C, Tartaglione R, Jing Z, Labbate LA, Debakey S: Psychological conditions diagnosed among veterans seeking Department of Defense care for Gulf War-related health concerns. J Occup Environ Med 1999, 41:384–392.View Article
                    141. Ford JD, Campbell KA, Storzbach D, Binder LM, Anger WK, Rohlman DS: Posttraumatic stress symptomatology is associated with unexplained illness attributed to Persian Gulf War military service. Psychosom Med 2001, 63:842–849.
                    142. Nisenbaum R, Barrett DH, Reyes M, Reeves WC: Deployment stressors and a chronic multisymptom illness among Gulf War veterans. J Nerv Ment Dis 2000, 188:259–266.View Article
                    143. Smith TC, Jacobson IG, Smith B, Hooper TI, Ryan MA, Team FT: The occupational role of women in military service: validation of occupation and prevalence of exposures in the Millennium Cohort Study. Int J Environ Health Res 2007, 17:271–284.View Article
                    144. Smith B, Smith TC, Gray GC, Ryan MA: When epidemiology meets the Internet: web-based surveys in the Millennium Cohort Study. Am J Epidemiol 2007, 166:1345–1354.View Article
                    145. Jacobson IG, Smith TC, Smith B, Keel PK, Amoroso PJ, Wells TS, Bathalon GP, Boyko EJ, Ryan MA: Disordered eating and weight changes after deployment: longitudinal assessment of a large US military cohort. Am J Epidemiol 2009, 169:415–427.View Article
                    146. Smith B, Chu LK, Smith TC, Amoroso PJ, Boyko EJ, Hooper TI, Gackstetter GD, Ryan MA: Challenges of self-reported medical conditions and electronic medical records among members of a large military cohort. BMC Med Res Methodol 2008, 8:37.View Article
                    147. Chyba M, Washington L: Questionnaires from the National Health Interview Survey, 1985–1989. Vital Health Stat 1993.,1(31):
                    148. Centers for Disease Control and Prevention: Behavioral Risk Factor Surveillance System Survey Questionnaire. Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention; 1995.
                    149. The University of Iowa: Female and family health questionnaire: the agricultural health study. Iowa City: University of Iowa; 1993.
                    150. Agricultural Health Study. http://​aghealth.​nci.​nih.​gov/​questionnaires.​html
                    151. Jacobson IG, White MR, Smith TC, Smith B, Wells TS, Gackstetter GD, Boyko EJ: Self-reported health symptoms and conditions among complementary and alternative medicine users in a large military cohort. Ann Epidemiol 2009, 19:613–622.View Article
                    152. Cameron L, Leventhal EA, Leventhal H: Seeking medical care in response to symptoms and life stress. Psychosom Med 1995, 57:37–47.
                    153. Department of Veterans Affairs: Longitudinal Health Study of Persian Gulf War Era Veterans Questionnaire. Washington: Department of Veterans Affairs; 2002.
                    154. Stouffer S, Guttman L, Suchman E: Studies in Social Psychology in World War II: Measurement and Prediction. Princeton: Princeton University Press; 1950.
                    155. Bradburn N: The Structure of Psychological Well-Being. Chicago: Aldine; 1969.
                    156. Kroenke K, Spitzer RL, Williams JB: The PHQ-15: validity of a new measure for evaluating the severity of somatic symptoms. Psychosom Med 2002, 64:258–266.
                    157. Lange JL, Schwartz DA, Doebbeling BN, Heller JM, Thorne PS: Exposures to the Kuwait oil fires and their association with asthma and bronchitis among Gulf War veterans. Environ Health Perspect 2002, 110:1141–1146.View Article
                    158. Ferris BJ: Epidemiology standardization project. Am Rev Respir Dis 1978, 118:55–88.
                    159. Ware JE Jr, Sherbourne CD: The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 1992, 30:473–483.View Article
                    160. Ware JE, Kosinski M, Dewey J: How to Score Version 2 of the SF-36 Health Survey. Lincoln: Quality Metric, Inc; 2000.
                    161. Ware JE Jr, Snow K, Kosinski M, Gandeck B: SF-36 Health Survey: Manual and Interpretation Guide. Boston: The Health Institute, New England Medical Center; 1993.
                    162. Ware JE Jr: SF-36 health survey update. Spine 2000, 25:3130–3139.View Article
                    163. Ware JE, Sherbourne CD, Davies AR: A Short-Form Health Survey (SF-36). New York: RAND Corporation; 1988.
                    164. Feeny D, Furlong W, Torrance GW, Goldsmith CH, Zhu Z, DePauw S, Denton M, Boyle M: Multiattribute and single-attribute utility functions for the health utilities index mark 3 system. Med Care 2002, 40:113–128.View Article
                    165. Forman-Hoffman VL, Carney CP, Sampson TR, Peloso PM, Woolson RF, Black DW, Doebbeling BN: Mental health comorbidity patterns and impact on quality of life among veterans serving during the first Gulf War. Qual Life Res 2005, 14:2303–2314.View Article
                    166. Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessely S, Wright D, Wallace EP: Development of a fatigue scale. J Psychosom Res 1993, 37:147–153.View Article
                    167. Krupp LB, LaRocca NG, Muir-Nash J, Steinberg AD: The fatigue severity scale. Application to patients with multiple sclerosis and systemic lupus erythematosus. Arch Neurol 1989, 46:1121–1123.View Article
                    168. Smets EM, Garssen B, Bonke B, De Haes JC: The Multidimensional Fatigue Inventory (MFI): psychometric qualities of an instrument to assess fatigue. J Psychosom Res 1995, 39:315–325.View Article
                    169. Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, Komaroff A, International Chronic Fatigue Syndrome Study Group: The chronic fatigue syndrome: a comprehensive approach to its definition and study. Ann Intern Med 1994, 121:953–959.
                    170. Wolfe F, Ross K, Anderson J, Russell IJ, Hebert L: The prevalence and characteristics of fibromyalgia in the general population. Arthritis Rheum 1995, 38:19–28.View Article
                    171. Wolfe F, Smythe HA, Yunus MB, Bennett RM, Bombardier C, Goldenberg DL, Tugwell P, Campbell SM, Abeles M, Clark P, et al.: The American College of Rheumatology 1990 Criteria for the Classification of Fibromyalgia: Report of the Multicenter Criteria Committee. Arthritis Rheum 1990, 33:160–172.View Article
                    172. Szarek M, Bell I, Schwartz G: Validation of a brief screening measure of environmental chemical sensitivity: the Chemical Odor Intolerance Index. J Environ Psychol 1997, 17:345–351.View Article
                    173. Kreutzer R, Neutra R, Lashuay N: Prevalence of people reporting sensitivities to chemicals in a population-based survey. Am J Epidemiol 1999, 150:1–12.View Article
                    174. Black DW, Doebbeling BN, Voelker MD, Clarke WR, Woolson RF, Barrett DH, Schwartz DA: Multiple chemical sensitivity syndrome: symptom prevalence and risk factors in a military population. Arch Intern Med 2000, 160:1169–1176.View Article
                    175. Kang H, Mahan CM, Murphy FM, Lee K, Simmens SJ, Young H, Levin P: Evidence of a deployment-related Gulf War syndrome by factor analysis. Arch Environ Health 2002, 57:61–68.View Article
                    176. Spencer PS, McCauley LA, Joos SK, Lasarev MR, Schuell T, Bourdette D, Barkhuizen A, Johnston W, Storzbach D, Wynn M, et al.: U.S. Gulf War veterans: service periods in theater, differential exposures, and persistent unexplained illness. Toxicol Lett 1998, 102–103:515–521.View Article
                    177. Committee on Government Reform and Oversight: Gulf War Veterans' Illnesses: VA, DOD continue to resist strong evidence linking toxic causes to chronic health effects. HR 105–388. Washington: U.S. Government Printing Office; 1997.
                    178. Blake DD, Weathers FW, Nagy L, Kaloupek DG, Klauminzer G, Charney DS: Clinician Administered PTSD Scale. West Haven: National Center for PTSD, Behavioral Sciences Division; 1990.
                    179. Blake DD, Weathers FW, Nagy LM, Kaloupek DG, Gusman FD, Charney DS, Keane TM: The development of a Clinician-Administered PTSD Scale. J Trauma Stress 1995, 8:75–90.View Article
                    180. Weathers FW, Huska JA, Keane TM: The PTSD Checklist- Military Version (PCL-M). Boston: National Center for PTSD; 1991.
                    181. Weathers FW, Ford J: Psychometric Review of PTSD Checklist (PCL-C, PCL-S, PCL-M, PCL-PR). In Measurement of Stress, Trauma, and Adaptation. Edited by: Stamm BH. Lutherville: Sidran Press; 1996.
                    182. Wolfe J, Erickson DJ, Sharkansky EJ, King DW, King LA: Course and predictors of posttraumatic stress disorder among Gulf War veterans: a prospective analysis. J Consult Clin Psychol 1999, 67:520–528.View Article
                    183. Keane TM, Caddell JM, Taylor KL: Mississippi Scale for Combat-Related Posttraumatic Stress Disorder: three studies in reliability and validity. J Consult Clin Psychol 1988, 56:85–90.View Article
                    184. Spitzer RL, Williams JB, Gibbon M, First M: Structured Clinical Interview for DSM-III-R Patient Edition (SCID-P). New York: Biometrics Research Department, New York State Psychiatric Institute; 1989.
                    185. Horowitz M, Wilner N, Alvarez W: Impact of Event Scale: a measure of subjective stress. Psychosom Med 1979, 41:209–218.
                    186. Phillips CJ, Matyas GR, Hansen CJ, Alving CR, Smith TC, Ryan MA: Antibodies to squalene in US Navy Persian Gulf War veterans with chronic multisymptom illness. Vaccine 2009, 27:3921–3926.View Article
                    187. Chyba MM, Washington LR: Questionnaires from the National Health Interview Survey, 1985–89. Vital Health Stat 1993, 1:1–412.
                    188. Helmer DA, Flanagan ME, Woolson RF, Doebbeling BN: Health services use among Gulf War veterans and Gulf War era nondeployed veterans: a large population-based survey. Am J Public Health 2007, 97:2145–2148.View Article
                    189. Edwards W, Berlin M: Questionnaires and Data Collection Methods for the Household Survey and the Survey of American Indians and Alaska Natives. PHS 89–3450. Rockville: Public Health Service; 1989. [In National Medical Expenditure Survey, Methods 2]
                    190. Black DW, Doebbeling BN, Voelker MD, Clarke WR, Woolson RF, Barrett DH, Schwartz DA: Quality of life and health-services utilization in a population-based sample of military personnel reporting multiple chemical sensitivities. J Occup Environ Med 1999, 41:928–933.View Article
                    191. Gallops M, Laufer R, Yager T: Revised Combat Scale. In Legacies of Vietnam: Comparative Adjustments of Veterans and Their Peers. 3rd edition. Edited by: Laufer R, Yager T. Washington: US Government Printing Office; 1981.
                    192. Rosenheck R, Becnel H, Blank A, Farley F, Fontana A, Friedman MJ, Fulton J, Gelsomino J, Grishman M, Gusman FD, et al.: Returning Persian Gulf Troops: First-year Findings. Report of the Department of Veterans Affairs to the United States Congress on the Psychological Effects of the Persian Gulf War. West Haven: Evaluation Division of the National Center for PTSD, Department of Veterans Affairs; 1992.
                    193. Keane TM, Fairbank J, Taylor KL, Mora C: Brief reports: clinical evaluation of a measure to assess combat exposure. Psychol Assessment 1989, 1:53–55.View Article
                    194. Wolfe J: Applying principles of critical incident debriefing to the therapeutic management of acute combat stress. Boston: National Center for PTSD; 1990.
                    195. Coughlin SS, Kang HK, Mahan CM: Alcohol use and selected health conditions of 1991 Gulf War veterans: survey results, 2003–2005. Prev Chronic Dis 2011, 8:A52.
                    196. Jacobson IG, Ryan MA, Hooper TI, Smith TC, Amoroso PJ, Boyko EJ, Gackstetter GD, Wells TS, Bell NS: Alcohol use and alcohol-related problems before and after military combat deployment. JAMA 2008, 300:663–675.View Article
                    197. Ewing JA: Detecting alcoholism: the CAGE questionnaire. JAMA 1984, 252:1905–1907.View Article
                    198. Smith B, Ryan MA, Wingard DL, Patterson TL, Slymen DJ, Macera CA: Cigarette smoking and military deployment: a prospective evaluation. Am J Prev Med 2008, 35:539–546.View Article
                    199. Mahan CM, Kang H, Dalager N, Heller JM: Anthrax vaccination and self-reported symptoms, functional status, and medical conditions in the National Health Survey of Gulf War Era Veterans and Their Families. Am J Epidemiol 2004, 14:81–88.
                    200. Black DW, Carney CP, Forman-Hoffman VL, Letuchy E, Peloso P, Woolson RF, Doebbeling BN: Depression in veterans of the first Gulf War and comparable military controls. Ann Clin Psychiatry 2004, 16:53–61.View Article
                    201. van der Steen JT, Kruse RL, Szafara KL, Mehr DR, van der Wal G, Ribbe MW, D'Agostino RB: Benefits and pitfalls of pooling datasets from comparable observational studies: combining US and Dutch nursing home studies. Palliat Med 2008, 22:750–759.View Article
                    202. Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C: Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 1999, 28:1–9.View Article
                    203. Sutton AJ, Higgins JPT: Recent developments in meta-analysis. Stat Med 2008, 27:625–650.View Article

                    Copyright

                    © McNeil et al; licensee BioMed Central Ltd. 2013

                    This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://​creativecommons.​org/​licenses/​by/​2.​0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

                    Advertisement