Epidemiologic Perspectives & Innovations - Latest Comments

Re: Estimating uncertainty in observational studies of associations between continuous variables: Example of methylmercury and neuropsychological testing in children. Reply by M. Goodman, P. Mink and L. Barraj

Michael Goodman — 2007-12-06T00:00:00Z

We thank Drs. Budtz-Jørgensen, Keiding and Grandjean for their comments. The following is our response to the specific concerns raised in their letter.

1. … the paper rests on an apparent disagreement between the findings of epidemiological studies carried out in the Seychelles and the Faroe Islands. However, we have previously shown that the results of the Seychelles study are sufficiently variable to be in statistical agreement with a mercury-associated deficit on the Boston Naming Test reported from the Faroe Island.

RESPONSE: Systematic errors are unavoidable in observational epidemiology, and it is reasonable to assume that neither the Seychelles Child Development Study (SCDS) nor the Faroe Islands Study (FIS) are immune to imperfections of study design and implementation. For this reason it is only fair that our sensitivity analyses addressed uncertainty in both studies.

2. Goodman et al. could have reached the same conclusion, had they converted the cord blood regression coefficient from the Faroes study to the hair concentration scale (multiply by 5) for comparison with the regression coefficient of the Seychelles study. A combination of the two estimates would generate a statistically significant effect, as was also reported by Axelrad et al. based on a meta-analysis of the studies in New Zealand, Seychelles, and Faroes.

RESPONSE: Rescaling in the Axelrad (2007) paper did not change the interpretation of the Boston Naming Test result for the SCDS study [1]. Using the original scale, the β estimate was -0.012 with a standard error of 0.046; after rescaling, the corresponding estimate changed to -0.038 (SE 0.144). Performing a meta-analysis on the two studies does not eliminate, but merely hides, the disagreement between FIS and SCDS results. The New Zealand study did not use BNT.

3. The analyses of the Seychelles and the Faroe Island data have included a large number of covariates. The mercury effect estimate originally reported from the Faroes changed only negligibly by including a large number of additional covariates including many of those Goodman et al. consider important.

RESPONSE: We agree that the impact of unaccounted confounding alone may be modest; this was confirmed by our sensitivity analyses. However, as shown in Figures 1 and 2 in our paper, when combined with other modest sources of bias the overall error may be substantial.

4. The analyses carried out by Goodman et al. fail to take into account the effects of the covariates already adjusted for, and the calculations erroneously indicate that unbiased effect estimation can be achieved without knowing the relationship between the confounders present and the unmeasured confounder.

RESPONSE: We used adjusted effect estimates in our sensitivity analyses. Our paper does not indicate that that unbiased effect estimation can be achieved without knowing the relationship between the confounders present and the unmeasured confounder. Unfortunately, we did not have information on relationship between the confounders present and the unmeasured confounder. This particular caveat of our sensitivity analyses is discussed in the article along with other limitations.

5. The authors further assume that the unmeasured confounder has an unrealistically high correlation with both exposure and outcome variable. Assumptions on information bias and selection bias are also extreme.

RESPONSE: The differences in exposure levels between participants and non-participants in the FIS have been reported [2, 3] and, in fact, exceed the differences assumed in our selection bias simulation. The reported participation rate in the SCDS also falls within the proposed scenarios [4]. We demonstrated the potential effect of confounding by home environment and the need for a comprehensive parental IQ evaluation in an earlier publication [5]. The correlation coefficients between potential confounders and exposure are similar to those reported in the FIS. The potential misclassification due to fatigue, timing and sequencing of testing and lack of adequate blinding also finds support in the literature [6, 7].

6. Preliminary data from the Seychelles suggest that adjustment for beneficial effects of fish intake in that population will also result in mercury-associated deficits becoming statistically significant, in accordance with the findings from the Faroes.

RESPONSE: The relevance of the abstract by Strain et al (2007) is unclear as it does not appear to examine the BNT results [8]. We are pleased, however, that Strain and colleagues felt it was necessary to examine uncertainty associated with previously unaccounted confounding.

In conclusion, we would like to re-emphasize that our sensitivity analyses aimed to estimate the potential systematic error for each study and in either direction. We are well aware that sensitivity analyses that are based on assumptions may be less informative than those based on real data. Nevertheless, in the absence of sensitivity analyses, one implicitly assumes that systematic error had no effect on the study results, an assumption that is even more difficult to defend. We hope that the methods of our sensitivity analyses will prove informative and helpful to researchers who wish to examine uncertainty in their data or in published studies by other researchers. Whether or not our findings add to the “current understanding of developmental neurotoxicity caused by maternal methylmercury exposure” is, of course, a matter of opinion.

References

1. Axelrad DA, Bellinger DC, Ryan LM, Woodruff TJ: Dose-response relationship of prenatal mercury exposure and IQ: an integrative analysis of epidemiologic data. Environ Health Perspect 2007, 115:609-615.

2. Grandjean P, Weihe P, White RF, Debes F, Araki S, Yokoyama K, Murata K, Sorensen N, Dahl R, and Jorgensen PJ.: Cognitive deficit in 7-year-old children with prenatal exposure to methylmercury. Neurotoxicol Teratol 1997, 19:417-428.

3. Grandjean P, Weihe P: Neurobehavioral effects of intrauterine mercury exposure: potential sources of bias. Environ Res 1993, 61:176-183.

4. Marsh D, Clarkson TW, Myers GJ, Davidson PW, Cox C, Cernichiari E, Tanner MA, Lednar W, Shamlaye C, Choisy O, Hoareau C, Berlin M: The Seychelles study of fetal methylmercury exposure and child development: Introduction. Neurotoxicology 1995, 16:583-596.

5. Mink PJ, Goodman M, Barraj LM, Imrey H, Kelsh MA, Yager J: Evaluation of uncontrolled confounding in studies of environmental exposures and neurobehavioral testing in children. Epidemiology 2004, 15:385-393.

6. Baron I: Neuropsychological Evaluation of the Child. New York: Oxford University Press; 2004.

7. Sattler J: Assessment of Children: Cognitive Applications. 4th edn. San Diego: Jerome M. Sattler, Publisher, Inc.; 2001.

8. Strain J, Bonham M, Davidson P, Myers G, Thurston S, Clarkson T, Stokes-Riner A, Janciuras J, Sloane-Reeves J, Cernichiari E, et al: Long-chain polyunsaturated fatty acids and mercury. Presented at the International Conference on Fetal Programming and Developmental Toxicity; Faroe Islands. 2007

Second response by M. Goodman, L. Barraj, P. Mink and D. Flanders

Michael Goodman — 2007-12-06T00:00:00Z

It is not correct that we “assumed a correlation of 0.8 between an unmeasured confounder and the BNT response.” Rather we assumed a range of correlations with a minimum of 0.2 and a maximum of 0.8. Budtz-Jorgensen and colleagues suggest that unaccounted confounding did not affect their study results, thereby offering an assumption of their own. We accepted this assumption and re-ran the sensitivity analyses to correct only for information and selection bias. The results were as follows.

For the Faroe Islands Study (FIS) after adjusting for information and selection bias the slope (SE) changed from -0.0190 (0.0063) as originally reported by the FIS group to -0.0022 (0.0392). As a reminder, adjustment for selection, information and confounding bias in our paper resulted in a corrected FIS slope of -0.0024 (0.0439). For the Seychelles Child Development Study (SCDS) the observed slope was -0.0120 (0.0460), adjustment for information and selection bias changed the result to -0.0122 (0.2000). In our paper, further correction of SCDS results for confounding produced a slope of -0.0132 with a SE of 0.2240.

In summary, even if one were to argue that the effect of unaccounted confounding should be ignored, it is important to remember that the overall uncertainty associated with systematic error may be a result of bias from multiple sources. Thus, even in the absence of any unaccounted confounding, our conclusions remain essentially the same.

rejoinder

Esben Budtz-Jorgensen — 2007-12-06T00:00:00Z

Aurthors: Esben Budtz-Jørgensen and Niels Keiding (Department of Biostatistics, University of Copenhagen, Denmark)

Philippe Grandjean (University of Southern Denmark, Odense, Denmark; and Harvard School of Public Health, Boston, MA, USA)

As a rejoinder to Goodman et al.'s response, we would agree that sensitivity analyses can be useful when evaluating evidence from observational studies. However, such calculations are informative only if based on realistic assumptions about the unknown parameters. Thus, an observed exposure-response relationship can always be removed by postulating the presence of a strong unmeasured confounder. Correlations between the confounder and the exposure and response parameters should therefore be realistic and based on available evidence. In their analysis of the Faroese data, Goodman et al. assume a correlation of 0.8 between an unmeasured confounder and the BNT response. Accordingly, the confounder would explain 64% of the variation of this outcome. This assumption is extreme. In our recent publication (1), we included an extended set of 20 potential confounders, but, together with mercury exposure, they accounted for less than 24% of the variation. In their response, Goodman et al. fail to comment on their misleading assumption. Rather than raising questions on the validity of the Faroes results, their study suggests that unrealistic assumptions are required to explain away the significant impacts of developmental methylmercury exposure.

References

1.Budtz-Jørgensen E, Keiding N, Grandjean P, Weihe P: Confounder selection in environmental epidemiology: Assessment of health effects of prenatal mercury exposure. Ann Epidemiol 2007, 17: 27-35.

Extreme assumptions are unnecessary

Esben Budtz-Jorgensen — 2007-11-14T00:00:00Z

Authors: Esben Budtz-Jørgensen and Niels Keiding (Department of Biostatistics, University of Copenhagen, Denmark)

Philippe Grandjean (University of Southern Denmark, Odense, Denmark; and Harvard School of Public Health, Boston, MA, USA)

The paper by Goodman et al. (1) aims at modeling the impact of extreme uncertainties on the results of an observational study, in this case the risk of developmental neurotoxicity associated with maternal dietary exposure to methylmercury during pregnancy. Thus, the paper rests on an apparent disagreement between the findings of epidemiological studies carried out in the Seychelles and the Faroe Islands. However, we have previously shown that the results of the Seychelles study are sufficiently variable to be in statistical agreement with a mercury-associated deficit on the Boston Naming Test reported from the Faroe Islands (2). Goodman et al. could have reached the same conclusion, had they converted the cord blood regression coefficient from the Faroes study to the hair concentration scale (multiply by 5) for comparison with the regression coefficient of the Seychelles study. A combination of the two estimates would generate a statistically significant effect, as was also reported by Axelrad et al. (3) based on a meta-analysis of the studies in New Zealand, Seychelles, and Faroes.

The analyses of the Seychelles and the Faroe Island data have included a large number of covariates. The mercury effect estimate originally reported from the Faroes (4) changed only negligibly by including a large number of additional covariates (5,6) including many of those Goodman et al. consider important. The analyses carried out by Goodman et al. fail to take into account the effects of the covariates already adjusted for, and the calculations erroneously indicate that an unbiased effect estimation can be achieved without knowing the relationship between the confounders present and the unmeasured confounder. The authors further assume that the unmeasured confounder has an unrealistically high correlation with both exposure and outcome variable. It is no surprise that the result of an epidemiological study is sensitive to assumptions about such a very strong confounder. Assumptions on information bias and selection bias are also extreme.

Recent evidence suggests that the previously published analyses of the data may have underestimated the effect of mercury exposure. We have examined the confounding effect from beneficial influences of fish nutrients and found that adjustment for maternal fish intake during pregnancy leads to an increase in the estimated mercury toxicity (7). Preliminary data from the Seychelles suggest that adjustment for beneficial effects of fish intake in that population will also result in mercury-associated deficits becoming statistically significant (8), in accordance with the findings from the Faroes. We have previously argued that the apparent difference between population studies in New Zealand, Seychelles, and Faroes, is likely due to uncertainties, with little reason for controversy (9). This was confirmed by the meta-analysis of Axelrad et al. (3).

Adjustment for imprecision in covariates and the confounding effects of fish intake would likely make the studies even more similar and result in stronger effect estimates.

The calculations presented by Goodman et al. (1) are therefore of little relevance to our current understanding of developmental neurotoxicity caused by maternal methylmercury exposure.

Email: Esben Budtz-Jørgensen* - E.Budtz-Joergensen@biostat.ku.dk ; Niels Keiding - nk@biostat.ku.dk; Philippe Grandjean - PGrandjean@health.sdu.dk.

* Corresponding author

References

1.Goodman M, Barraj LM, Mink PJ, Britton NL, Yager JW, Flanders WD, Kelsh MA: Estimating uncertainty in observational studies of associations between continuous variables: example of methylmercury and neuropsychological testing in children. Epidemiol Perspect Innov 2007, 4:9

2.Keiding N, Budtz-Jørgensen E, Grandjean P: Prenatal methylmercury exposure in the Seychelles (letter). Lancet 2003, 362: 664-5.

3.Axelrad DA, Bellinger DC, Ryan LM, Woodruff TJ: Dose-response relationship of prenatal mercury exposure and IQ: An integrative analysis of epidemiological data. Environ Health Perspect 2007, 115: 609-15.

4.Grandjean P, Weihe P, White RF, Debes F, Araki S, Yokoyama K, Murata K, Sørensen N, Dahl R, Jørgensen PJ: Cognitive deficit in 7-year-old children with prenatal exposure to methylmercury. Neurotoxicol Teratol 1997, 19: 417-28.

5.Budtz-Jørgensen E, Keiding N, Grandjean P, Weihe P: Confounder selection in environmental epidemiology: Assessment of health effects of prenatal mercury exposure. Ann Epidemiol 2007, 17: 27-35.

6.Choi AL, Budtz-Jørgensen E, Jørgensen PJ, Steuerwald U, Debes F, Weihe P, Grandjean P: Selenium as a potential protective factor against mercury developmental neurotoxicity. Environ Res 2007 (epub).

7.Budtz-Jørgensen E, Grandjean P, Weihe P: Separation of risks and benefits of seafood intake. Environ Health Perspect 2007, 115: 323-7.

8.Strain JJ, Bonham MP, Davidson PW, Myers GJ, Thurston SW, Clarkson TW, Stokes-Riner A, Janciuras J, Sloane-Reeves J, Cernichiari E, Shamlaye CF, Duffy EM, Robson PJ, Wallace JMW. Long-chain polyunsaturated fatty acids and mercury (abstract 39). Presented at International Conference on Fetal Programming and Developmental Toxicity, Faroe Islands, 20-24 May, 2007 [http://www.pptox.dk]

9.Grandjean P. Mercury Risks: Controversy or just uncertainty? Publ Health Rep 1999, 114: 512-5.

Klemper's Biased Judgement

Stanley Gage — 2006-11-17T00:00:00Z

Klemper, in extrapolating the results of his study, argues in light of his results, that a 90 day antibiotic treatment does not provide any noticalbe improvement in patient health and, therefore, extended antibiotic treatment is of no value.

The conclusion fails to take notice of the fact that an equally vaild argument based on this study outcome would be that a 90 day antibiotic treatment is insufficient to provide any improvement in patient health and that longer duration antibiotic treament may be necesary to reduce or eliminate the efects of desseminatd Lyme.

The real upshot of the issue of the results as published is to question that they were published at all. Publishing bad science is worse than publishing nothing. In the case of the Klemper article the author/researcher obviously approaced the issue with the hypothesis that desseminated or chroninc Lyme does not exist. By fabricating a poorly constructed experiment and then deriving biased and incomplete conclusions and extensions of those conclusion he does the medical and scientific community a significant disservice.

It is unclear in the information presented if the original Klemper study was peer reviewed prior to publication but it is clear that such poorly conducted research does not belong in the respected journals of medical literature.

Thanks for publishing the exellent article, Generalizability in two clinical trials of Lyme disease

Marisa Battilana — 2006-11-17T00:00:00Z

Thanks you for publishing Dr. Cameron's exellent article, Generalizability in two clinical trials of Lyme disease.

In my nearly two years as a volunteer in the Lyme community, I have met over 2,000 people who either have been personally affected by late stage Lyme disease or who know someone who has. The majority of the folks affected benefited greatly from extended treatment with antibiotics, most for more than just a few months. Many found out the hard way that they needed more than short term treatment: when taken off antibiotics, they relapsed within a short time. And, when they resumed treatment, they saw significant improvement.

Articles like Dr. Cameron's are long overdue. Thank you.

Marisa Battilana

NIH Headline Still Stands - Long Enough?

Phyllis Mervine — 2006-11-17T00:00:00Z

Five years after the original flawed description of the New England Medical Center [aka Klempner] clinical trial was posted on NIH websites, thanks are due to Dr. Cameron for providing compelling evidence why it should be changed to “Chronic Lyme Disease Study Shows 3 Months of Antibiotic Treatment Inadequate.”

As one of the patient representatives on the Advisory Panel for the NIH Clinical Trials, I so requested our NIH Lyme program officer Phil Baker in September 2001. Referring to other Panel members who had discussed the limited applicability of this treatment trial, I wrote: “We all know that the NEMC study was not designed to prove, could not prove and did not prove what your current headline suggests that it did prove. Patients are already being affected by the biased reporting of the study results. It would be a serious ethical lapse to allow the biased headline to remain unaltered while doctors and insurance companies use it to justify denials of treatment to seriously ill patients.”

Dr. Baker did not reply and the headline still stands. Five years later patients are still being denied treatment on this basis. Have we suffered enough yet?

NIH News Release: Chronic Lyme Disease Symptoms Not Helped by Intensive Antibiotic Treatment, 12 June 2001

http://www.nih.gov/news/pr/jun2001/niaid-12.htm

NIH NIAID: Clinical Alert: Chronic Lyme Disease Symptoms Not Helped by Intensive Antibiotic Treatment, 12 Jun 01

http://www.nlm.nih.gov/databases/alerts/lyme.html

Chronic Lyme Disease Daniel Cameron Study

Peggy McCann — 2006-11-17T00:00:00Z

Thank you for publishing Daniel Cameron's review on the Kempler clinical trials of lyme disease.

The new IDSA guidelines are based on VERY selective research studies such as Kempler's paper. They conveniently and/or deliberately ignored over 18,000 conflicting articles and research studies published on Medline.

I am very upset by the guideline's arrogant dismissal of patients who don’t get better in the time span and with the dose of antibiotics they deem appropriate. So those unfortunate enough to continue suffering multiple symptoms that don’t conform to their blue-print, are left without hope for getting their life back.

Do you have any idea how awful that scenario is to patients and their families? Or to the family physicians who are told to treat no further, even though their patient is improving but still ill? To Lyme literate doctors and researchers who are either ignored or threatened with sanctions against their practice?

What is going on? Why?

If all doctors follow these guidelines(which reflect conclusions drawn from Kempler's extremely biased study) we will soon see a catastrophic rise in people with irreversible and lifelong neurological and physical symptoms that could have been prevented or at least, the damage minimized and controlled.

Long-term antibiotics may be necessary and I for one will gladly suffer any antibiotic “risk” to get my life back again.