EPA Data Transparency, RIP

Peter Van Doren

The Washington Post reported last month on the early demise of the EPA scientific data transparency rule from procedural irregularities. The rule was less than three years old.

The rule sounds perfectly reasonable. It would have required the Environmental Protection Agency to use scientific research when setting pollution exposure standards only if the original data were publicly available, allowing other researchers to examine and replicate findings.

Increased transparency in data used in empirical research and the facilitation of replication of studies are like mom and apple pie. In an ideal world, such practices are the very essence of the scientific method. In practice, academic journals in many disciplines already require that data used in empirical and experimental work be available for replication.

The EPA data in question, studies on the health impacts of pollution, do include confidential health information. But researchers across a wide swath of disciplines have found ways to release similar data while still shielding the identities of people included in their studies. The International Committee of Medical Journal Editors, no strangers to the limitations of data that include personal information, affirmed their commitment to responsible data sharing.

But the struggle over transparency in environmental research used by the EPA isn’t really about transparency. It is the latest battle in a decades-long war over the appropriate level of clean air exposure standards. On one side, environmental and public health advocates argue that pollution standards should be lowered, primarily based on the results of two studies. On the other side, Republicans and the Trump administration contend that the two studies are flawed and that EPA standards should be maintained at their current level.

As I explained when the data transparency rule was first proposed:

The Harvard Six Cities Study (SCS) and the American Cancer Society Study (ACS), both published in the 1990s, provide the data on which the EPA estimates all mortality risks from air pollution. Both studies looked at the health damage caused by particulate matter (PM), which accounts for 90 percent of the estimated benefits from emission regulation, and found that higher levels of PM exposure were associated with increased mortality. And these studies both rely on individual health data given under conditions of confidentiality.

So the opponents are probably correct that the transparency rule is a clever attempt to undermine the current basis for EPA regulation of PM. And reduced PM exposure is the exclusive basis on which current conventional pollution regulation is justified because the benefits of additional emissions controls on other conventional pollutants are low. In this environmentalists’ nightmare, a transparency mandate ends additional regulation of air quality by the EPA.

But contrary to the Times assertion, this a not simply a tale of good science versus evil polluters. The SCS has been the subject of intense scientific scrutiny and much criticism because of results that are biologically puzzling. The increased mortality was found in men but not women, in those with less than high school education but not more, and those who were moderately active but not sedentary or very active. Among those who migrated away from the six cities, the PM effect disappeared. Cities that lost population in the 1980s were rust belt cities that had higher PM levels and those who migrated away were younger and better educated. Thus, had the migrants stayed in place it is possible that the observed PM effect would have been attenuated.

Furthermore, a survey of 12 experts (including 3 authors of the ACS and SCS) asked whether concentration-response functions between PM and mortality were causal. Four of the 12 experts attached nontrivial probabilities to the relationship between PM concentration and mortality not being causal (65 percent to 10 percent). Three experts said there is a 5 percent probability of noncausality. Five said a 0-2 percent probability of noncausality. Thus 7 out of the 12 experts would not reject the hypothesis that there is no causality between PM levels and mortality. Based on these findings, a 95 percent confidence interval would include zero mortality effect for any reductions in PM concentration below 16 micrograms per cubic meter.

So the scientific fragility of the two studies has been known for some time. Despite that fragility, in December 2012, EPA set a much lower fine-PM standard of 12 micrograms per cubic meter of air to be met by 2020.

In December 2020, the Trump EPA retained the December 2012 standard despite an EPA staff recommendation of between 8 and 9 micrograms per cubic meter.

The EPA does not have to revise the standard for another five years so unless the Biden administration accelerates the timetable for revisiting the fine-PM standard, the end of the brief life of the transparency rule will not have any immediate air-quality regulatory consequences. But the next time the EPA sets fine-PM standards, the dispute about the use of the Harvard and American Cancer studies will recur.

If the EPA used other studies, the estimated benefits from PM exposure reduction are reduced. This study, for example, concluded that declining PM concentrations from 1999 to 2011 resulted in an additional 150,000 life-years per year, which, if valued at $100,000 per life-year, would equal $15 billion in annual benefits. The EPA estimates that the annual compliance costs of the 1990 Clean Air Act standards were $44 billion in 2010. With these lopsided costs and benefits, it is certainly true that eliminating the ACS and SCS studies from consideration and forcing reliance on other studies would result in less stringent regulation except in areas with bad geography, such as Los Angeles, that prevents pollution dispersion.

So, given that the outcomes of cost-benefit analyses about PM exposure will vary depending on which studies from the refereed literature one uses, the wrestling match over the role of these different studies in EPA decisions will continue unless Congress eliminates EPA discretion to set standards. The birth and death of the transparency rule is simply the latest installment of the fifty-year fight over appropriate stationary source pollution control.