Research Highlights Podcast
April 16, 2025
Reexamining air quality regulations
Lutz Sager and Gregor Singer discuss the Clean Air Act and how to estimate its impact on air quality.
Source: orkhv
The Clean Air Act has been an essential tool for reducing air pollution in the United States. But standard estimation methods may overstate its impact, according to a paper in the American Economic Journal: Economic Policy.
Authors Lutz Sager and Gregor Singer reexamined the 2005 regulations targeting fine particulate matter (PM2.5) and found that improvements in air quality were closer to a 3 percent reduction in pollutants rather than the 10 percent suggested by conventional methods. However, they also found that the benefits from cleaner air may be larger than previous estimates suggested.
Sager and Singer recently spoke with Tyler Smith about methods for properly estimating regulatory impacts that feature time trends and the implications for other measures based on estimates of air quality improvements.
The edited highlights of that conversation are below, and the full interview can be heard using the podcast player.
Tyler Smith: What did the Clean Air Act regulations you studied require, and who did they affect?
Lutz Sager: What we are actually studying here is the latest round of regulations under the Clean Air Act. These came into effect in 2005, and they targeted a pollutant that's called PM2.5, which stands for particulate matter that's smaller than 2.5 micrometers in diameter. It's a type of pollutant that has been linked to some of the worst health and well-being damages in the literature. At the federal level, the Environmental Protection Agency defines a certain standard level or cutoff value for the concentration of this pollutant. And then it has a ground network of pollution monitors where it measures the level of this PM2.5 in the air. If for a certain amount or number of years, the average value of the pollution concentration is higher than that threshold, then that triggers the regulation. Those areas where the regulation is triggered are designated “nonattainment” areas because they are in violation of the standard. And then the EPA asks the state to submit a plan, a so-called state implementation plan, where they actually lay out the different measures they will take to bring their problem areas down below the threshold. In 2005, about 208 counties were in violation of the standard. That may not sound like much—it's less than 10 percent of all the counties—but they happen to be the most polluted counties where more people live and more economic activity and pollution takes place. Overall, roughly 90 million people were targeted by the regulation.
Smith: Could you explain in simple terms what this difference-in-differences approach is, and why this approach might not be appropriate in the context that you're studying?
Sager: Some regions get hit by the regulation, the so-called nonattainment areas, and then other regions are not subject to the extra regulation. What difference-in-differences does is more or less a simple before and after comparison. We look at the pollution level before 2005 in the nonattainment areas that become regulated and compare it to the pollution level afterwards. And then we compare that change over time to the before and after change over time in the control areas that did not have regulation. This can tell us something about what may have been the effect of the policy treatment, but that's only the case under certain assumptions. The key assumption is what we call the parallel trends assumption. The idea is that we really want to compare apples to apples. In other words, we want to have the assumption fulfilled that if the policy didn't happen, then those treated areas would have evolved in the same way over time as the control areas. That is where in our study we saw some problems, because once we looked at the data and we just plotted it on a graph, the evolution of pollution levels over time in the nonattainment and attainment areas just didn't look like those lines were running in parallel even before 2005, before the policy was introduced.
Smith: Could you explain the strategies that you used to overcome this problem,and the pros and cons of each of the techniques you tried?
Gregor Singer: The general problem is that we're trying to solve this issue with these differential time trends. The treated and control units don't seem to be on the same time trend. We have three different alternative estimation methods in the paper. The first one is a difference-in-differences setup, but you explicitly control for the differential trends. We have a trend included in the estimation that controls for these different evolutions over time based on the baseline PM2.5 concentrations. The advantage is that you get to keep the full sample for this exercise. The disadvantage is that you need to be sure that you have properly controlled for these underlying trends.
The second alternative estimation approach is a combination of difference-in-differences and matching. The idea with matching is that you look for each treated unit, in our case census tracts within the control space somewhere in the US that look similar. Now, we actually have some visibility on the areas where the EPA had no information at the time of the regulation, and this is because we're using this more comprehensive satellite data that does not only rely on ground monitors, which are not in all areas. We have some tracts in non-treated areas that are in attainment and those can be matched to polluted areas that have actually been treated. At the same time, even within those larger nonattainment areas, there's a lot of variation of baseline pollution. We can match each census tract in the treated areas to a census tract in the control areas, and then you can run your difference-and-differences on this matched sample. The key advantage here is that you will have a selected control group that looks much more similar. The disadvantage is you can't match every tract. Some tracts just look too different to find a suitable control.
The third way we address this is with a regression discontinuity setup. Here the idea is to take advantage of the design of the policy itself. The policy has this threshold value of 15 micrograms. And every county above 15 micrograms is placed into nonattainment. The idea is that if you just compare those counties around the threshold—so a county that has a value of 15.1 versus a county that has a value of 14.9—there are no other differences between the treated and control units. The main disadvantage here is that you only can use a small subset of counties just around the threshold. So each of these three methods has their pros and cons, but luckily and importantly, they all produce very similar results between them.
While we overestimate the effectiveness of the regulation in terms of how effective it was in reducing pollution, we underestimate the benefit of reducing air pollution in this context.
Gregor Singer
Smith: When you put these methods together, what was the actual effectiveness of the 2005 fine particulate regulation?
Sager: If we look at the first five years of the policy treatment, we find that the average effect of a nonattainment designation in those areas that were subject to more regulatory scrutiny was that they experienced over five years a PM2.5 concentration reduction of about 0.4 micrograms per cubic meter, which is approximately a 3 percent reduction. There's also some heterogeneity. We see that some areas experience bigger effects and some areas have smaller effects. Simple difference-in-differences estimation lands at an effect of about 10 percent pollution reduction. Our alternatives suggest a 3 percent pollution reduction. We think that is because we properly account for those different time trends, while difference-in-differences may be overestimating the benefits.
Smith: What does your research imply for other estimates that might be tied to air quality regulations?
Singer: That's really interesting because if these nonattainment effects are smaller, if you employ these estimation methods that take care of these trends, then this can have knock-on effects for downstream applications. In particular we have two applications of these findings. The first one looks at who benefits from these policies. Because we have such granular data on pollution and the composition of residents within those areas, we can look at urban versus rural areas and at Black versus White Americans. There's a long-standing literature that documents that Black Americans have been exposed to much larger pollution concentrations than White Americans. We also see that in our data. The difference is around 13 percent higher PM2.5 exposure levels on average. The somewhat good news is that because Black Americans live in more polluted areas, and the Clean Air Act targets more polluted areas, they're also more likely to benefit from the policy. Indeed, we also find that Black Americans got a larger reduction in their exposure to PM2.5 than White Americans. This contribution of the Clean Air Act is a little bit lower than what previous research has found, just because the regulation has been less effective to begin with.
Our second application is an important question in the economics of air pollution: What's the benefit of a one unit reduction in PM2.5 concentration? One way to get at this question of how people value air quality is to look at house prices because people pay more for houses that are surrounded by cleaner air. The work in this literature sometimes relies on these nonattainment designations to tease out the effect of pollution on house prices. But if these nonattainment effects are overestimated, then this likely impacts this estimation of how pollution capitalizes into house prices. We use house price data for that, and we find that once we account for these trends with either of our three methods, the effect of a one unit improvement in air quality is actually much larger. So the benefits as capitalized into house prices are around two times as large once you account for these confounding trends. So while we overestimate the effectiveness of the regulation in terms of how effective it was in reducing pollution, we underestimate the benefit of reducing air pollution in this context.
♦
“Clean Identification? The Effects of the Clean Air Act on Air Pollution, Exposure Disparities, and House Prices” appears in the February 2025 issue of the American Economic Journal: Economic Policy. Music in the audio is by Podington Bear.