Photo by gecua.

The National Bureau of Economic Research (NBER) has released a draft of a working paper that uses China’s own officially reported Air Pollution Index (API) and satellite data to evaluate air pollution control measures instituted before, during, and after the 2008 Olympics games in Beijing.

To recap, Beijing shut down or slowed operations of many heavy-industrial factories in neighboring provinces. They also reduced the number of vehicles on the road by half, through an odd-even license plate policy that is still maintained today to address Beijing’s increasingly congested roadways.

Some have questioned the veracity of Beijing’s claims of improved air quality in time for the Olympic games.

Irregularities in the monitoring of air quality account for all reported improvements over the last nine years,” said Steven Q. Andrews, the author of the study, in a telephone interview. Mr. Andrews published an op-ed article about his study on Wednesday in the Asian edition of The Wall Street Journal.

Granted, Du Shaozhong, deputy director of Beijing’s Environmental Protection Bureau, did not add much confidence by way of refuting Andrews’ claims in telling the NY Times, “We have patience for that small group of people who don’t understand, and we believe that, as time passes, and in the face of facts, they will eventually understand.”

However, Chen and colleagues at NBER were able to use a satellite-derived measures of air quality (Aerosol Optical Depth) to determine that the aggressive air pollution control measures Beijing implemented prior to the Olympic games did result in measurable improvements in air quality, which were not merely the result of “statistical manipulation.”

They were able to demonstrate that the average API of Beijing dropped from 109.01 (on the API index, not what would be considered a ‘blue-sky day’) in 2000 and 2001 to 76.69 (a blue-sky day) during the Olympic Games. They did notice a lag in the air quality improvements, stating that most of the improvement happened after the Games started.

In comparison, the AOD of Beijing (which shows a positive relationship with air pollution) started to decline before the Games, continued to decline during the Games, and reached the lowest level 2-6 months after the Games.

Shortly after these observations of improvement, the AOD measurements reverted, suggesting that although the results were real, they were temporary.

Of course, there are several caveats to this preliminary analysis (and using satellite-derived measures of air quality in general):

1) Independent, in-situ measurements “from the ground” were not available for comparison, other than the officially-reported API data, so the authors note with caution that air quality improvements determined are more “relative” than “absolute.”

2) Because the AOD measure captures the amount of radiation absorbed, reflected, and scattered due to the presence of both solid and liquid particulates suspending in the atmosphere, sources of aerosol can be both anthropogenic (i.e. combustion) and natural (i.e. dust storms). Although the algorithms are fairly robust, the results of gauging air quality with AOD measurements can vary regionally. Lower accuracy might be found in geographic regions in China where dust storms are more prevalent. Furthermore, it is difficult to apply models derived in one region to another.

3) AOD measurements are also sensitive to weather and climate, so some data could be missing depending upon whether there are clouds present in a given frame.

Nevertheless, this report provides a useful counterpoint to the popular narrative that Beijing was somehow manipulating air quality statistics rather than achieving real improvements before, during, and after the Olympics.

My research is currently using the same satellite data to evaluate the performance of air pollution control policies in Chinese megacities, including Beijing but also Shanghai, which underwent similar measures to improve air quality for the Expo held last year. I am still in the phases of data analysis, so check back for updates in the near future.