Environment Canada and NRTEE versus the Fraser Institute: An issue of quality

The Fraser Institute recently issued a report on Canadian air quality that achieved moderate coverage, but little critical scrutiny  (apart from a devastating critique from the Suzuki Foundation). In Canadian Environmental Indicators – Air Quality, Fraser Senior Economic Researcher Joel Wood claims that concentrations of five major pollutants, including the particularly worrying duo of fine particulate matter and ground level ozone, are in decline. Wood even accuses ENGOs and the Canadian Medical Association of trying to “scare” the public with claims that are “exaggerated or outright wrong”, particularly in their concerns about ozone.

But in a shocking development (or perhaps not), it turns out that Environment Canada’s official air quality indicator for ozone shows a statistically significant worsening over the last two decades, while both fine particulate matter and ozone show no significant trend in the last decade. The Environment Canada indicators, the very existence of which appears to have been totally missed by Wood and the various reviewers of his work, are based on complete warm season averages weighted by population, and thus provide a much better indicator than Wood’s ad hoc analysis.

This same ineptitude can also be seen in Wood’s short climate policy analyses, where he gets the basics totally wrong, while questioning the validity of climate science itself. Thus the ignorance, incompetence and bias of  Canada’s self-styled “leading public policy think tank” is demonstrated once again.

AIR QUALITY, FASER INSTITUTE STYLE

Joel Wood is the latest addition to Fraser’s Centre for Environmental Studies and Centre for Risk Regulation section (following in the footsteps of the long-departed but unlamented Kenneth Green). In fact, he seems to be the whole department at this point.  He started in at Fraser in 2010 after completing his PhD work under Fraser Senior Fellow (and climate science gadfly) Ross McKitrick at the University of Guelph. Wood’s c.v. shows no academic journal publications yet, but he does have three “working papers”, apparently based on his “three essay” PhD thesis, as well as numerous newspaper columns.

The summary of Canadian Environmental Indicators – Air Quality [PDF] features five “key conclusions” the first of which is very general.

Air quality in most of Canada has improved greatly since the 1970s and continues to improve.

The first part is not too controversial, if overly broad, and we don’t need a tax-subsidized think tank to tell us that. Environment Canada’s National Air Pollution Surveillance Program (NAPS)  already shows this very well.

Has air quality improved since NAPS first started measuring levels of air pollutants in 1970? The answer is decidedly yes.

NAPS supports this by displaying Canada-wide trends for four air pollutants (lead, sulphur dioxide, volatile organic compounds and particulate matter) from 1970.

But the back half of the Fraser report’s first conclusion, that air quality “continues to improve” is less well supported. To explore this issue, let’s skip to the last conclusion.

Statistical analysis suggests that concentrations of ground-level ozone and ultrafine particulate matter, the two air pollutants of most concern for human health, have been declining across Canada since 2000.

Wood admits that ground-level ozone has been persistently problematic, especially in Ontario. First he considers the range of monitoring results based on the 2000 Canada wide standard for ozone. That standard established an upper limit 65 parts per billion measured over 8 hours. A particular location is deemed to be in compliance with these standards according to the level of the rolling-three year average of the fourth highest 8-hour reading within each year. Thus the standard addresses days at the upper end of health risk, and not an average or ongoing level.

Here is the chart for Ontario (at p. 39):

As can be easily seen, almost all stations are failing the standard year in and year out. Wood acknowledges the problem, although somehow that fact did not make it into any of the summaries. He then goes on to perform panel regression of various measures of exceedence of the Canada-Wide Standard.

In statistical tests meant to estimate trends in “O3 concentrations”, panel regression is applied to the fourth-highest eight hour average (i.e. the Canada Wide Standard) for each station and year. This test was applied to each of the eight provinces from 1980-2009 and for Canada as a whole from 2000-2009. For 1980-2009 Wood found significant  decreasing trend in all provinces except Saskatchewan (which has lower levels in any case) and Nova Scotia. And in the key Canada wide test for 2000-2009, Wood claims a declining trend of 0.47 ppb per year (i.e. 4.7 ppb over the decade), with a significance of 99%.

The first issue to deal with is the panel regression itself, which often appears misapplied by econometricians to produce overestimates of significance when applied to scientific data. The eight provincial 30-year trends have improbably small standard errors ranging from 0.05 to 0.2 ppb per year, and the 2000-2009 Canada trend standard error is only 0.07 ppb per year.

REAL AIR QUALITY INDICATORS FROM ENVIRONMENT CANADA AND THE NRTEE

But the applicability of panel regression is the least of the problems here. To understand this, let’s turn to Canada’s National Air Quality indicators (first pointed out to me by the Suzuki response).

National Air Quality

The air quality indicators focus on measuring two key elements of smog: fine particulate matter (PM2.5) and ground-level ozone (O3). Often appearing as a haze in the air, smog is a mixture of many pollutants, with PM2.5 and O3 being its major components. The level of these pollutants in the air (air quality) has been linked to a number of adverse effects on health and the environment. The concentrations of PM2.5 and O3 in outdoor ambient air are influenced by many factors such as local emissions sources, weather conditions and the transport of air pollution over a long distance.

And then there’s this caveat on the good news about pollutants.

While concentrations of major pollutants, such as sulphur dioxide, have decreased since 1970, ongoing measurements and research on health effects have made it apparent that pollutants, such as fine particulate matter (less than 2.5 microns) and ozone, are still of concern.

Environment Canada is in its fourth year of providing these indicators, which are based on a 2003 report from the National Roundtable on Energy and the Environment (NRTEE), Environment and Sustainable Development Indicators for Canada [PDF]. And look at what the official ozone indicator shows from 1990-2009:

There is a statistically significant upward trend in ground-level ozone from 1990-2009, while the 2000-2009 subset shows a slight but insignificant decrease, as seen in the following versions. First we have a replication of the 1990-2009 chart, followed by 2000-2009. The latter trend line is dashed to show that its slope is not significantly different from zero, given the wide standard error of the fit.

We can summarize the trends derived from the two different ozone “air quality indicators” as follows.

Range_____	Fraser________	Environment_Canada
1980-2009 *	-3.0 [-1.3, -4.8]
1990-2009		+2.0 [+0.4, +3.6]
2000-2009	-4.67 [-6.0, -3.3]	-1.0 [-5.6, +3.6]

Canadian ground ozone trends in ppb/decade [95% confidence interval]. Statistically significant trends in bold.

* Note:  Fraser 1980-2009 overall trend estimated as average of provincial trends in table 12, weighted by number of observations in each province.

Why such a big difference? Frankly, a big part of the answer was obvious as soon as I saw the Environment Canada figure above: the real air quality indicators are weighted by population. On top of that, the real indicators also use a complete average, rather than the ad hoc exceedence measure by Wood. Here’s the NAPS explanation from the very complete Data Sources and Methods for the Air Quality (O3 and PM2.5) Indicators:

The warm-season average of ground-level concentrations is the average of the highest daily maximum 8-hour average concentrations during the period from April 1 to September 30.

There are also daily and yearly data presence criteria that must be satisfied for a station to be included.

The NRTEE laid out its rationale for this approach:

The national Air Quality Trend Indicator will provide an indication of the exposure of Canadians to a particularly harmful type of air pollutant— ground-level ozone (O3). This indicator is a population-weighted measure of exposure to ozone. …

Ground-level ozone was chosen for this indicator for two reasons: the availability of reliable data linking human health effects to specific ambient concentrations of low-level ozone, and the existence of an extensive ambient concentration time series.

And the NRTEE’s preliminary conclusion?

The [pilot version] chart shows that the Air Quality Trend Indicator increased slightly between 1986 and
2000. In plain terms, this suggests that air quality (as measured by this indicator) has not improved over this time period. Although several factors can influence this indicator (weather being perhaps the most significant), it still suggests that our efforts to reduce pollution have not yet eased the burden of ozone exposure experienced by most Canadians.

It gets even better. Monitoring stations are sometimes merged as station locations are moved nearby. Others are excluded because of confounding elevation effects. If this all starting to sound familiar, it should; these are exactly the sort of subtleties that get missed when economists try to analyze scientific data, whether temperature or ozone concentration.

The exceedence approach could be a useful supplement to the already existing ambient air quality indicator. But it would have to at the very least incorporate population weighting and data quality checks.  And there’s no reason to presume that such measures, properly calculated would be inconsistent with the current official air quality index – quite the opposite.

Meanwhile, characterization of air quality trends should be based on the most robust relevant information available, not on the Fraser Institute’s misleading and irrelevant analysis.

Now we can appreciate just how shaky Wood’s conclusion truly is.

The results discussed here call into question an assumption used in the Canadian Medical Association 2008 report. When making forecasts of the number of air pollution deaths and air pollution related medical costs, the Canadian Medical Association report relies on the assumption that O3  concentrations in Canada will remain constant into the future. However, this assumption is not consistent with the trends in historical monitoring data.

Well, maybe the assumption of constant O3  concentrations in the future should be questioned, but in the opposite direction. It could well be too early to conclude that ozone concentration levels will not continue to rise in the long term. That’s especially true as ground-level ozone is expected to worsen with global warming (a consideration that Wood is predisposed to ignore, as we shall see).

How did Wood get everything so wrong? And how did Wood and everyone else involved miss the existence of official air quality indicators? First, there is undoubtedly the typical bias involved in all of the Fraser Institute’s work, which bears the hallmarks of preconceived conclusions in search of supporting evidence.

But there is also willful ignorance at play here. It’s highly unlikely that the review process involved actual experts in environmental health issues. Certainly the acknowledged informal reviewers suggest an insular impulse to consult a close circle of researchers who may be more senior but are as equally ignorant of the subject as the author. In this case, the acknowledged reviewers were all economists close to Woods. That list includes Fraser research V.P. Neils Veldhuis, Wood’s mentor Ross McKitrick, and University of Alberta professor Andrew Leach, another McKitrick protege who got his undergrad degree at Guelph.

BLOWING THE DOG WHISTLE

So why should we care about any of this? After all, other than some right-wing columnists and Conservative politicians, no one could possibly take this report on air quality seriously.  And this is supposed to be a blog about climate science and attacks thereupon.

Well, for one thing, there is an implicit, if unstated, message about climate change here. If ENGOs and the medical profession are shown to be “alarmist” about a supposedly straightforward issue like air quality, that can only reinforce the “alarmist” label for those same ENGOs’ advocacy of climate mitigation policies. Make no mistake: there is a snarling pack  of bloggers and columnists who can hear that dog whistle loud and clear. Take for instance (please) Vancouver Province columnist Jon Ferry, in a piece entitled Enviro groups snub good-news climate study.

The report’s key point was that air quality in Canada had improved greatly since the 1970s. And, of course, good news about the environment is the last thing B.C.’s eco-warriors want to hear. …

They are alarmists who’ve helped create a multimillion-dollar, taxpayer-funded industry around scaring people and making them feel guilty about their alleged sins against Mother Earth.

Look no further than B.C.’s Pacific Institute for Climate Solutions, set up in 2008 by then-premier Gordon Campbell with $95 million of public money. It rarely misses a chance to serve up global-warming propaganda.

Only last Friday, an opinion piece appeared in the Vancouver Sun by its executive director, Tom Pedersen, with some overheated declarations about warming oceans, retreating permafrost . . . and the need for gas taxes to keep rising.

And on and on: through “diehard warming activist” James Hansen, Al Gore, the Medieval Warm Period, “top scientists” opposing the “consensus” (you guessed it, the WSJ 16) and big government looking for an excuse to raise taxes. Phew!

WOOD TAKES ON CLIMATE POLICY AND CLIMATE SCIENCE

Then there’s the fact that attacking climate science and policies is never very far from the top of the Fraser agenda. And, as the only “environmental” economist currently on staff, Wood has already jumped in to this area.

In two smaller publications last year, Wood tries his hand at climate policy analysis, and displays a woeful ignorance of the basics. In the March-April 2011 edition of Fraser Forum, Wood presents an analysis of renewable fuel  regulations. Along the way he mentions in a footnote Canada’s 2020 target for GHG reduction, from the 2008 Turning the Corner plan.

The federal government has committed to reducing GHG emissions to 80% of 2006 levels by 2020 (2020 emissions would need to be 159.6 Mt less than 2008 emissions), see Environment Canada, 2008.

But this target had been superceded by the 2009 U.N. Cancun Accord more than a year before; Canada’s new short-term GHG target is 17% below 2005 levels by 2020. That doesn’t make a huge difference absolute terms, but it it’s telling that well over a year later, Wood still didn’t get it right.

That mistake was corrected in Wood’s next foray, an analysis of Liberal and Conservative GHG emission reductions targets in the run-up to the May 2011 Canadian election. Liberals and Conservatives offer job killing climate policies appeared in the Calgary Beacon and Canadian Manufacturing (wow – syndication!) and contained this new howler.

The Conservatives propose reducing emissions by 17 per cent of 2005 levels by 2020, while the Liberals propose reducing emissions by 80 per cent of 1990 levels by 2050. Of the two, the Conservative target is more ambitious, since it requires larger annual average reductions of emissions.

Well, let’s check Wood’s math and calculate per year reductions required under both targets starting in 2012 and ending in 2020 for the Conservatives (9 years) and 2050 for the Liberals (39 years). First we need an estimate for 2011 GHGs. At the time Woods wrote, the latest available projection for 2011 was 710 Mt.

Then we need to calculate the respective targets, and look at the reduction in GHGs required each year from 2012 on. I’ve summarized some of the numbers here.

Years_and_GHGs	Conservatives	____Liberals
Baseline year	2005	1990
Baseline  GHGs	731 Mt	592 Mt
Target year	2020	2050
Target GHGs	607 Mt	118 Mt
Reduction per year  (from 2012 on)	11.5 Mt/yr	15.2 Mt/yr

Don’t get me wrong. I would actually be impressed if Canadian public policy resulted in whittling away at our massive carbon footprint at anywhere in the neighbourhood of 11-15 Mt per year, year in year out. Unfortunately, there is no chance of this actually happening under current Conservative climate and economic development policies.

The main point here, however, is that once again the resident environmental expert at “Canada’s leading public policy think tank” is completely clueless and has no idea what he is talking about.

But it gets better:

Setting aside the questionable validity of the evidence suggesting a human contribution to climate change, both plans would have only negligible impact on global greenhouse gas emissions. After all, Canada currently accounts for about two per cent of global emissions.

We’re used to the well worn 2% meme used by “ethical oil” proponents to claim that it doesn’t matter what Canada does. But what’s this about the “questionable validity of the evidence suggesting a human contribution to climate change”?

Now that goes a lot further than Joe Oliver or even EthicalOil.org. Wood is actually positing that there may not be any significant human contribution at all to climate change, not even a small one.  This is hardly a realistic proposition and frankly borders on the cretinous.

Wood expressed a similar sentiment in the introduction to his 2010 three-essay PhD dissertation, delivered at the University of Guelph under his mentor, one Ross McKitrick.

Air pollution may also create external damages at the global level. For example, the burning of fossil fuels releases carbon dioxide into the atmosphere, and high levels of the atmospheric stock of carbon dioxide may contribute to global climate change.

May contribute. Or may not contribute at all. Apparently such bland unsupported (and unsupportable) dismissals of entire fields of study passes for scholarship in the Guelph economics department.

If Wood is going to question the “validity” of the overwhelming evidence for a strong human contribution to global warming, he should at least state the basis for his belief. Is it all just a hoax? Or does he consider the egregiously flawed work of his mentor McKitrick more reliable than the thousands of papers and multiple lines of evidence on the side of  actual science?

It is indeed ironic that ENGOs are under virulent attack by the likes of climate science agnostics (to put it charitably) Joe Oliver and Fort McMurray MP Brian Jean.

For the real scandal of the Canadian public policy discourse is how right-wing think tanks like the Fraser Institute continue to get away with inept and biased “research”, all with an effective subsidy by virtue of their charitable tax status.

Whether one considers research quality, transparency or egregious conflicts of interest, the Fraser Institute simply does not measure up to a reasonable standard that should be expected from organizations granted the privilege of charitable status. That shortfall is especially glaring when the Fraser Institute is compared to a top ENGO like the Pembina Institute, which is manifestly superior in each of these dimensions.

That’s a theme to which I’ll be returning very soon. It’s time to focus on the real issues of public policy in Canada.