Environment Canada and NRTEE versus the Fraser Institute: An issue of quality

The Fraser Institute recently issued a report on Canadian air quality that achieved moderate coverage, but little critical scrutiny  (apart from a devastating critique from the Suzuki Foundation). In Canadian Environmental Indicators – Air Quality, Fraser Senior Economic Researcher Joel Wood claims that concentrations of five major pollutants, including the particularly worrying duo of fine particulate matter and ground level ozone, are in decline. Wood even accuses ENGOs and the Canadian Medical Association of trying to “scare” the public with claims that are “exaggerated or outright wrong”, particularly in their concerns about ozone.

But in a shocking development (or perhaps not), it turns out that Environment Canada’s official air quality indicator for ozone shows a statistically significant worsening over the last two decades, while both fine particulate matter and ozone show no significant trend in the last decade. The Environment Canada indicators, the very existence of which appears to have been totally missed by Wood and the various reviewers of his work, are based on complete warm season averages weighted by population, and thus provide a much better indicator than Wood’s ad hoc analysis.

This same ineptitude can also be seen in Wood’s short climate policy analyses, where he gets the basics totally wrong, while questioning the validity of climate science itself. Thus the ignorance, incompetence and bias of  Canada’s self-styled “leading public policy think tank” is demonstrated once again.


Joel Wood is the latest addition to Fraser’s Centre for Environmental Studies and Centre for Risk Regulation section (following in the footsteps of the long-departed but unlamented Kenneth Green). In fact, he seems to be the whole department at this point.  He started in at Fraser in 2010 after completing his PhD work under Fraser Senior Fellow (and climate science gadfly) Ross McKitrick at the University of Guelph. Wood’s c.v. shows no academic journal publications yet, but he does have three “working papers”, apparently based on his “three essay” PhD thesis, as well as numerous newspaper columns.

The summary of Canadian Environmental Indicators – Air Quality [PDF] features five “key conclusions” the first of which is very general.

Air quality in most of Canada has improved greatly since the 1970s and continues to improve.

The first part is not too controversial, if overly broad, and we don’t need a tax-subsidized think tank to tell us that. Environment Canada’s National Air Pollution Surveillance Program (NAPS)  already shows this very well.

Has air quality improved since NAPS first started measuring levels of air pollutants in 1970? The answer is decidedly yes.

NAPS supports this by displaying Canada-wide trends for four air pollutants (lead, sulphur dioxide, volatile organic compounds and particulate matter) from 1970.

But the back half of the Fraser report’s first conclusion, that air quality “continues to improve” is less well supported. To explore this issue, let’s skip to the last conclusion.

Statistical analysis suggests that concentrations of ground-level ozone and ultrafine particulate matter, the two air pollutants of most concern for human health, have been declining across Canada since 2000.

Wood admits that ground-level ozone has been persistently problematic, especially in Ontario. First he considers the range of monitoring results based on the 2000 Canada wide standard for ozone. That standard established an upper limit 65 parts per billion measured over 8 hours. A particular location is deemed to be in compliance with these standards according to the level of the rolling-three year average of the fourth highest 8-hour reading within each year. Thus the standard addresses days at the upper end of health risk, and not an average or ongoing level.

Here is the chart for Ontario (at p. 39):

As can be easily seen, almost all stations are failing the standard year in and year out. Wood acknowledges the problem, although somehow that fact did not make it into any of the summaries. He then goes on to perform panel regression of various measures of exceedence of the Canada-Wide Standard.

In statistical tests meant to estimate trends in “O3 concentrations”, panel regression is applied to the fourth-highest eight hour average (i.e. the Canada Wide Standard) for each station and year. This test was applied to each of the eight provinces from 1980-2009 and for Canada as a whole from 2000-2009. For 1980-2009 Wood found significant  decreasing trend in all provinces except Saskatchewan (which has lower levels in any case) and Nova Scotia. And in the key Canada wide test for 2000-2009, Wood claims a declining trend of 0.47 ppb per year (i.e. 4.7 ppb over the decade), with a significance of 99%.

The first issue to deal with is the panel regression itself, which often appears misapplied by econometricians to produce overestimates of significance when applied to scientific data. The eight provincial 30-year trends have improbably small standard errors ranging from 0.05 to 0.2 ppb per year, and the 2000-2009 Canada trend standard error is only 0.07 ppb per year.


But the applicability of panel regression is the least of the problems here. To understand this, let’s turn to Canada’s National Air Quality indicators (first pointed out to me by the Suzuki response).

National Air Quality

The air quality indicators focus on measuring two key elements of smog: fine particulate matter (PM2.5) and ground-level ozone (O3). Often appearing as a haze in the air, smog is a mixture of many pollutants, with PM2.5 and O3 being its major components. The level of these pollutants in the air (air quality) has been linked to a number of adverse effects on health and the environment. The concentrations of PM2.5 and O3 in outdoor ambient air are influenced by many factors such as local emissions sources, weather conditions and the transport of air pollution over a long distance.

And then there’s this caveat on the good news about pollutants.

While concentrations of major pollutants, such as sulphur dioxide, have decreased since 1970, ongoing measurements and research on health effects have made it apparent that pollutants, such as fine particulate matter (less than 2.5 microns) and ozone, are still of concern.

Environment Canada is in its fourth year of providing these indicators, which are based on a 2003 report from the National Roundtable on Energy and the Environment (NRTEE), Environment and Sustainable Development Indicators for Canada [PDF]. And look at what the official ozone indicator shows from 1990-2009:

There is a statistically significant upward trend in ground-level ozone from 1990-2009, while the 2000-2009 subset shows a slight but insignificant decrease, as seen in the following versions. First we have a replication of the 1990-2009 chart, followed by 2000-2009. The latter trend line is dashed to show that its slope is not significantly different from zero, given the wide standard error of the fit.

We can summarize the trends derived from the two different ozone “air quality indicators” as follows.

Range_____ Fraser________ Environment_Canada
1980-2009 * -3.0 [-1.3, -4.8]
1990-2009 +2.0 [+0.4, +3.6]
2000-2009 -4.67 [-6.0, -3.3] -1.0 [-5.6, +3.6]

Canadian ground ozone trends in ppb/decade [95% confidence interval]. Statistically significant trends in bold.

* Note:  Fraser 1980-2009 overall trend estimated as average of provincial trends in table 12, weighted by number of observations in each province.

Why such a big difference? Frankly, a big part of the answer was obvious as soon as I saw the Environment Canada figure above: the real air quality indicators are weighted by population. On top of that, the real indicators also use a complete average, rather than the ad hoc exceedence measure by Wood. Here’s the NAPS explanation from the very complete Data Sources and Methods for the Air Quality (O3 and PM2.5) Indicators:

The warm-season average of ground-level concentrations is the average of the highest daily maximum 8-hour average concentrations during the period from April 1 to September 30.

There are also daily and yearly data presence criteria that must be satisfied for a station to be included.

The NRTEE laid out its rationale for this approach:

The national Air Quality Trend Indicator will provide an indication of the exposure of Canadians to a particularly harmful type of air pollutant— ground-level ozone (O3). This indicator is a population-weighted measure of exposure to ozone. …

Ground-level ozone was chosen for this indicator for two reasons: the availability of reliable data linking human health effects to specific ambient concentrations of low-level ozone, and the existence of an extensive ambient concentration time series.

And the NRTEE’s preliminary conclusion?

The [pilot version] chart shows that the Air Quality Trend Indicator increased slightly between 1986 and
2000. In plain terms, this suggests that air quality (as measured by this indicator) has not improved over this time period. Although several factors can influence this indicator (weather being perhaps the most significant), it still suggests that our efforts to reduce pollution have not yet eased the burden of ozone exposure experienced by most Canadians.

It gets even better. Monitoring stations are sometimes merged as station locations are moved nearby. Others are excluded because of confounding elevation effects. If this all starting to sound familiar, it should; these are exactly the sort of subtleties that get missed when economists try to analyze scientific data, whether temperature or ozone concentration.

The exceedence approach could be a useful supplement to the already existing ambient air quality indicator. But it would have to at the very least incorporate population weighting and data quality checks.  And there’s no reason to presume that such measures, properly calculated would be inconsistent with the current official air quality index – quite the opposite.

Meanwhile, characterization of air quality trends should be based on the most robust relevant information available, not on the Fraser Institute’s misleading and irrelevant analysis.

Now we can appreciate just how shaky Wood’s conclusion truly is.

The results discussed here call into question an assumption used in the Canadian Medical Association 2008 report. When making forecasts of the number of air pollution deaths and air pollution related medical costs, the Canadian Medical Association report relies on the assumption that O3  concentrations in Canada will remain constant into the future. However, this assumption is not consistent with the trends in historical monitoring data.

Well, maybe the assumption of constant O3  concentrations in the future should be questioned, but in the opposite direction. It could well be too early to conclude that ozone concentration levels will not continue to rise in the long term. That’s especially true as ground-level ozone is expected to worsen with global warming (a consideration that Wood is predisposed to ignore, as we shall see).

How did Wood get everything so wrong? And how did Wood and everyone else involved miss the existence of official air quality indicators? First, there is undoubtedly the typical bias involved in all of the Fraser Institute’s work, which bears the hallmarks of preconceived conclusions in search of supporting evidence.

But there is also willful ignorance at play here. It’s highly unlikely that the review process involved actual experts in environmental health issues. Certainly the acknowledged informal reviewers suggest an insular impulse to consult a close circle of researchers who may be more senior but are as equally ignorant of the subject as the author. In this case, the acknowledged reviewers were all economists close to Woods. That list includes Fraser research V.P. Neils Veldhuis, Wood’s mentor Ross McKitrick, and University of Alberta professor Andrew Leach, another McKitrick protege who got his undergrad degree at Guelph.


So why should we care about any of this? After all, other than some right-wing columnists and Conservative politicians, no one could possibly take this report on air quality seriously.  And this is supposed to be a blog about climate science and attacks thereupon.

Well, for one thing, there is an implicit, if unstated, message about climate change here. If ENGOs and the medical profession are shown to be “alarmist” about a supposedly straightforward issue like air quality, that can only reinforce the “alarmist” label for those same ENGOs’ advocacy of climate mitigation policies. Make no mistake: there is a snarling pack  of bloggers and columnists who can hear that dog whistle loud and clear. Take for instance (please) Vancouver Province columnist Jon Ferry, in a piece entitled Enviro groups snub good-news climate study.

The report’s key point was that air quality in Canada had improved greatly since the 1970s. And, of course, good news about the environment is the last thing B.C.’s eco-warriors want to hear. …

They are alarmists who’ve helped create a multimillion-dollar, taxpayer-funded industry around scaring people and making them feel guilty about their alleged sins against Mother Earth.

Look no further than B.C.’s Pacific Institute for Climate Solutions, set up in 2008 by then-premier Gordon Campbell with $95 million of public money. It rarely misses a chance to serve up global-warming propaganda.

Only last Friday, an opinion piece appeared in the Vancouver Sun by its executive director, Tom Pedersen, with some overheated declarations about warming oceans, retreating permafrost . . . and the need for gas taxes to keep rising.

And on and on: through “diehard warming activist” James Hansen, Al Gore, the Medieval Warm Period, “top scientists” opposing the “consensus” (you guessed it, the WSJ 16) and big government looking for an excuse to raise taxes. Phew!


Then there’s the fact that attacking climate science and policies is never very far from the top of the Fraser agenda. And, as the only “environmental” economist currently on staff, Wood has already jumped in to this area.

In two smaller publications last year, Wood tries his hand at climate policy analysis, and displays a woeful ignorance of the basics. In the March-April 2011 edition of Fraser Forum, Wood presents an analysis of renewable fuel  regulations. Along the way he mentions in a footnote Canada’s 2020 target for GHG reduction, from the 2008 Turning the Corner plan.

The federal government has committed to reducing GHG emissions to 80% of 2006 levels by 2020 (2020 emissions would need to be 159.6 Mt less than 2008 emissions), see Environment Canada, 2008.

But this target had been superceded by the 2009 U.N. Cancun Accord more than a year before; Canada’s new short-term GHG target is 17% below 2005 levels by 2020. That doesn’t make a huge difference absolute terms, but it it’s telling that well over a year later, Wood still didn’t get it right.

That mistake was corrected in Wood’s next foray, an analysis of Liberal and Conservative GHG emission reductions targets in the run-up to the May 2011 Canadian election. Liberals and Conservatives offer job killing climate policies appeared in the Calgary Beacon and Canadian Manufacturing (wow – syndication!) and contained this new howler.

The Conservatives propose reducing emissions by 17 per cent of 2005 levels by 2020, while the Liberals propose reducing emissions by 80 per cent of 1990 levels by 2050. Of the two, the Conservative target is more ambitious, since it requires larger annual average reductions of emissions.

Well, let’s check Wood’s math and calculate per year reductions required under both targets starting in 2012 and ending in 2020 for the Conservatives (9 years) and 2050 for the Liberals (39 years). First we need an estimate for 2011 GHGs. At the time Woods wrote, the latest available projection for 2011 was 710 Mt.

Then we need to calculate the respective targets, and look at the reduction in GHGs required each year from 2012 on. I’ve summarized some of the numbers here.

 Years_and_GHGs Conservatives ____Liberals
Baseline year 2005 1990
Baseline  GHGs 731 Mt 592 Mt
Target year 2020 2050
Target GHGs 607 Mt 118 Mt
Reduction per year  (from 2012 on)
11.5 Mt/yr 15.2 Mt/yr

Don’t get me wrong. I would actually be impressed if Canadian public policy resulted in whittling away at our massive carbon footprint at anywhere in the neighbourhood of 11-15 Mt per year, year in year out. Unfortunately, there is no chance of this actually happening under current Conservative climate and economic development policies.

The main point here, however, is that once again the resident environmental expert at “Canada’s leading public policy think tank” is completely clueless and has no idea what he is talking about.

But it gets better:

Setting aside the questionable validity of the evidence suggesting a human contribution to climate change, both plans would have only negligible impact on global greenhouse gas emissions. After all, Canada currently accounts for about two per cent of global emissions.

We’re used to the well worn 2% meme used by “ethical oil” proponents to claim that it doesn’t matter what Canada does. But what’s this about the “questionable validity of the evidence suggesting a human contribution to climate change”?

Now that goes a lot further than Joe Oliver or even EthicalOil.org. Wood is actually positing that there may not be any significant human contribution at all to climate change, not even a small one.  This is hardly a realistic proposition and frankly borders on the cretinous.

Wood expressed a similar sentiment in the introduction to his 2010 three-essay PhD dissertation, delivered at the University of Guelph under his mentor, one Ross McKitrick.

Air pollution may also create external damages at the global level. For example, the burning of fossil fuels releases carbon dioxide into the atmosphere, and high levels of the atmospheric stock of carbon dioxide may contribute to global climate change.

May contribute. Or may not contribute at all. Apparently such bland unsupported (and unsupportable) dismissals of entire fields of study passes for scholarship in the Guelph economics department.

If Wood is going to question the “validity” of the overwhelming evidence for a strong human contribution to global warming, he should at least state the basis for his belief. Is it all just a hoax? Or does he consider the egregiously flawed work of his mentor McKitrick more reliable than the thousands of papers and multiple lines of evidence on the side of  actual science?

It is indeed ironic that ENGOs are under virulent attack by the likes of climate science agnostics (to put it charitably) Joe Oliver and Fort McMurray MP Brian Jean.

For the real scandal of the Canadian public policy discourse is how right-wing think tanks like the Fraser Institute continue to get away with inept and biased “research”, all with an effective subsidy by virtue of their charitable tax status.

Whether one considers research quality, transparency or egregious conflicts of interest, the Fraser Institute simply does not measure up to a reasonable standard that should be expected from organizations granted the privilege of charitable status. That shortfall is especially glaring when the Fraser Institute is compared to a top ENGO like the Pembina Institute, which is manifestly superior in each of these dimensions.

That’s a theme to which I’ll be returning very soon. It’s time to focus on the real issues of public policy in Canada.


20 responses to “Environment Canada and NRTEE versus the Fraser Institute: An issue of quality

  1. The title of “public policy analyst” is an interesting one.
    Does anyone know the real qualifications for that?
    I’ve been reading a lot of words by people with titles like that.

  2. Did he actually get a “degree” from Guelph or was it a “radian”? After all McKitrick can’t tell the difference.

  3. If anyone really want to know what sort of mumbo-jumbo McKitrick et al. use you can see some of the madness in his recent non peer reviewed paper. An example is his “Equation (2)” which is supposed to relate “the 1979-2002 trend in the CRUTEM3v gridded surface climate data” to a number of variables. These variable include such factors as “local population change from 1979 to 2002”, “per capita income change
    from 1979 to 2002”, “total Gross Domestic Product (GDP) change from 1979 to 2002”, “coal consumption change from 1979 to 2002”, “GDP density (national Gross Domestic Product per square kilometer) as of 1979” and “the average level of educational attainment”.

    After waving a few chicken bones over his equation and staring at a virgin sheep’s entrails he reached the following conclusion:

    “But the unrestricted model tells a very different story. GDP and educational attainment correlate with temperature trends, as do the rate of population change and coal consumption”.

    It must be very cold these days in Guelph.

    If anyone wants the details they can find them here:

    Click to access 2012-02.pdf

    And he was invited to make a presentation before a Senate Committee.

  4. Dear Deep Climate-
    You mentioned recently that the Fraser Institute gets a lot of foreign funding, especially if you consider foreign-controlled Canadian companies. Are you going to talk more about that?


    • Yes, but it depends what you mean by a “lot”. It appears to be comparable to Pembina, one of the supposed “puppets of foreign interests”. But in both cases it’s about 15% or so for 2010.

  5. Here is an article about the Fraser Institute.
    “University of Pennsylvania Again Ranks Fraser Institute as No. 1 Canadian …”


    This article has a lot of spelling mistakes. Twice they say “tip” instead of “top.” They say “in to” instead of “into.” They write “pass” instead of “past.”

    The whole article sounds like gibberish, not English. What is this all about? Read it all carefully a few times. See the contact information at the bottom.

    • Not sure, but I doubt it is the original.

    • As an academic retired from a public University in the US Pacific Northwest, that subtitle “News From All Online Universities” stirred up old interests, unrelated to climate. So I looked at the most recent entries, and make two observations. First all the articles i looked at, on all subjects, are written in the same gibberish and second, thought they are difficult to make any sense out there are enough slips (use of “I”) to make me suspicious that all of it is the writing of a single person. Only quoted excerpts taken from elsewhere are in standard English and make unambiguous sense. It appears to be a nonsense blog created for the sole purpose of selling ads to on line educational programs. There are lots of similar fake web sites. So why are you paying any attention to what is said there about the Fraser Institute, or anything else?

  6. Here is the same article in English. The press release is put out by a Dean Pelkey.
    It is claimed that the Fraser Institute is ” a global group of allied organizations in 80 countries.”

    They link to what the University of Pennsylvania supposedly said about them. Really, a lot of this seems like an echo-chamber.

    • The University of Pennsylvania is a hotbed of libertarian economics and home to Abraham Wyner of McShane and Wyner fame (or should that be infamy?).

    • The UPenn rankings are pretty skewed to the right, but with a few reasonable more centrist think tanks also present.

      Besides Fraser at #1, other “tin foil hat” think tanks in the Canada/Mexico/Caribbean top 30 include Montreal Economic Institute (9), MacDonald-Laurier (12) and the Frontier Centre for Public Policy (28). More reputable ones included C.D. Howe (8) and Conference Board of Canada (21).

      I have to admit I don’t know most of the others and I doubt that I will bother looking into them.

  7. An interesting pair of comments on my post at Reddit.

    [Scroll down to third comment and fourth comments]

    The timing of this is really weird.

    I just finished a project for a fourth year university course in Environmental Economics, and we had to collect data from Environment Canada’s National Air Pollutant Surveillance (NAPS) Network. From 1980 to 2009, many of the groups found increasing trends in ground level ozone.

    I can say that my study of Montreal exhibited readings that are exceeding the National Ambient Air Quality Objectives of 25 parts per billion in a 24 hour period, and 15 parts per billion over an annual average (maximum acceptable levels), and it is still increasing trend.

    The Fraser Institute is kidding itself to claim that this isn’t a problem. It’s a very serious problem.


    r/Canada: I felt compelled to show you my slideshow from the assignment. I added a couple slides to help explain a couple things. You can find it in PDF format here.

    • I don’t see specific reference to cutting monitoring stations measuring ground-level ozone and local air quality in general. But the cuts would compromise tracking of ozone and other pollution from distant or transborder sources (e.g. Asia and the U.S.). And, actually, even long distance pollution within our borders.

  8. Wait, you can get a Ph.D. by writing three essays? Dammit, I’m getting ripped off here.

  9. Have you seen this article? Canada is going to dismantle ozone monitoring sites. Wonder why!


    Aside from generalized objections to denialism, whether of tobacco, acid rain, or climate change, ozone denialism is quite pertinent to issues of climate.

    1. Ozone is the third most potent greenhouse gas.

    2. Although peaks of ozone have been mitigated, the pervasive, constant, background level is inexorably increasing as precursors travel across oceans and continents. Aside from impairing human health (and I don’t even know of ANY studies of what it does to animals and insects, but it would be foolish not to think they are also affected) ozone causes significant reductions in annual crop yield and quality. Oh, and longer-lived perennial vegetative species, such as trees, that are exposed to cumulative damage season after season, are dying off at an accelerating rate…so, in terms of climate, that means goodbye major carbon sink.

    ► Surface O3 will have an increasingly detrimental impact on future crop yields. ► Yields of wheat, soybean, and maize could be reduced by up to 26% globally in 2030. ► Global losses up to 17% are possible even in an optimistic scenario of future O3. ► Agricultural losses worldwide may be worth $17–35 billion annually by 2030. ~ Princeton University, Global Crop Yield Reductions Due To Surface Ozone, 2010

    • Why? Because the government is run by ignorant rightwing politicians and their ignorant young political aides who are deep in denial. They are concentrating on expanding the oil sands industry in a misguided attempt to make Canada into a petrostate and they refuse to make a real effort to reduce GHG emissions. And they muzzle government scientists because they prefer their delusions to reality.