By Deep Climate
Today I continue my examination of the key analysis section of the Wegman report on the Mann et al “hockey stick” temperature reconstruction, which uncritically rehashed Steve McIntyre and Ross McKitrick’s purported demonstration of the extreme biasing effect of Mann et al’s “short-centered” principal component analysis.
First, I’ll fill in some much needed context as an antidote to McIntyre and McKitrick’s misleading focus on Mann et al’s use of principal components analysis (PCA) in data preprocessing of tree-ring proxy networks. Their problematic analysis was compounded by Wegman et al’s refusal to even consider all subsequent peer reviewed commentary – commentary that clearly demonstrated that correction of Mann et al’s “short-centered” PCA had minimal impact on the overall reconstruction.
Next, I’ll look at Wegman et al’s “reproduction” of McIntyre and McKitrick’s simulation of Mann et al’s PCA methodology, published in the pair’s 2005 Geophysical Research Letters article, Hockey sticks, principal components, and spurious significance). It turns out that the sample leading principal components (PC1s) shown in two key Wegman et al figures were in fact rendered directly from McIntyre and McKitrick’s original archive of simulated “hockey stick” PC1s. Even worse, though, is the astonishing fact that this special collection of “hockey sticks” is not even a random sample of the 10,000 pseudo-proxy PC1s originally produced in the GRL study. Rather it expressly contains the very top 100 – one percent – having the most pronounced upward blade. Thus, McIntyre and McKitrick’s original Fig 1-1, mechanically reproduced by Wegman et al, shows a carefully selected “sample” from the top 1% of simulated “hockey sticks”. And Wegman’s Fig 4-4, which falsely claimed to show “hockey sticks” mined from low-order, low-autocorrelation “red noise”, contains another 12 from that same 1%!
Finally, I’ll return to the central claim of Wegman et al – that McIntyre and McKitrick had shown that Michael Mann’s “short-centred” principal component analysis would mine “hockey sticks”, even from low-order, low-correlation “red noise” proxies . But both the source code and the hard-wired “hockey stick” figures clearly confirm what physicist David Ritson pointed out more than four years ago, namely that McIntyre and McKitrick’s “compelling” result was in fact based on a highly questionable procedure that generated null proxies with very high auto-correlation and persistence. All these facts are clear from even a cursory examination of McIntyre’s source code, demonstrating once and for all the incompetence and lack of due diligence exhibited by the Wegman report authors.
This is the final instalment in a series of posts documenting dubious scholarship and unattributed sources in the background chapter of the touchstone of climate contrarians known as the Wegman Report. That report has been touted as Exhibit A proving the “destruction” of Michael Mann’s “hockey stick” graph by self-styled climate auditor Steve McIntyre.
Previously, I found extensive passages bearing “striking similarity” to a classic text by the distinguished paleoclimatologist (and “hockey stick” co-author) Raymond Bradley in the background sections on tree rings and on ice cores. Subsequently, the background section on social networks was found to contain material apparently drawn without attribution from a variety of sources, including Wikipedia and several text books.
This time, I’m looking at section 2.2 (see Wegman Report PDF at p. 15), which gives the background of key statistical concepts, including Principal Component Analysis. Astonishingly, even this section appears to contain a significant amount of unattributed material from other sources, although quite a bit less than the other sections. Again, Wikipedia appears to be a key source, along with a couple of text books.
I’ll also introduce some refinements to the text analysis, based largely on John Mashey’s recent innovations. Those refinements allow a better characterization of the relationship between various passages in Wegman et al and their apparent antecedents, as well as permitting a quantitative analysis based on word counts.
Terence Corcoran may well have just unleashed the National Post’s biggest whopper yet about climate science – and that’s saying something.
Corcoran’s commentary on the recent Russell “climategate” email review lays one error-laden defamation on top of another, as he attempts to demonstrate that the report “provides plenty of evidence that climate science has been and remains an uncertain shambles”. Oh, and apparently the review “portrays climate science as a field filled with uncertainty, debate, lack of openness, data hoarding and ill-will.”
Along the way, Corcoran even manages to confuse a little known Phil Jones graphic with Michael Mann’s “hockey stick” millenial temperature reconstruction. This leads to the astonishing (and entirely wrong) accusation that the hockey stick creators “eliminated some of the data from 1960 forward … and then spliced on actual temperature data”. Yet neither the “hockey stick” graph (the real one) nor the associated Mann et al study are mentioned in the report at all!
In this installment, I’ll look at another technique in the climate auditor’s toolbox, namely selective quotation. Once again, our example case study will involve accusations by Steve McIntyre concerning the use of paleoclimatologist Keith Briffa’s tree-ring based reconstruction in a key figure from the IPCC Third Assessment Report.
Arguing from a cherrypicked selection of quotes from the “Climategate” emails, McIntyre has claimed that IPCC authors Chris Folland and Michael Mann pressured Briffa to submit a reconstruction that would not “dilute the message” by showing “inconsistency” with multi-proxy reconstructions from Mann and Briffa’s CRU colleague Phil Jones. Briffa “hastily re-calculated his reconstruction”, sending one with a supposedly larger post-1960 decline before. According to McIntyre, Mann resolved this new “conundrum” and simply “chopped off the inconvenient portion of the Briffa tree-ring data”.
But a review of the emails – including some that have never been quoted before – clearly contradicts McIntyre’s version of events:
- Jones and Briffa were concerned that Mann had an outdated version of the Briffa reconstruction, and both urged the adoption of the newer “low frequency” one, more appropriate for comparison with other multi-century reconstructions.
- Far from pressuring Briffa to change his reconstruction right away, Mann questioned whether an immediate change was required, or even possible, and counselled waiting for the next revision.
- CRU colleague Tim Osborn advised Mann that he and Briffa “usually stopped” the “low frequency” reconstruction in 1960, and went one better in his later “resend” to Mann, by explicitly removing the post-1960 data.
I’ll also show how McIntyre has changed his narrative along the way , in an effort to prove that the true “context” of the famous “trick” to “hide the decline” is somehow an indictment of the IPCC. (Speaking of which, be sure to take the poll at the end about McIntyre’s next move). But first, once again, here is the cause of all the fuss, namely Figure 2-21 from Chapter 2 of the IPCC Third Assessment Report – Working Group I: The Scientific Basis (2001).
Do you have what it takes to be a climate auditor? Try the following fun test and find out. And at the same time, you can’t help but learn something about the fine art of argumentation from charts as practiced by the master himself, Steve McIntyre, and refined in his most devoted media outlet, the U.K. based Mail on Sunday.
Here is the chart that is held to epitomize the “trick” to “hide the decline”: figure 2-21 from the IPCC ‘s Third Assessment Report, showing key temperature reconstructions.
Today I continue my exploration of the dubious scholarship in the contrarian touchstone known as the Wegman report, this time focusing on the report’s background section on social network analysis. As many readers may recall, Wegman et al used a simplistic analysis of co-author relationships to speculate about supposed lack of independence between researchers in paleoclimatology, accompanied by lapses of rigour in the peer review process. This, of course, echoed similar accusations by self-styled climate auditor Steve McIntyre.
In both the original Wegman report and a subsequent follow-up paper by Yasmin Said, Wegman and two others, the background sections on social network research show clear and compelling instances of apparent plagiarism. The three main sources, used almost verbatim and without attribution, have now been identified. These include a Wikipedia article and a classic sociology text book by Wasserman and Faust. But the papers rely even more on the third source, a hands-on text book that explores social network concepts via the Pajek analysis software package – the same tool used by the Wegman team to analyze “hockey stick” author Michael Mann’s co-author network.
Not only that, but the later Said et al paper acknowledges support from the National Institutes on Alcohol Abuse and Alcoholism, as well as the Army Research Laboratory, raising a host of new issues and questions. And chief among those questions is this: Will George Mason University now finally do the right thing and launch a complete investigation of the actions and scholarship of Wegman and Said?
[Updated Feb. 8: (editing and extension of summary and document list update)]
Perhaps the most disturbing episode in the “hockey stick” controversy was the investigation of climate scientists by the U.S. House Energy and Commerce Committee under Republican representatives Joe Barton and Ed Whitfield, and a subsequent report for that same committee by an “independent” panel led by George Mason University statistics professor Edward Wegman. In light of various renewed “skeptic” allegations of scientific misconduct against Michael Mann and Phil Jones, and my recent revelation of possible plagiarism and other questionable scholarship in the Wegman report, a complete review of the events of 2005-2006 would seem to be in order.
In short, the Energy and Commerce Committee refused the offer of a proper scientific review from the National Academy of Sciences in favour of an investigative process that was ad hoc, biased and unscientific. And the report resulting from that process is tainted with highly questionable scholarship.
I can now fill in important gaps in the timelines of the initial investigation and the Wegman panel. But more importantly my review has led to some startling conclusions:
- Not only was the original Barton-Whitfield investigation (in the form of intimidating letters) inspired by the allegations of “climate science auditor” Steve McIntyre, but the defining impetus seems to have been a little known Cooler Heads Coalition-Marshall Institute sponsored presentation by McIntyre and sidekick economist Ross McKitrick in Washington barely a month beforehand.
- Energy and Commerce Committee Republican staffer Peter Spencer played a key but hitherto undisclosed role in the investigation and the subsequent Wegman panel report, and apparently acted as the main source and gatekeeper of climate science information for the panel.
- Steve McIntyre was in communication with the Wegman panel, at least concerning technical questions around replication of his work. The full extent of McIntyre’s communications or meetings with Spencer or other staffers, as well as Wegman panelists, is still unknown. However, the record shows there were at least two intriguing opportunities for face-to-face meetings in Washington during the Wegman panel’s mandate.
All this, along with new information about the circumstances of the Wegman panel’s formation and mandate, raises serious doubts about the supposed independence of the Wegman panel.
[Update, Dec. 19: This post has been substantially revised to remove speculation about Donald Rapp’s possible role in the Wegman report. I apologize for any embarrassment caused to Donald Rapp or Edward Wegman by that speculation.
The post has also been updated to reflect new information about the provenance of Wegman et al’s section on tree ring proxies, as well as more background detail on some of the events leading up to the Wegman report. There are also more details about large swathes of unattributed material found in the Wegman report and in Donald Rapp’s book Assessing Climate Change.
It is clear that the circumstances and contents of both the Wegman report and Rapp’s text book deserve closer scrutiny.
Dec. 20: Comments are now open again.]
As Climategate devolves into a rerun of old battles about the “hockey stick” graph, I thought I would revisit the roots of that benighted controversy and take a look at the chaotic events of a few years back when the politicization of science (also known as the Republican war on science) really took hold.
I was planning to write another installment of the “In the beginning” series on Steve McIntyre. So I decided to take a look at the infamous Wegman report that Republican congressman Joe Barton relied on to ensure that the “numbers added up” (or not, as he was sure was more likely). The 2006 report was the work of a mysterious “ad hoc” committee led by George Mason University statistics professor Edward Wegman, along with David Scott and Yasmin Said. With its near-veneration for putative hockey-stick destroyers Steve McIntyre and Ross McKitrick, and its scornful denunciation of Michael Mann and his “social network” of like-minded researchers, the report has been a touchstone for contrarians.
But then I started thinking about something that had always bothered me. How could a trio of statistical experts, all on their own, hope to write a report on a field, climate science, of which they had no previous knowledge or experience?
Part of the answer lies in the close examination of the Wegman report. Surprsingly, extensive passages from Wegman et al on proxies have turned up in a skeptic text book by contrarian author Donald Rapp. And at least one of these common passages on tree ring proxies closely follows a classic text by noted paleoclimatologist Raymond Bradley, but with a key alteration not found in the original. Moreover, Wegman’s section on social networks appears to contain some unattributed material from Wikipedia and from a classic sociology text.
[This post was updated on December 12 to provide additional context that strengthens the original analysis – see below for details.]
This so-called Climategate is really getting out of hand, isn’t it?
Steve McIntyre has published allegations – twice now – that an internal IPCC authors’ debate about the inclusion of Briffa’s tree-ring reconstruction in a key figure from the 2001 WG1 Third Assesement Report was driven by concern about the post-1960 “decline” in tree-ring widths, a decline that showed a marked divergence with the instrumental tempertaure record.
The Climategate Letters show clearly that the relevant context is the IPCC Lead Authors’ meeting in Tanzania in September 1999 at which the decline in the Briffa reconstruction was perceived by IPCC as “diluting the message”, as a “problem”, as a “potential distraction/detraction”. [Emphasis added]
McIntyre even claims that lead author Michael Mann worried that showing the series with this decline would give “fodder” to “skeptics”.
But even a cursory examination of the emails in question shows that the discussion was really about other aspects of the reconstruction, specifically obvious discrepancies between Briffa’s reconstruction and the other two under consideration over the major part of the reconstruction’s length. Thus, once again, McIntyre’s speculations are shown to be utterly without foundation.
Even worse, McIntyre left out intervening sentences within the actual proffered quotes in what appears to be an unsophisticated attempt to mislead.
The blogosphere, or at least the climate contrarian part of it, has been up in arms over supposed misconduct and plagiarism on the part of climate scientists. First, there was a minor kerfuffle earlier this summer over NOAA’s citation of the surfacestation.org website instead of an online publication by Anthony Watts. That turned out to be just the latest in a series of accusations of misconduct from ClimateAudit.org.
But that was nothing compared to the furor over the Corrigendum to the Steig et al. paper, “Warming of the Antarctic ice-sheet surface since the 1957 International Geophysical Year” (just published in Nature). Steig et al. were accused of failing to acknowledge the role of Hu McCulloch in identifying an error in the calculation of trend significance. The corrigendum thus was held to constitute an act of plagiarism.
Meanwhile, some of the same bloggers who have risen up in righteous indignation and made groundless accusations against Steig et al, have been strangely silent regarding a real act of plagiarism, namely EPA economist Alan Carlin’s wholesale appropriation without attribution of large swathes of Patrick Michaels’ World Climate Report.
And, of course, an examination of the accusations against Steig et al shows them to be completely baseless. In fact, Hu McCulloch has apparently already withdrawn his accusations, although at present we have yet to see any apologies or retractions from Steig et al’s accusers.