Elitism & Science

As the he-said-so-he-must-have-meant pop-psychology goes on over the unexpectedly published EA CRU data, some of the discussion turns to how scientists (or, to be more specific, academic researchers) involved should behave professionally.

We’re all – even most academic researchers – human. We can expect Phil Jones and his team to be angry, to scorn those who question his theories, especially when he sees those theories as vital to the future of humanity. And so he does. So would most of us, though we may be a little more careful about committing those things to email.

Dr Spencer, a well known skeptic, has quite a lot to say about such ‘elitist’ behaviour

Good -isms

But there’s nothing wrong with being elite, with being amongst the best at doing a job. And being able to discriminate on merit – on the ability to do things well – is a vital part of any society that intends to improve its lot.

We similarly must discriminate between ages if we want to avoid sharing a wing with Sidney Cook. We discriminate between religions to book holidays, and when providing meals to guests. We discriminate between sexes if we’re heterosexual, or homosexual. We discriminate between sexual preferences to ensure that those that can’t discriminate between ages get to share a wing with Sidney Cooke. We discriminate again between ages to allow certain ages to get away with not making that discrimination.

And this is all good, if a bit Sir Humphrey.

Bad -isms

But if we consider ‘elitism’ as we consider ‘racism’ (discriminating for differences in behaviour or ability that don’t exist), then we’ve got a much more unpleasant attitude. Then we get people who think that their superior expertise gives them remit to protect that expertise by denying evidence to others, remit to use political or organisation clout to deny them access to publish, or remit to disregard any work by anyone else purely because they are not also ‘officially’ elite.

I’m not convinced however by Spencer’s claim that the CRU team are elitist in that way. Yes they believe themselves right, they believe Spencer and McIntyre and McKintrick and all the other hundreds of skeptical scientists are wrong, and they act as the mini tribe that most of us act when we consider ourselves ‘us’ and others ‘them’. There’s nothing particularly unusual with showing they despise people they think are very wrong and are undermining their hard efforts. Even when it’s rather callous.

And there’s nothing particularly evil about abusive comments from experts about other people’s competence. These are arguments over merit, based on their opinions of each other’s work.

Ordinary tribal -ism.

Declaring that those opinions matter only when they are part of the community is not so good. Apparently they are only worth considering when published in approved journals. Journals that publish them are not approved of, and should be ousted from the community. By somewhat underhand means. Which makes for a nice, comfortable, insular, self-reinforcing community, or ‘ivory tower’ as it is usually known.

So they appear to ‘discriminate against’ McIntyre for example because he’s not part of their community, rather than because he’s not ‘elite’. His theories are ‘discredited’ because they are not published in the community journals, rather than because they are wrong.

He certainly doesn’t fit the community: he publishes openly, on t’interweb, where anyone can and does criticise his work (of course, the CRU community is also now doing this, inadvertantly, and they don’t like it). He has a background in statistics, not environment, and he generally sticks to statistical analysis. And while he’s definitely not an enthusiast for The Cause, he’s careful to remain neutral on what the final conclusion will be.

That Science Thang Agiin.

More importantly than disregarding opinion outside the community (we’re all busy anyway, how much time have we got to consider every criticism everywhere?) or the ordinary abuse and wishful thinking, there are the fairly deliberate discussions about (mis)interpreting the data to fit the cause (eg Bishop Hill, Delingpole – these include some rather dubious criticisms of the emails, but some are very telling).

The complete opposite of the much-vaunted stereotyped scientist that is curious about the differences between theory and observation, and investigates them.

Even so, if these particular twonks demonstrate poor professionalism, bordering and perhaps crossing to deliberate manipulation, misrepresentation and destruction of the data, that only means some of these folks do (some are much better behaved). It would be poor science to infer that’s the case for all climatologists, or reflects on the conclusions of the climatology community as a whole.

Though we might want to check that the wider community is more professional – more scientific – in the same way that we might want to check any other organisation for systematic incompetence when we uncover some in a core part of it.

Anyhow, a few paragraphs from Spencer’s article make much better points about how we outside these academic research communities should view the work that they do:

One of the biggest misconceptions the public has about science is that research is a straightforward process of making measurements, and then seeing whether the data support hypothesis A or B. The truth is that the interpretation of data is seldom that simple.

There are all kinds of subjective decisions that must be made along the way, and the scientist must remain vigilant that he or she is not making those decisions based upon preconceived notions. Data are almost always dirty, with errors of various kinds. Which data will be ignored? Which data will be emphasized? How will the data be processed to tease out the signal we think we see?

Hopefully, the scientist is more interested in discovering how nature really works, rather than twisting the data to support some other agenda. It took me years to develop the discipline to question every research result I got. It is really easy to be wrong in this business, and very difficult to be right.

We can see that we need to do better than ‘hope’, if we are to get any reliable science to inform our votes, lobbying and ‘lifestyles’ on this matter.

Update:Judith Curry (I think this climate researcher), talks about tribalism and the duty of public release here

Hacking: It’s Good for Science

Over the last few days the global warming communities – those ‘for’ and ‘against’ – have been deluged by the news that the computer systems at Hadley’s Climate Research Unit (part of the British Met Office*) have been hacked and the data posted on t’interwebs:

• WUWT: Hadley CRU hacked
• Climate Audit
• Real Climate (staffed by many ‘close’ to the CRU victims)

The alleged docs are here (along with on-line searchable access to the emails) but of course, this is the internet, and you can make up anything you like and post it. (Update: Downloading and expanding it, it appears to include 100mb of uncompressed code and data, mostly tree-ring/bristlecone proxy rather than weather station measurements. If this is made up, then someone’s been very busy; but there is also a danger that it is ‘mostly real’ with some key edits)

Assuming for the moment that these are real, and that Phil Jones does in fact admit it, then this is not good for the reputation of Science-The-Human-Endeavour. The tone and contents of the emails squash any claim that ‘you can trust us, we’re scientists, we’re objective and only interested in the facts’ (but then, we know that humans don’t do science)

It doesn’t even help, much, the scientific debate on global warming. As the above discussions show, the main responses are around dishonesty and legality (which are somewhat open to interpretation), rather than analysing the facts and the data. But then the scientific debate has always been very sparse across this general debate; everyone claims to have science on their side and will point to authority, to motivations, to allegiance, to politics, to vested interests, to the number of people working on it, even to assumed ideologies, in order to bolster that claim, but few will actually discuss the science. Well, the science is difficult OK?

But that will come. After the quote mining and short-term tribalist gloating is over, the Big Win for science is the simple straightforward forced releases of data that so far has been kept hidden, for possibly good but still also hidden commercial reasons (That is, CRU wouldn’t show any evidence for why it should be kept hidden, because they claimed to have lost that evidence).

Real Science – that is, the accumulation of a systematic body of knowledge, rather than the insular world of messy so-called-iterative academic research – requires rigour. It requires openness. It requires criticism, whether deserved or not, to tighten arguments and improve evidence quality, and expose gaps and risks. In other words, it requires independent review, or at the very least the threat of it.

Openness is forced internally in any organisation or project that practices ‘due diligence’. We have seen it introduced to medicine in only the last generation or so; many academic organisations** have been reluctant, slow and late to that particular party, for all kinds of ordinary people and practical reasons.

This hack – an externally forced openness – will not do much good in the short term, especially to those involved. But in the long term, we can hope to see researchers who inform public policy become openly professional – and scientific – throughout their work, because now they know that someone, internal or external, may come along one day soon and let unfriendly people examine it. All of it.

Update:Judith Curry (I think this climate researcher), talks about tribalism and the duty of public release here

—

* I’m not actually clear on the differences in responsibilities and allegiances of the Hadley center, The British Met Office, and East Anglia University’s Climate Research Unit. I don’t think they are either.

** And plenty of private organisations too. I’m just picking on researchers whose work is used to drive public policy (and I’ve made some changes to the text to make this clear)

CO2 vs Temperature

The idea that CO2 levels force temperature changes are key to the claims of global warmers. This post looks at how CO2 changes match temperature changes in the records.

Correlation

There are lots of ways to find out how well two sets of values correlate, and a relatively simple way is to plot one against the other. If there’s a good positive correlation, we should get a graph something like this:

We can see that as X is higher, so is Y, and when X is lower, Y is lower. More than that, the larger that X is, so Y is larger.

Negative correlations simply swap direction, and might look like this:

The better the correlation, the closer the points are to a straight line.

This approach can help in noisy or chaotic systems, where a ‘by eye’ glance at data can get taken in by apparent but irrelevant patterns. It can also be useful when there are apparent overall similar trends over a long term.

Pre-Graphing Checks

According to the physics, CO2 slows the re-radiation of heat from the earth. So we expect – at the small changes in CO2 that we see from month to month or year to year – that the more CO2 that is added or removed, the more temperature will change. (The relationship is not linear on the larger scale, but we don’t need to worry about that).

We need to allow for time lags, at least between CO2 changing and the temperature of the instruments recording it changing. Mauna Lao is assumed to be ‘well mixed’ CO2, and the repeating patterns imply CO2 mixing on the order of a month is sufficient. CO2 effects tend to be on the whole troposphere, and there is likely to be some delay while the various temperature ‘pressures’ cause the ground to heat up. It’s hard to see, given daily variations, that this would take much longer than a day, but this assumption is definitely ‘iffy’.

Another potential lag is that of the semi-mythical ‘climate sensitivity’ effects. The more enthusiastic climatologists need these to multiply the ordinary not-very-dangerous temperature rises expected from CO2 change (less than 1C for doubled CO2 levels) in order to get to the scary dangerous temperature rises (3-6C). As these may be decoupled by much more than a month, we cannot assume that any correlation seen is accurate.

The system is very noisy, so we shouldn’t expect a good correlation anyway; there are many effects on temperature other than CO2.

We also need to be wary of seasonal effects. Each year the globe’s CO2 levels fluctuate (as you can see in Mauna Lao’s graph of CO2 levels, which has that ripple effect) and the temperatures similarly change over the year. So we might get a correlation that is a result of the earth’s orbit rather than the cause of one value on the other – the classic ‘correlation is not cause’. So we may have to check against annual values too, which may have their own systematic errors due to the rather arbitrary calendar year cutoff.

As an aside: many people have already plotted CO2 levels against temperature. This is not particularly useful; CO2 has risen nearly exponentially since the 1950s, so plotting CO2 levels is almost entirely equivalent to plotting time.

The Graph

But anyway, using Hadley CRU temperature records, Law Dome ice proxy CO2 records (1850-1978, low resolution thus the stripey effect) and Mauna Lao instrument CO2 records (1959-2008), this is a plot of the month to month (Mauna Lao) or year to year (Law Dome) change in average global temperature against the change in atmospheric CO2 for the same period.

And, rather interestingly, there’s almost no correlation at all. No matter how CO2 changes, the temperature changes in an almost completey independent manner.

With a month lag (ie, comparing the CO2 change with the temperature change of the following month), there’s still none. However if the ordinary CO2/temperature forcing has a lag longer than a week or so (ie, it reaches the same order as the granularity of the records), then this is probably the wrong approach.

Other articles

CO2 Science
Collapse of arguments for high climate sensitivity
CO2 Emissions vs atmospheric increase

Oh no, not the Hockey Stick again

The “Hockey Stick controversy” centers around a graph published by a Dr Michael Mann. It showed how temperatures had remained fairly steady, until mankind started burning lots of coal and oil after which the globe heated alarmingly:

Because the world is mostly American, it’s an ice hockey stick rather than a real field hockey stick. The long steady decline is the hockey stick handle, the sudden rise the blade:

The wiki article linked above outlines the main arguments about whether it’s valid, but there are a couple of other interesting things.

It became centre-piece to the IPCC 2001 report appearing in the summaries, and featuring (if I recall, I haven’t counted recently) eleven times throughout the complete reports.

That overt, specific and limited selection raised skeptic alarm bells. If there is overwhelming evidence for something, then you expect to see overwhelming evidence. Not a focus on one graph from one paper.

1998 as an end date

The first ‘interesting thing’ is that the graph stops in 1998, an unusually warm year, with an end temperature of around 0.7C above the 1961-90 baseline, which is indeed worrying.

Now that’s just a matter of timing rather than fraud (Mann et al’s paper was published in 1998). But by the time the 2001 IPCC report was released it was clear that 1998 had been unusual; 1999 and 2000 were only around 0.25C according to Hadley (0.35C according to GISS).

This rather makes a mockery of the complaints about ‘deniers’ using 1998 as a start point to show that the world is cooling. It’s not right to cherry pick such a year to show a trend now, and it wasn’t right for the IPCC to do so then.

It also indicates a bias in Evil Big Climate; observations that show unusual warming are accepted and published with warnings of runaway effects. Observations of unusual cooling – as in early 2008 – are bounded by warnings of temporary anomalies and ‘noise’

Consensus? What Consensus?

More interestingly it ran counter to the existing ‘consensus’ that there was a medieval warm period and a victorian ice age, supported by various accounts and proxy measurements dotted across the globe.

Which makes a mockery of the warmists cry that a ‘consensus’ should be accepted, even putting aside that a ‘consensus’ is not a measure of scientific fact.

The new Hockey Sticks

The arguments between McIntyre et al and Mann et al led to that particular Hockey Stick being abandoned, and the results of some new studies were collected into a new iconic graph:

As you can see these – and particularly the later ones, coloured red – are no longer Hockey Stick shaped; there is no long straight handle with a marked sudden change at industrialisation. Nevertheless these are called ‘hockey sticks’ by the faithful in memory of the original one.

All that is left is the blade, and even that is artificial. Despite including studies up to 2005 (click on the graph to see the references), the Big Black Line of recent temperature records stops at the steep slope we saw in the unusual year of 1998.

The Latest Hockey Stick

And still there’s some ‘interesting’ choices being made for graphs, these easily-understood, easily-published, easily-misleading summaries of evidence.

Recently Mann published a new one in September 2008:

We can see the medieval warm period and victorian ice ages are returning, more pronounced. The big red line at the end, forming the blade and shooting up past 0.8C by the year 2000, is based on Hadley’s CRUT land only dataset. The grey line hidden behind it is Hadley’s official global temperature estimate.

However CRUs global monthly and average (land) temperatures only break 0.8C in 1998; the average for the subsequent ten years between then and the study being published is 0.6C. Which would put the end of the red line on the graph, rather than have it shoot off into the infinite unknowns in 2000.

(The baselines look similar: CRU land only just passes its own 0C baseline in 1945 according to those records and similarly does so on Mann’s graph).

But the past proxies are not all land only; the black line is land+sea, Moberg 2005 combines ocean and lake data, and Mann and Jones 2003 is of mean surface temperature. So if we replace the red line with Hadley’s global temperature estimate, we get something like this instead:

which, while “unprecedentedly high”, is not quite so alarming.

(Not that we should read too much into the temperature records alone)

With thanks to Sophist et al at Bad Science. See also McIntyre’s more thorough castigation of the original

Apparent Trends and Random Variations

Take a look at these graphs:

You can see trends and features: The noisy but steady drop in the first, a similar but slower rise in the second. The fourth has a slow rise followed by a sharp and maybe accelerating rise. And so on.

The thing is, these are randomly generated, using what are known as ‘random walks’. There are no underlying trends, no feature-causing events. There are just a set of random numbers added together. And these charts have not been specially picked, they are just six consecutive runs of the same random walk spreadsheet. Try it yourself:

Make Your Own Random Walk

Random walks are created where a value is changed by, rather than set to, a random value at each step.

For example, get a willing volunteer to walk across a field tossing a coin; if it shows heads he takes a step to the left and one forward, and if it shows tails he takes a step to the right and one forwards. If he’s really helpful, carries a leaky paint pot, and walks across several fields you get something like those graphs above.

However as it’s raining out and I don’t want to waste any paint, they were created in a very simple OpenOffice spreadsheet: enter the formula A1+RAND()*1 -0.5 into cell B1. This gives B1 a number between -0.5 and +0.5. Then copy it into, say, 200 cells below. Each cell creates a random number between -0.5 and 0.5 and adds it to the previous number.

What is perhaps remarkable, if you think of random numbers as being, well, random, are these long trends of steadily decreasing or increasing apparent trends.

Smoothing and Trends

So those are examples of randomly generated graphs that at first sight look quite similar to many graphs that we get when measuring features of the environment.

For example, if we look at a buoy bobbing about on the sea, it appears to change height from the sea floor in an almost random, unpredictable way, as the overlapping waves, boat wakes, splashes and wind push it about. These small changes are not random, or noise; the measurements are measurements of height at a particular point, and so are signal; they tell us how high the buoy is. However they are chaotic, and so (for most practical purposes) not predictable.

The difficulty is in telling the difference between random systems and ones that do actually have underlying trends. Sometimes simply time well tell: long observations of those random walks above will give us fewer and fewer consistent patterns. Long observations of buoy heights give us predictable tides.

When we’re looking at ways in which complex systems work, we can sometimes find underlying causes by smoothing out the inconvenient small scale changes that confuse the larger patterns. We look to remove the ‘noise’ to reveal the underlying signal.

This requires long observations though, where the patterns can be consistently and reliably repeated, and it requires looking at the right scale in the data. If we smoothed our buoy height data over weeks, we may see seasonal patterns but would miss out the tidal ones.

Linear fits

The simplest trend analysis is to see whether the data is tending to go up or down overall. We can see in the first two graphs at the top above that there is a steady fall; what about the last one? We can find out by a method called ‘linear regression’. The way it works doesn’t really concern us, but using it gives us a line through the data that is as close as it can be to every single point in that data set. The angle of the line tells us how fast the values have been increasing or decreasing, overall.

Sometimes this doesn’t tell us anything very useful. We can see in that last graph a sudden drop at the end; is this merely a disturbance to the underlying trend, or is it part of the underlying trend’s events? Similarly the third graph, with it’s large trough in the middle, doesn’t lend itself well to a straight line.

In fact none of them do. The key thing to remember here is that there is no underlying trend in these graphs; they are merely random numbers added together.

Fooling the Eye

Being humans we tend to look for patterns and trends, and the way our mind is wired we’ll spot them too – even in random data. This is probably because of something excitingly dangerous such as being able to spot predators, prey or mates in the dappled jungles of Africa. Whatever it is, it can lead us astray too, to think we’ve found things we haven’t.

Take the second graph above, and we can see that if we look at low scale trends (trends for short timescales), the trend lines (yellow) are much steeper – up and down – than the overall one (the blue one):

If we look at longer scales, the trend lines (red here) gradually flatten out to be come closer to the blue one:

Until we get to scales of the same order of magnitude as the whole graph, and the trend lines are very nearly those of the overall blue one:

This can lead us to think that the longer trends are ‘better’. But if we have a look at where that graph fits into the much longer run that it was clipped from, we can see that even the overall blue line trend of the clip (above the red bar) has little to do with the bigger picture:

The apparent smoothed trends we see above are only features of the length of the graph. They tell us nothing about longer term trends (well, they can’t, there aren’t any…).

Scale and Granularity

The above are actually clips from runs of 10,000 points. If we look at these longer ones, we can see similar effects: at no point do we start getting an overall smoothing, as the more steps we have, the more likely we are to have long runs of apparently biased direction. Here’s the first and third longer runs (the second is above):

(bear in mind the Y axis scales are different)

So What?

There is no ‘natural’ scale where a noisy-looking system can be smoothed out to. It is tempting to look at the data you have in front of you and fit a trend-line, but without more knowledge behind that data, that trend says nothing about any underlying one without something more.

We need either long enough observations to establish a pattern, and/or enough knowledge about the mechanics of the thing being recorded that we can relate features and trends in the data set to known changes in those mechnics.

For example, if someone’s body temperature is unusually high and increasing, we should worry. We should probably do more than just worry, but it’s not something to ignore because it might be random; I don’t know how the body works in detail, but I do know what a body temperature behaves like; it’s been observed so many times and for so long that patterns have been established, even if working knowledge has not.

So… Global Temperature…

So, yes, the next examples come from my favourite subject, Global Warming, because some people seem to have forgotten that it’s not enough to draw a straight lines through data and imply things about the future from it:

(IPCC showing how trends were increasing in the run up to their 2007 report, Working Group 1, chapter 3)

The recent claims that temperatures ‘are’ decreasing are on similarly shaky ground:

We could even take the full dataset that we have, which for Hadley runs 1850-2009, and look at the apparent trend there (ignore the green patch and line):

But, again, that tells us nothing about future behaviour by itself.

Some of the more frothing deluders enthusiastic GW advocates* say that small changes are ‘noise’ over an underlying trend or signal. However there is very little noise in the records; the values in datasets like Hadley’s are pretty much all signal (ignoring for the moment systematic errors).

There’s a fundamental error in an approach that dismisses inconvenient short-term variations as ‘natural’ but does not understand the range of time scales that ‘natural’ is valid for; there is no reason to assume that longer-term variations are not also natural. The temptation is just to ’smooth’ the data until it looks right to the eye, but that tells us interesting things about how the eye and brain interpret shapes and nothing about the data.

Summary

We really can’t say anything useful about temperature trends by just examining the recent record.

We need knowledge of how the climate works, captured usually as models. A lot of people are working hard to understand the climate based on the various records of various measurables; but most work on some small aspect of it, few but the most enthusiastic deluders claim anyone has complete understanding. And the good quality data is fairly recent, there are huge systematic problems with it (such as surface station placements, urban heat island effects, tidal station changes) and for anything more than a few decades we tend to have to use proxies and add another layer of systematic problems.

When we check the models, they need to be checked against features of the dataset, not carefully selected subsets or the overall trend. So we’ve seen a steady rise of CO2; why did temperature rise 1910ish-1940ish at the same rate as recently when little man-made CO2 was present? ie what was that natural variation and has it been included in our knowledge base – and eliminated as a candidate for the recent rise; do the models that ‘predict’ the 1980-2000 rise also ‘predict’ the earlier periods?

These models and their validation are key; it’s not sufficient to establish underlying trends by drawing a straight line through some data.

(Audit, full disclosure, etc: Example spreadsheet to create random walks and zip file of the 6 runs made )

* I must remember not to use the same tone here as I would in forum arguments. Apologies to tamino who was not at all frothing below.