Barriers to Open Research

One of the many so-called ‘requirements’ of good scientific research is openness, and the mythical welcoming of criticism that will result.

Humans don’t like doing this and tend to have to be forced to, through rigorous process, reams of paperwork and checks, and evil picky enforcers who come around and badger you to follow the process that not only means you have cleaned out the rotating cell-nutrienting widget breeder, but that shows that you have done so.

Full disclosure is a basic requirement of openness. Such disclosure is not necessarily public; it just means you have to show your working – all your working – so that other people can come along and check it.

Importantly, it also means showing it to people who will criticise it. That’s the point. If you just show it to your mates, there’s all sorts of social conventions that get in the way of proper criticism.

You need to show it to your rivals. To those who really will take it apart piece by piece. Otherwise you’re not really subjecting it to proper scrutiny. You ready for that? You want to do it? eh? eh? Then you’re a Proper Scientist, doing Proper Science. You Urban Spaceman you.

Thoroughness takes time

Because by making it ready for your competition to look at, you will run it through every possible check you can think of, that your mates can think of, your colleagues, boss, drinking companions, any handy six year old child. That’s a lot of work. It’s effort that takes you away from ‘real’ research. And it’s not fun – who likes writing up?

The Reputation

And you might even find stuff that’s wrong, with all the implications that has for your reputation and career and funding, let alone the extra work to fix it. It’s so much easier to just publish the results and ‘lose’ the rest of it… maybe no-one will notice… …perhaps for years …maybe not until you move on.

The Confusion

And besides which, if you publish it people might misread it. Misrepresent it. Those who aren’t experts like you might get ‘confused’ about your conclusions. They might ‘confuse’ others.

As an aside I do find the bandying about of ‘misinformation’ and ‘confusion’ by the more extreme climatologists quite ironic. All information can of course also be misinformation, and any complex subject is likely to be confusing. That they feel they can declare what is information and clear while denying open scrutiny is hubris not expertise.

It’s spectactularly ironic given the repeated use of the term ‘consensus’ – a piece of “misinformation” as it’s badly defined, not properly measured, and “confusing” as it’s not even relevant. 

The Education

A mostly sensible comment via  stoat’s blog brings out another problem with publishing more than you have to: the more you publish, and the more publically, the wider the potential audience of less expert people who will want to poke at your data and ask about it.

Though to use that as an excuse to deny FOI requests for data and code, rather than lessons, is ‘misleading’ to put it mildly.

And to complain that you’re receiving lots of requests for the evidence for something you claim is globally vital for the survival of the human race, seems a little, well, disconnected.

The Conclusion

So there are no real incentives to be open and thorough, and plenty against it. In some industries the long term benefits (ie, showing you’ve got it right) have been recognised and are forced onto unwilling staff, to a greater or lesser extent.

We have seen the effects of not having this imposed on the CRU team; the comments in Harry’s Readme (if it’s real) demonstrate quite how poor the work is understood even internally.  If these had to be prepared for external release, just think of all the extra work required to check everything rigorously. Of course, if it already had been openly published (even not publically), we wouldn’t see comments like Harry’s. Poor chap.

So what does this tell us about other climate research? Nothing really. That’s the point.