SoftestPawn’s Weblog

Oh no, not another blog!

Visualising the Remote Sensing Satellite data

Posted by softestpawn on March 5, 2011

To practice getting back into coding again, I wrote a quick visualiser for the global temperature satellite measurements, and it’s quite pretty so seems worth presenting.

The RSS data is already presented as graphs and outlines here, but I thought it would be interesting to see what happened across the globe over time.

The data files for the TLT (near surface ‘brightness’) from here:

I plotted these using Java into four panels:

  • top left simply shows brightness corresponding to the absolute temperature
  • top right is the anomaly according to the average data set. ie, each pixel was averaged over the time series, then for each time step the difference is shown. Negatives are blue, positives red.
  • Bottom left is the anomaly averaged over 12 months to remove seasonal effects. ie, each pixel was averaged as above across the whole series, then again for 6 months previous and subsequent to the time step, and the difference shown. Negatives are blue, positives red.
  • Bottom right is as above but over 24 months.

A typical frame looks like this:

Lots of not rights here: I’m not quite sure which months correspond to exactly which year, I’ve not got the colour distributions right so there are occasional wierd pixels, and the shape isn’t right.

It’s still quite pretty; you can see the seasonal shifts from north to south hemisphere, and the outline of the continents on the absolute temperature frame top left. The bottom two show the changing patterns of local weather over the medium term.

I assembled a subselection into an animated gif; every 3rd month and about half (1994-2007ish) the full time range (1985-2011) . This is a subset as my GIF animator only handles something like 70 frames (which is fair enough, should really find out how to do this as a proper video) which hopefully will show here, though it’s an 11MB file:

Code is available if you want it.

Used a trial version of A4Video (looks useful) to assemble this video for most of the time range (it’s top-and-tailed a bit):


Posted in Environmentalism, Global Warming | Leave a Comment »

BBC’s (Im)partial Science Reporting

Posted by softestpawn on September 23, 2010

The BBC is holding another review on its impartiality, this time on how it presents scientific subjects: Science impartiality review – terms of reference (PDF). It has existing guidelines, and it has held such reviews before on how it reports on subjects such as religion and the middle east. This is all Good Stuff, as the BBC’s reputation rests somewhat on the quality and reliability of its reporting, and reliability requires, among other things, impartial reporting. 

One of the many frustrations for medically trained scientists however is the airtime and article space given to ‘alternative’ treatments such as homeopathy, reiki, accupuncture and so on. These are treatments that have not passed the objective tests used to identify those that actually work. These tests (double blinded, randomised control groups, etc) are meant to bypass the personal and social prejudices and biases that affect our abilities to properly evaluate effectiveness. They do not always succeed.

The concern is largely that by giving publicity to unproved, useless and sometimes dangerous treatments, the BBC lends them credibility and authority, and so more people may be taken in by them.  By providing BBC publicity to such sites as JABS, people may believe them to be officially sanctioned.

And so these concerned people do not want the BBC to give equal space to these cranks, charlatans and quacks. Such reporting is not truly balanced, they claim. If you’re going to report science, they say, you should report scientific science not pseudoscience.

Scientific science vs pseudoscience

Which all sounds well and good, but the BBC does not have the funds or indeed the expertise to properly evaluate every controversial issue.  For a start, only a few controversies can be tested in the clearly objective way that medical treatments can. 

The BBC may instead decide to defer all evaluation to certain establishment scientists and report only the expert opinions of people with certain qualifications from certain institutions; but this is not scientific. It’s not uncommen for academic research scientists to fall prey to their own or others pseudoscience, even in related fields.

Nor does the BBC have the remit to make such evaluations or deferrals. A public controversy is one with many people who believe opposing things, for frequently unscientific reasons, and the BBC’s audience is public. If the BBC were to fail to report the views of such people and how they were derived, then it is failing to engage with or inform the discussion.

The concerned may argue that such a discussion is not a scientific one: a programme on ghosts has no place under the Science label for example. Yet the evaluation of sparse evidence is vital to science; a negative result is still a useful result. And we need not be sheltered from uncertain and ambiguous evidence, leaving us to make up our own minds – this too, is science. 

Impartial to the audience, not the evidence

Impartiality is not the same as correctness. The BBC can and should provide time to the different parties in a discussion that the general public is interested in.

This doesn’t mean having to give airtime to any old crackpot view, but if large proportions of the public are, say, worried about vaccinations then it is quite right of the BBC to air those concerns along with objective evaluations of them. The BBC rightly provides a platform for those advocates to present their case to the public, for the public to evaluate. 

The public – everyone – is indeed ignorant and stooopid about most subjects (who has time to evaluate everything?). But to be protected from our own folly and expertise by filtering what is presented to us leaves us in the hands – and frequently inexpert opinions – of those doing the filtering. 

So yes, let’s have links to sources so we can check back and do our own evaluations. Let’s have more entertaining educating articles and programmes such as those from More-or-Less and Ben Goldacre. And let’s have more time to hear the cases rather than have them forced into small soundbites. 

But let’s not start letting partisan groups decide on our behalf what we should hear about when it comes to science topics. Because that’s really not good scientific practice.

From Stuff and Nonsense &  DC’s Improbable Science, although the review started back in March. A cutdown version of this has been sent to the BBC’s feedback email:

Posted in Bad Journalism, Evidence Based Beliefs, Science | Tagged: , | Leave a Comment »

Not “Winning the War on…”

Posted by softestpawn on July 31, 2010

“The war on” terrorism, drugs, crime, poverty and so on frames these activities as a struggle with an end: either victory or defeat. As with the Second World War, the armies will destroy the enemy’s will to fight, the victors will introduce a Marshall Plan to fix everything that got broken, and then everyone settles down to enjoy peace and posterity.

If the end does not arrive nice and promptly, that the war is going on ‘too long’. The activity may not even be winnable. And if it’s not winnable, then we should give it up as pointless.

This metaphor seems suitable when ‘The War On..’ involves real war. The invasion of Iraq and the struggle for control of areas of Afghanistan had and have soldiers and gun battles and people deliberately trying to kill each other.

But real actual killing warfighting is only a fairly specific component of even a military-backed mission. Without other activities the overall intent can fail: the coalitions plainly won the war to control Iraq. They just never quite established that control themselves, and what they did they lost – through ‘factors other than war’. Even within modern armed conflicts we rarely operate with the clear remits for killing that we associate with clear warfare. The rules of engagement in Afghanistan are heavily restricted to reduce casualties amongst civilians; this necessarily prolongs the fire fights but is intended to improve the overall situation.

The wars on crime, on drugs, on poverty are more obviously foolish as useful metaphors.  There is no expected end state on any of these; we don’t expect crime to ever disappear, and definitions of poverty tends to change as our overall standard of living improves.

We could just give up the struggle; ‘the war on drugs’ could be ended (in defeat? victory?) by legalising all drugs. While it solves a few issues, it doesn’t suddenly solve debilating addictions. Some activities-other-than-war would still be needed.

Using metaphors like ‘the war on’ doesn’t just hide all these other important activities, it makes it difficult to appreciate them and balance them, and it provides the wrong references for judging progress.

‘Fighting crime’ for example involves quite a lot of not fighting at all: improving education to improve prospects, encouraging employment, changes to social and community attitudes.

A much more appropriate metaphor is gardening. There is no end-state ‘win’ in gardening; there are desired states for the garden or parts of it but work does not stop when you reach it. Indeed, some gardens require tremendous effort to maintain. You can change your intended result to make the work easier or harder,  but you have to keep working to maintain an approximation to that result.

And while “Crime Gardening” doesn’t really cut the spin mustard, some metaphors might be usable: Weeding Out Crime, Landscaping A New Society, Cultivating Good Relationships, and probably something about Roots.

Posted in Politics | Tagged: , , , , | 3 Comments »

The Painful Subject of… Interrogative Torture

Posted by softestpawn on July 20, 2010

Some time ago, following memos and documents released from the interrogations at Guatamalo and others, some writers argued that torture doesn’t work:

and a raft of others.

In this article, long awaited by both readers of this blog, I’m going to ask in a rather Delve-special way:

  1. Does torture work?
  2. Why, then, do some people claim that it does not?
  3. What do these claims and  the way they are supported tell us about the way ‘evidence-based policy’ should be examined and tested?

For the purposes of this article, torture is direct physical pain or maiming. Whereas this, including invasion of personal space, isolation and sleep deprivation and this use of psychologists are not. They are certainly unpleasant, but not in the same league as having – or threatening to have – your fingernails torn out. If they are ineffective it might simply be that they are not gruesome enough.

Also, I’m only looking at how it ‘works’ as a direct interrogation technique, rather than whether it ‘works’ as a suitably effective social tool.

While I go over the collected evidence, consider these two situations:

  1. A bloke knocks on your door and tries to persuade you to give him your car keys.
  2. A bloke breaks in through the door, pokes out one of your eyeballs and threatens to tear various new holes through your particularly painfuls until you tell him where your car keys are.

Where Torture Works

There is, I think, no need to give examples of victims under torture confessing to all kinds of things that they may or may not have done. Even the articles above recognise that ‘with torture you can make people say anything’. While this is rightly considered somewhat beside the point, if torture can force people to ‘confess’ to shameful things that would be against their principles, it should not be surprising that it might force people to confess what they know.

All the same, let’s have some evidence.

For fairly obvious ethical reasons there is little in the way of modern reliable random controlled trials on the efficacy of torture. Similarly, many regimes with widespread routine torture are thankfully gone and their records not generally available.

Lack of reliable monitored evidence however is not evidence of lack of effectiveness.  We can start by looking at historical accounts.

There is indirect evidence in the precautions taken by WW2 spies who were routinely going into environments where interrogative torture was likely. Organisation cells were created to prevent single points of failure, and key locations hidden from the staff.  This tells us that people who worked in these environments expected torture to give away information. It can be argued from the safety of this armchair that this is only circumstantial evidence – perhaps that fear was unfounded – and so should be discarded.

When it comes to more direct examples, despite the courage required to resist such terrible action, a stigma is attached to ‘breaking’ under torture.  More frequent – more newsworthy, more heroic – are accounts of resisting.

So where accounts give examples of victims revealing information, they are usually not named.  Some do and are controversial, for example Rene Hardy is said to have betrayed Jean Moulin among others.

Some less so include this obituary of Louis Handschuh, where an escapee is captured and tortured and gives up the names of thirteen others. And in this obituary of Andrée Peel, she was “betrayed by a fellow agent who had been arrested by the Gestapo and threatened with the torture of his family”

Accounts of Miguel Enriquez’s discovery and death in Cuba commonly give it due to MIR members talking under torture (and here)

Zoya Kosmodemjanskaja is betrayed by a captured colleague

Von Ruffin is named by another gay man who was tortured.

Succumbing to torture, someone named the entire Daman family

Henry Ballard betrays under torture the other members of the conspiracy to place Mary on the English throne

Turning to more ancient examples, the conquistador Pizarro tortured Incas to locate their king Atahualpa

Sinan Pasha captured and tortured – by impaling – Prince Jem’s couriers to force them to reveal their messages, as described by Freeman in “Jem Sultan“.

Rather famously for UK readers, Guy Fawkes was tortured over several days to extract the names of his co-conspirators, leaving him barely able to sign his confessions.

And so on.

None of the above however are primary (first-hand) accounts, and apart from Pizarro’s are poorly verified.  More careful work has been done by Darius Rejali and describes situations where it most definitely worked. Maybe not efficiently, but it’s not easy to directly compare that efficiency with that of any other method (more later).

In practice, however, we can see evidence all around us of how pain – or the threat of pain – causes us to do, say or give things we would rather not. We even have a name for it:

Robbery: Theft with Violence

Not all the violence in robbery is to cause compliance under duress. Sometimes it is to disable the victim. Sometimes to temporarily prevent the victim from interfering – a shove out of the way, a push to the floor. Sometimes it is for the fun of it.

But there are hundreds of thousands of robberies a year in the UK, and plenty of examples where pain or death – or the threat of it – resulted in compliance:

…also held up a newsagent at knifepoint and stole £150 earlier in the month…

…if you shout any more I will break your neck…

…assaulted her while demanding money. He then fled with a three-figure some of money…

…The men armed with weapons dragged Mr Bowers-Lovett from his bed and demanded he opened a safe. They continued to assault him and his wife before they escaped with a substantial amount of cash…

…two males rushed towards him from behind before threatening him and demanding he hand over his possessions. He handed over his wallet and mobile phone…

…Two men forced their way into the premises and threatened an 64 year old man with a knife before demanding that he handed over his money. The suspects then fled the scene with a sum of cash…

…he was confronted by a male carrying a knife, who demanded he hand over his money and mobile phone.  The 27-year-old complied…

…five masked men kicked down the front door of his house on Kingscroft Close, Streetly. They beat the victim, demanding the keys to his [car]… Also in the house were the man’s wife and two children…

…two men appeared behind her and pushed her inside, demanding that she open the safe. The men then escaped with a substantial amount of cash…

…Two men approached a 28-year-old man and threatened him with a knife, demanding the keys to the car. They punched him two or three times before driving off in it…

…assaulted him and his 21-year-old house mate. The housemate was then taken to a nearby cash point … by one of the robbers, while the other man was kept at home by the other robber. Both men left the address when they had returned from the cash point…

And that’s just a tiny sample of the terrifying ordeals where the victim complies under duress. Few approach the even more daunting prospect of facing deliberate torture for days, months or years on end.

The occasions when the robbery is resisted are more public, more newsworthy, more rare. While we can call these people heroes (or stubborn, perhaps) that is not a reason to call those who folded cowards. It is normal to succumb.


There are also of course lots of examples of victims resisting and saying nothing, or giving false information.  This appears to be the main argument in the articles linked to at the beginning: a rather extreme ‘test case’ (getting the location of a bomb from a terrorist) is proposed and then used as a representative example. And since the victim may lie, you can’t trust torture, and therefore torture does not work.

And there is some watery reasoning that torture, because it makes victims more suggestable and confused, also makes their testimony less reliable, and again, since it is not completely trustable, therefore it is useless.

Indeed, if you put someone on the rack in order to find the location of the bomb, and you hurt him until he tells you stuff, you can’t tell if what he’s told you is true. But then, anyone under any kind of interrogation may be lying to you, or confused, or suggestable. We might as well not bother asking anyone anything. All the same, torture may be about as reliable as any other interrogation but with more shouting, moral turpitude and cleaning up.

And so we hit the nub problem of any interrogation: how does a competent interrogator ensure that what he gets is good, useful information and not just what he wants to hear, or even biased by his own preferences?

It’s a good question. A difficult question. And if you don’t work out an answer soon, we’ll send someone around to beat one out of you.

Improving Reliability

Generally speaking, you check what has been said against what has or can be verified, against previous testimony, or other testimony. Once you’ve got those loops in place, it can be much easier to ‘encourage’ the victim to be truthful. It’s where the ‘ticking bomb’ scenario above falls down – it assumes that if you can’t verify information given, then torture might not work (as indeed any other interrogation might fail) and therefore torture doesn’t work at all.

It seems generally accepted that, when broken, “people will say anything to make the pain stop”, which is of course exactly the idea. They may make stuff up, but a competent interrogator does not take what is said at face value.

If you’re in the business of gathering contacts for example, false information leads to to dead ends. Useful information can lead to more evidence. And if your victim is still in the loop – if you come back to them having checked their information – then this can be used to ‘encourage’ better performance.

And if they know you will hurt them if they get anything wrong once it’s been verified, there’s a very good incentive to tell the Truth in the first place.  An incentive that isn’t there if you’re just chatting over tea and biscuits.

Evidence Based Policy

Evidence based policy is the new religion amongst some people, including myself and at least two of the bloggers linked to at the start. But to use it we need to  understand what it actually means.  Ben Goldacre, for example, is  a great public advocate of it but even he confuses written reports with ‘sound scientific evidence’.

We need to consider what evidence is valid, how much is sufficient, we need to have systems to include new evidence to adjust and perhaps reverse policy, methods to cope with uncertainty and incompleteness and poor quality, and mitigatations for  biases in its collection and summary.  And we need to remember that sometimes it is irrelevant.

It is difficult to ‘prove a negative’ (such as proving that aliens do not live in clouds, or that torture does not work), and when assembling circumstantial evidence to try and do so, it is vital to consider contradictory evidence and remove duplications under different disguises before drawing conclusions from it. Particuarly, it’s important to remember that just because something sometimes doesn’t work, this is no reason to reject that it does work sometimes, or even usually. “Smoking kills”, for example.

The supporting evidence given by the writers above are examples of collection bias, of morally-driven analysis:  the evidence collection (in this case, largely the blindness to widespread contradictory evidence) is shaped by the wishes of the collector.

Perhaps this bias is because it is a morally fraught subject. Dearly held fundamental principles tend to contradict each other when trying to weigh up the pros and cons of whether, or under what conditions, we as a society should tolerate torture. Some of them do not even compare well: the principles of acting vs not acting, of saving lives vs not being ‘dirtied’. Pretty much any conclusion is imperfect, and would result in pain and loss for some people somewhere.

Wouldn’t it be easy to just bypass these quite horrible comparisons? And gosh, here’s an opportunity. After all, if torture doesn’t work, then there’s no need to have to do all that difficult introspection.

By ‘removing’ the utility argument, we remove the need for any difficult moral argument.

The difficulty of the moral question encourages the selection, ignoring, twisting and shaping of evidence to fit the desired policy rather than the other way around. The intent, possibly, is to show that torture does not work, whether or not you think it’s moral, so therefore there’s no practical reason to use it. Rather than that torture is morally indefensible, whether or not it works.

So Torture Works?

Of course it does. Would you not have given up your car keys? Of course you would. Life and health is more important than a car. But would you have given up your children?  That’s a very different motivation.

So torture works sometimes, in the same way that “Smoking kills (sometimes)”. The reverse is not true: “Smoking does not kill” is a difficult claim to support.

And there are (sometimes) more effective methods available. Ordinary persuasion, perhaps via the introduction of alternative views from respectable leader figures, appears to (sometimes) offer good results. Someone who is converted will (sometimes) offer far better cooperation, but, of course, it’s only sometimes possible to convert dedicated foes.

Far more important is that we recognise torture’s every-day efficacy when we take a moral stand against it. Taking defenceless human beings and deliberately inflicting pain on them is a horrific thing to do. Robbery works, and it can tell us that torture works, but it’s not a reason to condone either.

The Moral Argument

Even putting aside the evidence list above, it is simply not good enough to claim that we shouldn’t torture because we have no evidence that it works. It remains a moral argument as follows:

(1) If someone produces evidence that certain methods of torture do work, this is not a reason to take it up as a tool of the community. The reasons not to torture are independent of whether we have thorough proof of its efficacy right now.

(2) There’s a huge number of people who have encountered torture, and as the world population connects up they become closer to us. To claim, loudly and widely in papers and blogs, that ‘we’ believe torture doesn’t work divides the comfortable armchair writers from the people who have ‘been there’. In particular, it shames those who have broken under it, who in my armchair view have no reason to be ashamed . Imagine, if you will, someone who has been through days, weeks, months of such hell and given away names of friends and family and colleagues, and telling them: “Torture doesn’t work”

(3) Most importantly, this blind-eye attitude cheats our ability to think through our ethics, and our ability to take a proper moral stance:

I oppose torture not because some dodgy armchair reasoning tells us it might not work sometimes.

I oppose it because my armchair principles call it vile.

Posted in Politics, Science | Tagged: , , | 7 Comments »

Most Scientists do Science

Posted by softestpawn on May 10, 2010

Following the letter in Science magazine, from scientists defending scientists when they should really be defending science, there was this editorial from Brooks Hanson.

Brooks makes a few points far more politely than I do about some of the shortcomings in some disciplines, and this one I think is particularly pertinent:

“The scientific community must recognize that the recent attacks stem in part from its culture and scientists’ behavior”.

But it follows the same odd cultural attitude that permeats much of research, that conflates ‘scientists’ with science – that lumps up researchers with the body of knowledge that society’s technology and wealth and health relies on.

What is this scientific community? Is he talking about academia here? In which case part of the recent attacks do indeed stem from some attitudes and behaviours of some academics. And when other academics leap to defend such people, calling on vague mythical ‘scientific processes’ (or ‘peer review’, or ‘intellectual scrums’) that are supposed to make academics trustworthy, then yes the tarring brush gets bigger.

But Brooks takes this much further and tries to include everyone. In particular I take exception to this:

Scientists must meet other responsibilities. The ability to collect, model, and analyze huge data sets is one of the great recent advances in science and has made possible our understanding of global impacts. But developing the infrastructure and practices required for handling data, and a commitment to collect it systematically, have lagged. Scientists have struggled to address standardizing, storing, and sharing data, and privacy concerns.

Actually, some scientists (astronomers, medics, gene researchers, taxonomers, seismologists, simulation writers, geologists, say) have long developed the infrastructures and practices and standards required. In some cases by taking those methods from the body of knowledge (the ‘science’) that exists in the commercial world.

Perhaps Brooks is looking for a way of spreading the blame around, of not picking on particular teams. But it hampers the very public perception that he says he’d like to improve, as it lumps all scientists up under one theoretical culture which has been tainted by the actions of a few, but which doesn’t exist everywhere.

In fact, it may well only exist in a very few remaining places, and it’s not right to pass off poor working practices of some groups onto others without cause, especially if they only happen to share a similar job title.

Posted in Science | Tagged: , | Leave a Comment »