The Tired Duck Dilemma

“If it looks like a duck, sounds like a duck and walks like a duck, it’s probably a duck”

A tired duck looking for a safe place to land looks for peaceful ducks on the ground as a sign that an area does not contain predators that would frighten it.

This is also how ducks are shot: an artificial duck is placed in the open, a duck squawker is squawked, and the lure might be moved gently with a fishing line. Passing tired ducks see peaceful duck is peaceful, and fly into the guns of the hidden hunters.

Cautionary tales like this are used to remind us not to judge by appearance; to avoid letting our prejudices drive our decisions without the right evidence.

But that’s a logical failure too. Tired duck is tired; it has to make a decision now, on the evidence it has, about whether to land or struggle to fly to the next possibly safe place. Waiting for more evidence carries risks too.

So what can tired duck do? It can use its background experience – its models of the world, its prejudices, its heuristics tempered by a bit of careful thought – to tell it things about likelihood. Does peaceful duck look and sound and move very much like a duck? Is it the right time of year for that kind of duck look and duck sound and those duck moves that it’s throwing? That judgement will depend strongly on experience in order to ‘fill in’ the assessed situation from tiny bits of evidence. And if tired duck judges it safe, tries to land and is shot, as it plummets to the ground it can always console itself that if it had gone somewhere else it would only have been faced with the same, tired dilemma.

Can We Please Tax The Outrage Bus Service?

For those of you who haven’t been subjected to the original ‘viral’ poster silliness from ’38 degrees’, here it is for reference:

The “Outrage Bus” was introduced by Arrse to describe how to quickly assemble flash-mobs of temporarily outraged people.

It seems there are now people who make a living from it. 38 degrees company accounts are (or were) here: http://38degrees.3cdn.net/04125b8e7c8cf1a742_gim6b3owb.pdf . About half a million profit and no corporate tax, year ending Sep 2011.

Time to Pop Popper’s Popularity

The modern solid sciences are based around the concept of ‘falsifiability’; that theories and hypothesis have to be ‘falsifiable’ in order to be scientifically valid.

Unfortunately there is little if any introspection in the solid sciences (that is left to beardy, sandally philosophy-of-science types, and what do they know of real science?) so solid scientists rarely reflect upon whether this falsifiability is a good, logical, or even scientific approach to research. And frankly, it’s not. This is embarrassing for me as I too, in common with many other from that tribe, have smugly declared that only through hypothesis testing can we do rigorous science. I wonder now, looking back, on what an arse I must have appeared to those with a more, well, scientific approach to science.

This “falsifiability” approach is essentially derived from Karl Popper’s thoughts on provability, given for example in his book “The Logic of Scientific Discovery”. Researchers (or ‘scientists’ or ‘philosophers of science’ or perhaps just ‘curious people’) in the late 19th and early 20th century were struggling with how theories and statements about the world can be supported or otherwise by facts. Popper’s book should really be seen as part of the discussion around concepts of proof rather than as the conclusion about scientific investigation that it has become in some quarters.

Popper started by ‘demarcating’ research: by categorising disciplines into things he thought were science (astronomy, physics) and those he thought were not (astrology and… psycho-analysis…) and then looked for common themes in each to define what makes research scientific and what makes it not. As an initial poke at a subject, looking for common themes is interesting, but as a scientific one it is appalling. It depends heavily on rather personal decisions about which discipline is scientific, and results in a circular rather than scientific argument: ‘I think these things are Science, therefore the way they do work must be scientific, and because they work that way therefore they are objectively Science’. In fact we can’t easily tell whether these disciplines have provided (or not) useful theories about the world because of their practices at the time or in spite of them.

His key conclusion from this personal demarcation was the statement that we can only logically safely deduce statements, not adduce or induce. That is, we cannot prove that theories are generally true, or even true for any untested range; we can only disprove a theory when we discover facts that contradict it. Therefore good theories are ones that can be contradicted.

That’s logically sound, and all very well, but the point about research is we want to find theories that predict. We want to be able to understand what will (probably) happen if we do something we have not done before. We want some idea of confidence in the untested areas of a theory. Deduction is nearly useless.

The result is the pointless and distracting ‘null hypothesis’ introduced to modern experiments. Because you can only ‘disprove’ theories, when Popperians come up with a new theory, they have to invent a ‘null hypothesis’ to have something to disprove in their experiments. Disproving this somehow ‘supports’ the experimenters hypothesis. In fact the null hypothesis has no logical value and its disproof can give a false impression. For example, if you have a theory that a new teaching method can improve student’s reading speed, the ‘null hypothesis’ will be that there is no difference. Now almost any experiment is likely to make some difference to student’s reading speeds, so the null hypothesis is nearly always disproved (there are better ways of framing this particular experiment, but they all revolve around trying to fit around a null hypothesis that has no value except to mark the work as ‘scientific’. Confidence Intervals would be better and more, well, scientific)

What we need, and is currently done with a rather ‘common sense’ rather than rigorous approach, is a systematic approach to understanding what parts of a theory we can be confident in (and for what degrees of confidence) over what ranges.

I’ll be back to you on that…

…possibly…

Debunking the Debunking Handbook

SkepticalScience has published the Debunking Handbook that is intended to summarise how you show an argument is wrong. Unfortunately… it is itself wrong in some fairly fundamental ways.

The summary at the beginning says:

“Debunking myths is problematic. Unless great care is taken, any effort to debunk misinformation can inadvertently reinforce the very myths one seeks to correct. To avoid these “backfire effects”, an effective debunking requires three major elements. First, the refutation must focus on core facts rather than the myth to avoid the misinformation becoming more familiar. Second, any mention of a myth should be preceded by explicit warnings to notify the reader that the upcoming information is false. Finally, the refutation should include an alternative explanation that accounts for important qualities in the original misinformation”

For a start this is phrased to suggest that you already know what is fact and what is ‘myth’. ie, this is not a way of evaluating an argument for its worth or otherwise, but a way of selling a specific argument.

It’s not a debunk manual, it’s a ‘spin’ manual.

First: the refutation must focus on core facts rather than the myth

Any non-trivial problem has myriads of facts that can be interpreted in different ways to suggest different conclusions – this is what makes understanding people, the world and the universe so interesting. This guide says you should push the facts that support your views and avoid analysing those that contradict them. This is far from ‘debunking’ an argument; to focus on specific facts and avoid others is spin.

The example given is the claim that (some) climate skeptics claim that the sun has driven recent climate warming. The debunking is supposedly that the sun’s measured total radiation output does not match warming in the last very few decades, and therefore the skeptic claim is wrong. By itself, this is Fine and Good, but ignores the myriad effects that various solar outputs – different particles and radiation wavelengths – have on the atmosphere and so temperatures. The conclusion may well be right, but the text ignores or oversimplifies the facts that support an alternate view and picks those that support the agenda of the so-called ‘debunker’. This is not a debunk, it’s a sell.

Second: any mention of a myth should be preceded by explicit warnings

This is an obvious statement of intent: a claim that the argument is wrong without saying why.

It’s not even a refutation, let alone a debunk.

Finally: the refutation should include an alternative explanation

This is clearly wrong as it has nothing to do with showing how the initial argument is wrong, and can result in missing the point.

If you claim that aliens move clouds around, I can counter with a similarly clueless argument that the clouds are sentient and move themselves. The discussion can then move to how silly it is that clouds are sentient, and so lose the focus from evaluating the original claim about aliens.

At worst, having shown that it is silly to think that clouds are sentient, a (poor) conclusion is that therefore aliens do indeed move clouds around, as the only alternative considered.

The Worldview Backfire Effect

It is ironic that such a publication should talk about how people are biased by their “worldviews and sense of cultural identity” without considering how they might affect the authors.

In particular I enjoyed the phrase “Self-affirmation and framing aren’t about manipulating people” because, clearly, they are (see also, for example, the very interesting article by Kahneman and Tversky Judgements Under Uncertainty). That’s what is interesting about them.

Removing framing to get at the underlying objective data and arguments is extremely difficult, and will continue to be so while publications like the “Debunking Handbook” encourage others to muddy the waters.

Aggregating Adversarial Argument

I am reasonably intelligent. I am interested in the topic, and so am well read and informed. My conclusions follow reasoned lines, match presented evidence well, and are rationally the most likely. They are scientific. Many other interested and intelligent people have also come to the same conclusions.

You, however, disagree. How can this be? Since my conclusions are rational and informed, you must be biased by your ideology, your vested interests. You must be selectively ignoring key evidence: denying the science. You must therefore be ‘anti-science’. Maybe you have been persuaded by ‘misinformation’ distributed by vested interests and lobbyists, using tried and tested techniques to appeal to your emotions rather than reason. Or maybe you just lack the mental skills to be able to properly assess the complexities and uncertainties.

Convinced? No? Then it must be because one ‘cannot reason someone out of a belief they did not reason themselves into’. I am still the rational, correct one, and you are, basically, unreasonable.

Yeah, right.

This inability to understand, assess, value and maybe even argue convincingly for wide-spread opinion that we disagree with is sign of intellectual weakness in an argument. It suggests ideologically-based bias: we have been too narrowly selective about the ways in which we assess the facts, the reasons and the effects. Can those of us interested in these things make good cases for both evolution and creationism? For homoeopathy? Both for and against late term abortion? Any abortion? For cake? Smoking in pubs? Permitting a new local Tesco’s? How much fruit and veg we ‘should’ eat? And for what?

If we cannot understand and adjust the priorities assigned to evidence and reason that forms the conclusions that many other people hold, then we probably haven’t understood the problem properly. Even if later, with hindsight, we find we have come to the ‘correct’ conclusion, it is likely by accident or social identity, not reason.

This is not about persuading or convincing, or about getting inside the heads of people you disagree with in order to change their mind. It’s about better understanding the problems and their associated issues, and so coming to conclusions that better usefully match the real world.