SoftestPawn’s Weblog

Oh no, not another blog!

Archive for the ‘Evidence Based Beliefs’ Category

When there is no evidence

Posted by softestpawn on October 13, 2009

or “Now where did I leave my glasses?”

An engineer, a mathematician and a physicist look out of a window of a train passing through the highlands of Scotland, and see a black sheep.

“Ah!”, says the engineer, “Look, they have sheep in Scotland!”.

The physicist looks at it and reflects “Well, we can say only there is at least one black sheep in Scotland”.

The mathematician looks at them both in surprise “That’s not right at all! All we can tell is that one side of one sheep in Scotland is black”

Ho. Ho.

This is one in a series of posts , about evidence and how it does or does not support a claim.

Such pedantry is over the top, but it serves to illustrate a simple point about what we can really infer from a piece of evidence, and its limitations.

It’s easy to decide what to believe when you have clear positive evidence in your hand: the photo of your girlfriend in bed with that sysadmin from finance is fairly firm (heh) evidence of her faithlessness.

It’s not so clear when there is no evidence for something, and interestingly there is more than one way in which we can not have evidence for something.

‘Bounding’ what we don’t know

Those who are old enough will know what it’s like: we lose our reading glasses, or the screwdriver, or pen or mug of tea we had in hand only a few minutes ago.

At this stage, if somebody rather stupidly asks “Well where did you leave them?” we would rightly and angrily reply “I don’t know, if I knew where they were, I wouldn’t have lost them”.

So we don’t know where they are, but we do know some places where they are not. They are not on Mars. And although we may not remember all the rooms we’ve been to since we last remember having them, we tend to remember unusual places; mine are not, for example, in the attic as I know I haven’t been there.

This gives us some limits, some ‘boundaries’, to the area of ignorance.

Reducing the area of ignorance

As we start to look we start to limit these boundaries further.

A quick walk around the usual rooms glancing at the surfaces for example is a good first stage search; it covers a lot of ground for a fairly likely result.

By discounting the places we’ve looked – and by starting with the most likely and easily surveyed places – we reduce the places the glasses could be.

“Knowing it is not” is not “Not Knowing”

Having thoroughly searched the mostly empty fridge, I know to a high degree of confidence that my glasses are not in there.

I have no evidence of that, and I have no proof that they are not (I may have forgotten to search the bowl of three week old leftover gravy) but my memories of looking are ‘evidence of no glasses in the fridge’ (‘evidence of lack’) rather than ‘no evidence of glasses in the fridge’ (‘lack of evidence’).

The latter though is still how you might reply to “Are they in the fridge?”, even though it doesn’t capture whether you’ve looked or not.

This causes problems when people want to know if there’s any danger in some treatment or chemical. To be told “There is no evidence of any harm” is useless; it doesn’t tell us if nobody’s checked, or if they’ve had a quick look and everything seems fine, or they’ve had a really very thorough search that would have turned up any significant harm and found nothing.

What we think we don’t know

So I continue with my exercise in limiting my ignorance, hoping one day to find my glasses so that I can carry on doing what I was doing… whatever that was… it will come to me in a moment… and sometimes we get a bit irrational. How many times, frustrated, have we looked in the same box, under the same small piece of paper that couldn’t possibly hide a pair of glasses?

Similarly our boundary reducing exercise is not ‘certain’; it may be I’ve looked somewhere but not seen them (after all, I’m not wearing my glasses). It may be I’ve looked in an area where they are hidden, and have declared and marked the whole area ‘glasses free’ when in fact it is not.

In more general terms, not finding doesn’t necessarily mean it’s not there: just because all the swans I’ve seen are white, does not mean there are no black swans.

This is where we reach the limits of our understanding of our limits of our ignorance. We rarely properly match the boundaries of what we think we don’t know with what we actually don’t know.

This gap is where my lost glasses still lurk when I give up and use an old pair: in the world of places I haven’t looked well enough, but can’t think of to look.

Theories of what might be

I was slightly too certain above about where my glasses are not, as aliens might have stolen them and taken them to the Mars.

And if we return to the railway carriage with the sheep-observing pedants, we might claim that “There are luminous pink sheep in Scotland with legs on one side shorter than the other. Scostmen hunt them down and turn them into haggis and bagpipes”.

We can make up any silly story we like (“Aliens live in clouds!” “Pixies ate my hamster!” “Magnets healed my cancer!” “Hair loss makes you sexy!”) and some may be accidentally true but it’s no more sensible to assume they are true without evidence than it is to believe in Garibaldi Mountain Shrews.

Not knowing something is no excuse to make up any old thing and then believe it to be true, any more than it is to believe that your glasses are in the kitchen, because you don’t know where they are, and they could be.

This is the ‘out’ for a lot of so-called ‘open minded’ views: “Just because you haven’t seen pixies, you must be close-minded to disbelieve them”.

Pixies might exist, this is true.

But when you consider all the things that might exist, such as invisible baby-eating multi-coloured pixie-swans that live with aliens in clouds, then you can see that believing in any random made up fantasy can be fun but it’s not very practical.

If someone tells you some far fetched story and says “well, you’ve got no evidence against it, so it could be true couldn’t it?” then the answer is “yes, and aliens are painting your ears”

“I don’t know”

A straightforward ‘we don’t know’ seems a bit of a cop out, and the mind abhors a vacuum, but this is no excuse to fill it with speculation and then infer ‘truths’ from them. (It’s fine to speculate and test: Perhaps the glasses are in the bathroom? I shall go and look)

Even if you’ve only ever seen white swans, you can’t be sure that all swans are white. You just might not have seen one that isn’t.

Yet lack of evidence is not evidence of lack; just because we haven’t seen something doesn’t mean it’s not there.

It’s alright to say we don’t know. It’s alright to say we think things are likely, or unlikely, but we’re not sure. And working out what we don’t know – or what exactly we’re not sure about – tells us a very valuable thing: what we still need to find out in order to know.

Background evidence

The above of course is a bit “simple”. It ignores all the background evidence we hold; there are very few sheep with black on one side and white on the other, so we can happily infer that a sheep is black from seeing the one side of it that is. This is material for another post…

Posted in Evidence Based Beliefs, Science | Tagged: , , | 2 Comments »

Deferring to Authority

Posted by softestpawn on August 1, 2009

“Deferring to Authority” is to claim someone’s opinion is valuable because they are an expert in the subject. “Arguing from Authority” is to claim that your opinion is valuable because you are an expert.

This is one in a series of posts about evidence and how it does or does not support a claim. Although this particular one is about opinion, not evidence…

There’s nothing very bad about using an expert’s opinion, but it can be seen as irrelevent in a controversial subject where the evidence is being discussed. If you are arguing about whether homeopathy can cure cancer then you want to look at the facts, not quote people who might have vested interests. Or others who might also have vested interests.

Workable Life

Few of us have the time, inclination, or expertise to carefully check every single thing that we decide. Using expert opinion is a perfectly sensible way to approach life.

There are just a few things to bear in mind:

  • Authority might not be Authority
  • Authority isn’t always expert
  • Authority can be wrong.

Authority might not really be authority

Gillian McKeith is famous for handing out advice on diet, and claims to be an expert on the subject. She even uses the title ‘Dr’ – a title that is restricted in the UK to try and prevent people from claiming a qualification they don’t have. The advertising standards authority were notified, proceedings pursued, and she has been forced to (mostly) stop using it. She has no training, qualifications or authority for her opinions on poo, carrots or lard-fried chips.

But you’ll notice here I’ve not put any links; this is my opinion. It’s based on expertise gathered from far too many hours on Ben Goldacre’s BadScience web site, where the actions to stop her misuse of the title were first formed by some of the forum regulars. So I am an expert, but you should check it yourself, rather than believe me… Google is only a few clicks away…

Authority might not be an expert

Expertise is often quite narrow. People who work in the ‘environment’ industry are not experts in all of it; thus the opinions of neither David Bellamy nor David Attenborough on global warming are expert. It goes further than that; an expert in one particular field of climate cannot claim expertise in the whole subject.

This specialisation can sometimes be quite surprising. Your General Practice doctor has quite a wide medical expertise, but when it comes to a road accident, you will probably be better off with a an amateur St John’s Ambulance volunteer.

Some authority qualifications are so broad as to be meaningless and offer no real authority at all. A “scientist” is generally just somebody who researches something and is no more intelligent for it than many other professions; a “government scientist” in particular is not an expert on everything, any more than you are or I am.

Scientists running in packs (or ‘committees’ as they are sometimes known) are similarly suspect. Pronouncements from Committees of Scientists on things outside their fields of expertise should be treated cautiously.

Authority might just be wrong

This is perhaps too obvious to bother mentioning, but in any controversial field we expect differences of opinions amongst experts; some of them must be wrong.

And sometimes great swathes of experts in a field can be quite spectacularly wrong.

So?

Expert opinion is a perfectly sensible thing to use. My GP’s expertise trumps pretty much anyone else I know on medical matters.

Just remember that sufficient evidence trumps opinion every time.

Further Reading

Argument From Authority (Wikipedia)

Deferring to Authority: Popular Science Communication as a language of control (PDF)

Posted in Evidence Based Beliefs, Metadebates, Politics, Science | Tagged: , , , , , | Leave a Comment »

Do Aliens Live In Clouds? Evidence Based Belief

Posted by softestpawn on July 15, 2009

All day long we are told stuff: stuff that’s true, stuff that’s not, stuff that may be, stuff that’s ambiguous, stuff that’s incomprehensible, stuff that’s interesting, stuff that’s quickly forgotten. Stuff that might save your life. Stuff that might save your hair.

Sometimes it’s stuff that isn’t very important. Did you know, for example, that powdered fish scales are used to clear beer when it’s brewed? Unless you’re a beer-drinking vegan, you probably don’t really care if it’s true or not. Someone can tell you that in the pub and you can nod knowingly or look amazed or dispute it, whatever the social occasion, ego, company and number of beers consumed calls for, and sod the facts, And many of us enjoy aromatherapy or horoscopes without caring whether they really do what they say they will.

But if it’s stuff that’s important then such gossip, rumour and hearsay isn’t good enough. If it’s going to affect our job (is that really illegal?), or our relationships (is she really having an affair?) or our health (will bacon really give you a heart attack?) or our family (will vaccinations really hurt your children? Will not vaccinating them be worse?) or our future (will using fossil fuels really kill us all?) then we need better information.

This is the first in a series of how evidence can help tell us what to believe. It introduces the context and various principles in the paragraphs below, which link to more detailed explanations

So we might defer to authority; that is, we ask someone who is an expert on the subject and use their opinion.

This is a generally workable approach, but if we want more than opinion (expert or not), rumour and gossip then we need facts that are relevant: we need evidence. And we need enough evidence. Until then, it’s just a fantasy.

We need to be clear about what it is we’re trying to show, about whether in fact we can show it, and what that means to our idea.

We are rarely going to get “certain proof”, so we need to understand some ordinary things about uncertainty. It may be that some things are so uncertain that we shall just have to settle for not knowing.

Diversions are everywhere: If someone makes an extraordinary claim, it’s up to them to find evidence to support it, it’s not up to you to find evidence to disprove it. It’s not up to you to provide a workable alternative. And a rubbished alternative doesn’t make the original claim right.

As new evidence arrives, we need to understand how it supports (or contradicts) either a new link in a chain of reasoning, or confirms (or contradicts) existing evidence. We need to check that it is actually new; that it’s not just the same facts wearing a different face.

Most of all, throughout what can sometimes be an eye opening, invigorating and mind-blowing exploration of all this information, we must be careful not to introduce our own bias; that we don’t put our opinion before the evidence, and especially that we collect only the evidence that fits our initially ignorant opinion.

In the end, when someone tells us that “Aliens live in clouds!!”, and it’s important for you to know, then our response is not “That’s silly, because…” but:

“Oh yes? Show me!”

In the meantime, we don’t know if aliens live in clouds, but we have no reason to think so.

And that’s alright.

(With thanks to the badscience discussion forums)

Posted in Evidence Based Beliefs, Metadebates, Science | Tagged: , , , | 5 Comments »