Judge cracks down on Bayesian stats dodginess in court

So when 2 babies from the same family die in unidentifiable circumstances the chances of it being cot death (sudden infant death syndrome) are more like 2 in 3 and not 1 in 6 billion.

Exactly

It isn't that difficult to understand. Shouldn't somebody on the defence team have known this?

One time in my life I actually wished I was on a jury.

An enlightening book, and also relevant due to the case the BBC is bringing up over that nurse convicted of killing elderly patients via insulin overdose, based primarily on the fact he was the only one of the staff who was on shift all the times the 'suspicious' deaths occurred. The only factor in that 'suspicion' being he was on shift at the time... a sort of self-fufilling prophesy made evident by the fact a death was later ruled out as not being suspicious when they double checked the logs and found he wasn't actually around. Sigh. Perhaps some defense lawyers out there should read it, so they know to recognise when statistics are be misused?

>>"My particular dislike of the use of DNA is that everyone is told that DNA is unique, except for identical twins (or clones). What people in court are not told is that they don't sequence the whole DNA (it can't be done, and has never been done)."

Though if people are given probabilities of a random person matching as well as a defendant does, that does at least imply that there isn't uniqueness in the matching process.

And how are you sure what people are told in court ?

Presumably a half-decent defence lawyer could get someone to go into details if they thought it would do some good?

>>"I think that for the layman, they should say something like; everyone's credit card number is unique to them. We found the credit card number of the person who bought the gun, and that 3 of the 16 digits in the defendants credit card number are the same, so therefore it was him."

But that would be highly misleading, since it's close to implying, if not actually implying, that the other 13 digits are different.

Even if someone said the *quite different*

"We found 3 digits from the purchaser's credit card number and the corresponding numbers on the defendant's card match them",

it would still be a fairly poor analogy, since the chance of a random card matching would be 1 in 1000.

has never been done...

slightly pedantic, but AFAIK if your name is/was Henrietta Lacks then your DNA has been sequenced entirely.

Mebbe

I know Ben's done good work on this but he wasn't the source. It's been so generally reported over the years (and no, not all using Ben as their source) that identifying any one source doesn't really work.

Bayes' Theorem

Agreed. I don't see any use of Bayes' Theorem in the two-deaths example as the author presented here. That was a straightforward case of the probability of an event and the probability of the event occurring twice, with the accompanying question of whether the two events are independent.

Possibly Bayes was used to compute the probability of the event in the first place - but if so, that's irrelevant to the case as described in the article.

Bayes Theorem says that if you know the probability of B given A, and you know the probability of A and B on their own, you can compute the probability of A given B. It's pretty straightforward, and it's also irrelevant to the probability of a second "cot death" following a first one. Here the thesis proposed by the prosecution is that A and B are in fact two independent occurrences of the same event; thus P(A|B) = P(B|A) = P(A) = P(B). No need for Bayes at all.

If anything, in that case, it's the defense that should have brought up Bayes, after explaining independence and correlation and other basic concepts.

I haven't read the decision, but I suspect the real issue at hand is the abuse of Bayesian Inference, which is an aspect of the Bayesian interpretation of probability theory. Essentially it's a way to answer the question "how likely is this interpretation of the data to be true, based on our initial probability estimates and the data we've collected since?". That makes more sense, in the context of the first example, where the defense would want to challenge a Bayesian interpretation that misstated the actual posterior confidence.

@Eddie Edwards

>>"THAT'S WHY WE USE BAYES' THEOREM. It's ONLY by applying Bayes' Theorem that you obtain paragraphs like the above."

No, it's perfectly simple to go from "1 in a million" to "there should be roughly 65 matching people in the UK population" by simple logic and extremely basic maths.

Bayes' theorem might be an expression of that, but the underlying logic would be there with or without any theorem, and for a court case, it would seem better to describe something simply in English than start chucking formulas around.

>>"Bayes' Theorem is not an "option", it is a necessity, otherwise travesties like Professor Sir Roy Meadow will happen constantly."

But surely the first problem there was an expert making the *medical* claim/assumption that cot deaths happen at random.

Given that assumption, if the assumption was wrong, wouldn't *any* maths be bound to give the wrong answer?

Indeed

See also: Colin Norris (possibly)

"and applied correctly it is atonishingly good at predicting likelihood"

I think the point the judge was making though is that it wasn't being applied correctly and thus erroneous conclusions were being drawn.

Sorry, are you related to amanfromMars, I didn't understand a word of what you said :-)

I always remember Bayes' theorem as the 'probability of an event B happening/not happening given that event A has happened/not happened, where A and B are independent events'

What a lot of people do is that they attach meaning to statistical results that don't exist e.g. the probability of the numbers 1 2 3 4 5 6 being the winning numbers in the national lottery is the same as any other random 6 numbers, but people attach meaning/ probability to the sequence 1 2 3 4 5 6.

perfectly rational statistics can also give the wrong impression, for example saying 40% of all sick days are taken on a Monday and Friday give the impression that there are excessive sick days taken on Monday and Friday, but Monday and Friday comprise 40% of the working week.

@xlq

>>"What is very worrying is the idea that you can convict something based on probability. Improbable things *do* happen. That is why they are improbable, not impossible."

Thanks for that lecture on probability. I'm sure we all needed it.

It seems you think that 'reasonable doubt' is wrong and courts should only convict based on absolute certainty.

Well, I guess that *would* save a lot of money in the judicial system.

Though on the other hand, when people start taking the law into their own hands, compared to the numbers of people wrongfully convicted, I wonder how many innocent people would get hurt in escalating vendettas, or as a result of wrongful accusations?