cause, chance and Bayesian statisticsa briefing document 

Cause, chance and Bayesian statistics is one in a series of documents showing how to apply empiric reasoning to social and psychological problems..  


Bayes [1], Thomas 17021761. An English theologian and mathematician who was the first to use probability assessments inductively. That is, calculating the probability of a new event on the basis of earlier probability estimates which have been derived from empiric data. Bayes set down his ideas on probability in “Essay Towards Solving a Problem in the Doctrine of Chances” (1763, published posthumously). That work became the basis of a statistical technique, now called Bayesian statistics. 
Index 

A key feature of Bayesian methods is the notion of using an empirically derived probability distribution for a population parameter. The Bayesian approach permits the use of objective data or subjective opinion [2] in specifying a prior distribution [3]. With the Bayesian approach, different individuals might specify different prior distributions. Classical statisticians argue that, for this reason, Bayesian methods suffer from a lack of objectivity. Bayesian proponents argue, correctly, that the classical methods of statistical inference have builtin subjectivity (through the choice of a sampling plan and the assumption of ‘randomness’ of distributions) and that an advantage of the Bayesian approach is that the subjectivity is made explicit [4]. However, a prior distribution cannot easily be argued to be strongly ‘subjective’. Bayesian methods have been used extensively in statistical decision theory. In this context, Bayes's theorem provides a mechanism for combining a prior probability distribution for the states of nature with new sample information, the combined data giving a revised probability distribution about the states of nature, which can then be used as a prior probability with a future new sample, and so on. The intent is that the earlier probabilities are then used to make ever better decisions. Thus, this is an iterative or learning process, and is a common basis for establishing computer programmes that learn from experience (see Feedback and crowding). Black and blue taxisConsider the witness problem in law courts. Witness reports are notoriously unreliable, which does not stop people being locked away on the basis of little more. Consider a commonly cited scenario. First piece of data: Second piece of data: Bayesian probability theory asks the following question, “If the witness reports seeing a blue taxi, how likely is it that he has the colour correct?” As the witness is correct 80% of the time (that is, 4 times in 5), he is also incorrect 1 time in 5, on average. For the 15 blue taxis, he would (correctly) identify 80% of them as being blue, namely 12, and misidentify the other 3 blue taxis as being black. For the 85 black taxis, he would also incorrectly identify 20% of them as being blue, namely 17. Thus, in all, he would have misidentified the colour of 20 of the taxis. Also, he would have called 29 of the taxis blue where there are only 15 blue taxis in the town! In the situation in question, the witness is telling us that the taxi was blue. But he would have identified 29 of the taxis as being blue. That is, he has called 12 blue taxis ‘blue’, and 17 black taxis he has also called ‘blue’. Therefore, in the test the witness has said that 29 taxis are blue and only been correct 12 times! Thus, the probability that the taxis the witness claimed to be blue actually being blue, given the witness's identification ability, is 12/29, i.e. 0.41. When the witness said the taxi was blue, he was incorrect therefore nearly 3 times out of every 5 times. The test showed the witness to be correct less than half the time. Bayesian probability takes account of the real distribution of taxis in the town. It takes account, not just of the ability of a witness to identify blue taxis correctly (80%), but also the witness’s ability to identify the colour of blue taxis among all the taxis in town. In other words, Bayesian probability takes account of the witness’s propensity to misidentify black taxis as well. In the trade, these are called ‘false positives’. The ‘false negatives’ were the blue taxis that the witness misidentified as black. Bayesian probability statistics (BPS) becomes most important when attempting to calculate comparatively small risks. BPS becomes important in situations where distributions are not random, as in this case where there were far more black taxis than blue ones. Had the witness called the offending taxi as black, the calculation would have been {the 68 taxis the witness correctly named as black} over {the 71 taxis the witness thought were black}. That is, 68/71 (the difference being the 3 blue taxis the witness thought were black); or nearly 96% of the time, when the witness thought the taxi was black, it was indeed black. Unfortunately, most people untrained in the analysis of probability tend to intuit, from the 80% accuracy of the witness, that the witness can identify blue cars among many others with an 80% rate of accuracy. I hope the example above will convince you that this is a very unsafe belief. Thus, in a court trial, it is not the ability of the person to identify a person among 8 (with a 1/8^{th}, or 12.5%, chance of guessing ‘right’ by luck!) in a prearranged line up that matters, but their ability to recognise them in a crowded street or a darkened alleyway in conditions of stress. Testing for rare conditionsVirtually every labconducted test involves sources of error. Test samples can be contaminated, or one sample can be confused with another. The report on a test you receive from your doctor just may belong to someone else, or be sloppily performed. When the supposed results are bad, such tests can produce fear. But let us assume the laboratory has done its work well, and the medic is not currently drunk and incapable. The problem of false positives is still a considerable difficulty. Virtually every medical test designed to detect a disease or medical condition has a builtin margin of error. The margin of error size varies from one test procedure to another, but it is often in the range of 15%, although sometimes it can be much greater than this. Error here means that the test will sometimes indicate the presence of the disease, even when there is no disease present. Suppose a lab is using a test for a rare condition, a test that has a 2% falsepositive rate. This means that the test will indicate the disease in 2% of people who do not have the condition. Among 1,000 tested for the disease and who do not have it; the test will suggest that about 20 persons do have it. If, as we are supposing, the disease is rare (say it occurs in 0.1% of the population, 1 in 1000), it follows that the majority (here, 95%, 19 in 20) of the people whom the tests report to have the disease will be misdiagnosed! Consider a concrete example [5]. Suppose that a woman (let us suppose her to be a white female, who has not recently had a blood transfusion and who does not take drugs and doesn’t have sex with intravenous drug users or bisexuals) goes to her doctor and requests an HIV test. Given her demographic profile, her risk of being HIVpositive is about 1 in 100,000. Even if the HIV test was so good that it had a falsepositive rate as low as 0.1% (and it is nothing like that good), this means that approximately 100 women among 100,000 similar women will test positive for HIV, even though only one of them is actually infected with HIV. When considering both the traumatising effects of such reports on people and the effects on future insurability, employability and the like, it becomes clear that the falsepositive problem is much more than just an interesting technical flaw. If your medic ever reports that you tested positive for some rare disorder, you should be extremely skeptical. There is a considerable likelihood the diagnosis itself is mistaken. Knowing this, intelligent physicians are very careful in their use of test results and in their subsequent discussion with patients. But not all doctors have the time or the ability to treat test results with the skepticism that they often deserve. How bad can it get?In general: The more rare a condition and the less precise the test (or judgement), then the more likely (frequent) the error. Consider the HIV test above. Many such tests are wrong 5%, or more, of the time. Remember that the real risk for our heterosexual white woman was around 1 in 100,000, but the test would indicate positive for 5000 of every 100,000 tested! Thus, if applied to a low risk group like white heterosexual females (who did not inject drugs, and did not have sex with a member of a highrisk group like bisexuals, or haemophiliacs, or drug injectors) then the HIV test would be incorrect 4999 times out of 5000! In general, if the risk were even less and the test method still had a 5% the error rate, the rate for false positives would be even greater. The false positive rate would also increase if the test accuracy were lower. 
Related further reading  
Intelligence: misuse and abuse of statistics  drugs, smoking and addiction  
establishment psychobunk  cause, chance and Bayesian statistics  
For related
empiric reasoning documents, start with Why Aristotelian logic does not work 
Endnotes

email email_abelard [at] abelard.org © abelard, 2002, 13 october the address for this document is https://www.abelard.org/briefings/bayes.htm 2200 words 
latest  abstracts  briefings  information  headlines  resources  interesting  about abelard 