cause, chance and Bayesian statisticsa briefing document 

Cause, chance and Bayesian statistics is one in a series of documents showing how to apply empiric reasoning to social and psychological problems..  


Index 


Black and blue taxis  
Testing for rare conditions  
How bad can it get?  
Endnotes  
IntroductionBayes [1], Thomas 17021761. An English theologian and mathematician who was the first to use probability assessments inductively. That is, calculating the probability of a new event on the basis of earlier probability estimates which have been derived from empiric data. Bayes set down his ideas on probability in “Essay Towards Solving a Problem in the Doctrine of Chances” (1763, published posthumously). That work became the basis of a statistical technique, now called Bayesian statistics. A key feature of Bayesian methods is the notion of using an empirically derived probability distribution for a population parameter. The Bayesian approach permits the use of objective data or subjective opinion [2] in specifying a prior distribution [3]. With the Bayesian approach, different individuals might specify different prior distributions. Classical statisticians argue that, for this reason, Bayesian methods suffer from a lack of objectivity. Bayesian proponents argue, correctly, that the classical methods of statistical inference have builtin subjectivity (through the choice of a sampling plan and the assumption of ‘randomness’ of distributions) and that an advantage of the Bayesian approach is that the subjectivity is made explicit [4]. However, a prior distribution cannot easily be argued to be strongly ‘subjective’. Bayesian methods have been used extensively in statistical decision theory. In this context, Bayes's theorem provides a mechanism for combining a prior probability distribution for the states of nature with new sample information, the combined data giving a revised probability distribution about the states of nature, which can then be used as a prior probability with a future new sample, and so on. The intent is that the earlier probabilities are then used to make ever better decisions. Thus, this is an iterative or learning process, and is a common basis for establishing computer programmes that learn from experience (see Feedback and crowding). 
Testing for rare conditionsVirtually every labconducted test involves sources of error. Test samples can be contaminated, or one sample can be confused with another. The report on a test you receive from your doctor just may belong to someone else, or be sloppily performed. When the supposed results are bad, such tests can produce fear. But let us assume the laboratory has done its work well, and the medic is not currently drunk and incapable. The problem of false positives is still a considerable difficulty. Virtually every medical test designed to detect a disease or medical condition has a builtin margin of error. The margin of error size varies from one test procedure to another, but it is often in the range of 15%, although sometimes it can be much greater than this. Error here means that the test will sometimes indicate the presence of the disease, even when there is no disease present. Suppose a lab is using a test for a rare condition, a test that has a 2% falsepositive rate. This means that the test will indicate the disease in 2% of people who do not have the condition. Among 1,000 tested for the disease and who do not have it; the test will suggest that about 20 persons do have it. If, as we are supposing, the disease is rare (say it occurs in 0.1% of the population, 1 in 1000), it follows that the majority (here, 95%, 19 in 20) of the people whom the tests report to have the disease will be misdiagnosed! Consider a concrete example [5]. Suppose that a woman (let us suppose her to be a white female, who has not recently had a blood transfusion and who does not take drugs and doesn’t have sex with intravenous drug users or bisexuals) goes to her doctor and requests an HIV test. Given her demographic profile, her risk of being HIVpositive is about 1 in 100,000. Even if the HIV test was so good that it had a falsepositive rate as low as 0.1% (and it is nothing like that good), this means that approximately 100 women among 100,000 similar women will test positive for HIV, even though only one of them is actually infected with HIV. When considering both the traumatising effects of such reports on people and the effects on future insurability, employability and the like, it becomes clear that the falsepositive problem is much more than just an interesting technical flaw. If your medic ever reports that you tested positive for some rare disorder, you should be extremely skeptical. There is a considerable likelihood the diagnosis itself is mistaken. Knowing this, intelligent physicians are very careful in their use of test results and in their subsequent discussion with patients. But not all doctors have the time or the ability to treat test results with the skepticism that they often deserve. How bad can it get?In general: The more rare a condition and the less precise the test (or judgement), then the more likely (frequent) the error. Consider the HIV test above. Many such tests are wrong 5%, or more, of the time. Remember that the real risk for our heterosexual white woman was around 1 in 100,000, but the test would indicate positive for 5000 of every 100,000 tested! Thus, if applied to a low risk group like white heterosexual females (who did not inject drugs, and did not have sex with a member of a highrisk group like bisexuals, or haemophiliacs, or drug injectors) then the HIV test would be incorrect 4999 times out of 5000! In general, if the risk were even less and the test method still had a 5% the error rate, the rate for false positives would be even greater. The false positive rate would also increase if the test accuracy were lower.

Related further reading  
Intelligence: misuse and abuse of statistics  drugs, smoking and addiction  
establishment psychobunk  cause, chance and Bayesian statistics  
For related
empiric reasoning documents, start with Why Aristotelian logic does not work 
Endnotes

email abelard at abelard.org © abelard, 2002, 13 october the address for this document is http://www.abelard.org/briefings/bayes.htm 2200 words 
latest  abstracts  briefings  information  headlines  resources  interesting  about abelard 