the Turing test and intelligence

site map

The document, decision processes by abelard, gives an empiric analysis of the Entscheidungsproblem..

Computing machinery and intelligence was published by Alan Turing in 1950.
The document, the Turing test and intelligence by abelard, gives further analysis.

INDEX

Introduction

Karl Lashley, after studying the unit of memory for thirty years, supposedly stated that he was not even sure whether the engram (unit of memory) even existed.

Some still imagine that it is as yet not possible to define intelligence. After much study, I do have some idea of the nature of intelligence, but ‘intelligence’ is a most complex issue (see also ‘intelligence’: misuse and abuse of statistics). In the process of my studies, I have picked up clues and various insights from a wide variety of sources, many of which I can not immediately remember. So, in advance, apologies to any unacknowledged originators upon whose shoulders I sit [1].

One of the better definitions I recall is that intelligence is fundamentally the ability to solve problems, particularly ‘unusual’ or ‘new’ problems. Another useful idea from the recesses of my memory, is, that one knows it as reality, when it leaps up and hits you in the face. A.M. Turing (1912 – 1954) in 1950 suggested a test (full text available on this site), since widely known as ‘the Turing test’ as a yardstick for determining whether a computer were to be regarded as ‘intelligent’[2].

We are then, testing the machine’s ‘intelligence’ via the ‘computer’s’ ability to fool our ‘intelligence’ within a particular context.

Put yet another way, if we are not intelligent enough to solve the riddle, then we will call the machine intelligent (like us?).

As a negative exposition, this could be thought as rather a ropey definition or it might be seen as similar to defining a mouse as intelligent if it failed to distinguish an elephant from another mouse without having sight or sound or smell of the pair. Conversely, maybe if we cannot tell the difference, we could consider that our intelligence was limited or lacking.

What we are testing is the ability of the ‘machine’ to fool individual humans. Perhaps we are then, labelling that ability to deceive as ‘intelligence’? Yet another possibility is that the human being, used as a comparison standard in the test, could perhaps persuade the tester that they were a machine.
 

Chess is not just a game

I shall use the example of chess widely in this paper, for chess is a very useful test of intelligence and it is very closely digitally defined (see Aristotelian logic does not work and the confusions of Gödel). Also, as a competitive ‘game’, it is possible to rate players quite closely in terms of relative ability and there is a sophisticated numbering system to do it.[7]

It is interesting that Turing used chess as a test-bed for working out early ideas on computers (as did Von Neuman: Von Neuman and Morgenster, 1944 [3]). Turing also worked with many of the leading British chess players when developing the code-breaking methods during the Second World War. It is also interesting that, despite his considerably greater mathematical insight and creativity, he was by comparison a rank amateur as a chess player. It is probable that chess players need different qualities, including a dash of deviousness.

The strongest chess machine so far developed is now able to push, and often defeat, even the greatest practising modern players.

It is useful to be clear that, just as an aeroplane far exceeds the birds as a flying machine, so also the present computer methods are very different to the human approach to problem solving. The computer functions by exhaustive analysis, while the human works by experience and associative methods.

The present chess computer programmes are essentially based upon principles laid out by Von Neumann[3], the only fundamental change has been the ever increasing speed of modern computer hardware with some minor software refinements.

With modern chess playing programmes, it is also possible to vary the style of the programs play, e.g. more aggressive, more strategic, more tricky etc, so the computer can effectively show some form of personality, the machine can also be programmed to learn from errors.

Unlike the human, the chess computer can be quickly and accurately cloned and it does not need decades to learn its trade, nor does it tire easily or suffer from headaches – at least as far as I know.

What of computer creativity in chess

In recent years, a computer found a great study and kept it all to itself until John Nunn talked things over with this (his) machine some months later. The importance of this happening was so great that I would expect it, a thousand years hence, to rank in the history of the twentieth century with the discovery of penicillin and the first step upon the moon.

Nunn/Database Study, 1995

1	a3!!
	1	a4? Nf3 2 a5 Nd4 3 a6 Nb5 is Mutual Zugzwang [8], e.g: 4 Kb7 (4 Nd5 Kc6 5 Nc3 Kb6!) 4...Nd6+ 5 Kb6 Nc8+=.
1	... Nf3
2	a4 Nd4
3	a5 Nb5
4	a6
	Mutual Zugzwang again, but this time with Black to play.

4	... Kd6
	4	... Kd8 5 Nd5 Kd7 6 Nc3+-
5	Kb7 Nc7
	5	... Kd7 6 Nd5 Nd6+ 7 Kb8 Nc8 8 Nb4 Kd8 9 Nc6+ Kd7 10 Ne5+ Kd8 11 Nc4 Kd7 12 Kb7 Kd8 13 Nb6 Nd6+ 14 Kc6+-
6	a7 Na8
7	Nd5!
	7	Kxa8?? Kc7=
7	... Kd7
8	Nb6+ Winning

   

Distinguishing

A frog can distinguish near and far: An object that is near and flying is ‘food’, so stick out tongue and eat. An object flying at distance is a likely nuisance, who may in its turn imagine you as lunch, so dive for cover.

You or I can distinguish between many individuals on the basis of looks, and often intelligence. There is much clear intellectual difference between Einstein and Darwin on the one hand and a Down’s syndrome individual, a child or a duck on the other. Intelligence is not merely ‘there’ or ‘not there’, but presents on a continuum in which we are potentially able to make distinctions.

 I have noted that when most people use the term intelligence, they are inclined to mean something like ‘What a fine person, they agree with me’ or, rather less often, ‘My, that is impressive’. Most tend to emphasise the first, for admitting the impressiveness of others is not widespread in human society apart from where people's own offspring are concerned.

As stated above, intelligence may be defined as an ability to solve ‘new’ problems. Thus, if one has experience of  IQ tests, the familiarity makes the tests become less valid.

If you meet a person who has gained some skill in a field of which you know little, that person can appear impressive and be thought impressive or intelligent. Alternatively, it may merely be that the person has received effective education in that skill. It is often difficult in this complex society to distinguish intelligence from learning.

Bernard of Chartres said, ‘We are like dwarfs on  the shoulders of giants, so that we can see more  than they, and things at a greater distance, not by virtue of any  sharpness of sight on our part, or any physical distinction, but  because we are carried high and raised up by their giant size’[1].

Thus also do we build our skills brick by brick. Whether we build a large and complex and impressive structure can depend much upon persistence, aside from any general vision. As has been said, ‘genius is 1% inspiration and 99% perspiration’[4].

Our computers also exert much high speed perspiration, while we are much less organised but rather good at forming associations. However, this skill we are also slowly learning to teach our inorganic friends. Currently, we have advantage in that our peripherals for gaining input from the outside world are comparatively more efficient. As Aristotle said, “There is nothing in the intellect that was not before in the senses” [5].

Intelligence, like hot or heavy, is a property of an object. It ‘is’ not an object of itself, it is something an object exhibits or does. It is an object in action (see also Stages of consciousness). As such, intelligence may be regarded as inherent in every object down to the level of stimulus-response, or even the ‘equal’ and opposite reaction of Newton’s descriptions.

So to the first ‘result’ of this analysis of the Turing test, yes, computers are ‘intelligent'

The problem then becomes ‘how intelligent?’ which expands outward into ‘how does this relate to human intelligence?’[6] How, then, does this ‘distinguishing apart’ of computer and human intelligence advance our understanding, assuming that it does?

As may be seen, what is regarded as intelligence is much bound up with the various levels of ‘knowledge’ of  various subjects, ourselves and of computers. As we provide our computers with ever better sensory peripherals and increasing databases, it is likely that increasingly we will think of them as intelligent.

Now we will return to the possibility of distinguishing human and computer intelligence, while remembering that computers solve problems generally by different methods than humans.

We humans most often perceive intelligence in the sense of stimulus and response from the outside, where we say, “Hey! That person is real clever”. We are not always distinguishing between sheer creativity and ‘mere’ learning. It is fairly clear that Turing did not make such fine distinctions in his test and Turing’s test is, after all, the context for this monologue. In similar manner, it seems strange that humans should so often concern themselves with the manner in which computers produce their impressive results, rather than the striking fact that they do.

Turing being a rather superior sort of fellow, I will assume that he was rather more confident and objective than average Jo and, therefore, that he meant intelligent in the ‘my, that is impressive’ mood, rather than the previously noted more common form, “they must be bright because they agree with me”. Turing also chose the human as his yardstick for intelligence, despite the considerable intelligence often shown by other animals. Turing is also clearly assuming a human intelligent enough to make the necessary distinctions.

All humans vary in their capacity and ability and knowledge. As a considerable sophisticate on these matters, I would expect it to be far harder to fool me in such a test than the average Jo, although there is that rather arbitrary five-minute rule proposed by Turing in the original test. I do rather like to take my time to consider problems, which could put me at some disadvantage as humanity’s representative. I am also rather prejudiced in that I rather like the idea of some intelligent computers as company. After all, they may well be rather more rational and friendly than most humans I meet.

Thus we gradually refine Turing's meaning to that of whether a reasonably
bright human can tell the human apart from the computer.

Duplicity

Off now on another side track. Could I distinguish a computer from a particularly dull human that communicated in grunts, especially if the computer were primed to respond with similar crudity. In such circumstances, I might doubt my own ability to discriminate.

Which human, then, is to carry out the test? Which is to be the human to represent humanity in this rather strange task of ‘proving’ that computers are not intelligent, or at least not able to fool us into thinking that they are human?

Chess computers have other interesting personality traits, they do not go round bleating ‘Did you win - Did you win - Did you win?’, an incantation widely heard at chess congresses. Pointless to explain that one learns much more from a loss than a win. Chess is a good way to learn, to keep the brain fit and the ego in check, a mental form of your local gymnasium. Those who see chess merely as a means of self-proof make the game experience uncomfortable and drive many of the better, more sensitive brains to analysis, correspondence, problems, studies and the like.

A computer is another matter, a computer is friendly, tireless, patient and is steadily becoming able to produce more interesting games than the games provided by weak, fallible and emotional humans. With a computer, you may repeatedly try different approaches in interesting positions or access data-bases of hundreds of thousands of games by able players. Further, computers can learn by various means to improve their game. I watch with mischievous joy as the computers steadily overhaul human arrogance and ego. Time perhaps we leave winning to our new friends, the machines, and remember that we are merely human.

What if the human comparator in the test were to deliberately simulate computer like responses? Well again, some may get very good at such a task; again, this may make errors quite likely. Humans are indeed very able dissemblers. In the test, then, we have another hidden assumption: an honest test subject and an honest tester.
 

The Turing test is then a rather sloppy definition of ‘intelligence’. From the foregoing, it is certainly not to be taken as a very clear definition of intelligence. Better to regard the test as a particular benchmark for growing intelligence in computers. That is, it is just one particular stage in the defining of the nature of intelligence. As can be seen however, in some degree, this test may have the potential for contamination by elements of  duplicity.

Perhaps we should also consider whether the human is really trying (though unfortunately there is no clear or fool proof manner in which we could test for that). This set of considerations rather undermines the usefulness of the Turing test. If our computers become sufficiently fluent, perhaps they may learn also to dissemble in order to claim the great prize of our approval of computer intelligence.

Maybe the ability to fool is a more interesting question than the original, rather sloppy, attempt to define ‘intelligence’. Turing, clearly one of the great thinkers and contributors to the advance of knowledge this century, achieved one of the more important functions of the creative scientist, he asked interesting questions. Whether he gave a useful test of machine intelligence is another matter.

A reverse Turing test!

Consider for the moment a reverse question: can a machine tell us (any of us) apart from another machine? If a machine cannot make such a distinction, are we still to call the machine intelligent? Or if it cannot, should we move to withhold  the accolade?

Perhaps this gives us a caution that we may have a prior difficulty and need a method for deciding just what we mean by being a ‘human being’ or being a ‘machine’. Perhaps we must consider whether there is any useful distinction worth attempting, rather than just considering the nature of being more ‘intelligent’ by this rather haphazard and sloppy means.

The real world includes the continuities of space and time. It is merely a human convenience and limitation that we seek to reduce so much to on/off ‘logic’.

Intelligence is a graduated measure because it is an inherent function of the real world. It is not either on or off. Intelligence has to function with analogue ‘problems’, humans are spacial objects dealing with other objects in space. Chess is a digital construction of our limited human minds, as are our attempts at intelligent test construction, as even is the digital manner in which we presently understand and describe our neuronal minds.

When we reduce ‘intelligence’ still further to some single rather simplistic ‘test’, we do mayhem to the reality we seek to understand, however ‘interesting’ or usefully provocative the exercise may prove.

Clearly a chess computer playing over a link would already defeat maybe all human attempts to distinguish it from a human player. In general, after all, it would already play more effective chess than the vast majority of, or even all, human players.

Kasparov claims to be able to distinguish computer play from human play. During a simultaneous event over the Internet, he once stopped playing a certain game, saying that he was up against a computer (it was supposed to be human opposition). He was losing at the time! Kasparov has also claimed, while losing against a computer, that the computer was being given human assistance!!

A number of chess players would fancy their chance of distinguishing humans from computers by virtue of the style. Mere strength of play alone therefore hardly proves they cannot so distinguish. I wish someone would test this!

Similar claims have been made with respect to jazz and computers. Also some knowledge-based computer systems are far more effective than their human competition, e.g. those for plant diseases and the identification of chemicals. Here, you can read about some of the things that computers are not so good at (yet).

I know very little of the workings of mechanical objects such as motor vehicles. It is highly likely that a discussion with a well organised knowledge-based computer programme would quickly leave me baffled with science, leaving me without any useful questions to ask the car computer genius. How then could I decide whether the computer was ‘intelligent’ or no? Could I perhaps decide that it were a computer on the basis that it knew far more than myself, or even that it was more consistent than the average disorganised human? I would then be deciding that it was a machine because it was superior in its behaviour. How then am I to call the computer less intelligent than the human because it is more effective?

While discussing the intelligence of machines, Turing, in his paper, even suggests injecting ‘irrationality’ into the functioning of a computer aimed at emulating human intelligence. Likely he had another good point.

Next, consider a very likely scenario not too long hence, when a computer, or even the web, gives access to so great and well structured a knowledge base that the machines know more than any individual. This is not such a difficult assumption, after all our great libraries are already such oracles. The only difference with our growing computer technology is that interactivity and retrieval is steadily growing more efficient than is the case with paper-based libraries.

I think it rather clear then that the Turing test offers a useful basis for clarifying our relationships with and our understanding of the wider world. It also gives us ideas upon the means by which we may make our silicon friends more like us where that is useful, but it hardly gives us a clear manner of distinguishing computer from human.

It has been said that much human progress is just a matter of adding another decimal point of accuracy to our knowledge. While such a view is simplistic, that is a function well-suited to computer-based information.

By the way, remembering from where I started: as for memory, human memory is primarily associative, but that is another essay!

References

von Neumann:
The mathematical cornerstone of von Neumann's theory of games is the “minimax theorem”, which he stated in 1928. Its elaboration and applications are in the book he wrote jointly with Oskar Morgenstern in 1944, Theory of Games and Economic Behavior.

The minimax theorem says that for a large class of two-person games, there is no point in playing. Either player may consider, for each possible strategy of play, the maximum loss that he can expect to sustain with that strategy and then choose as his “optimal” strategy the one that minimizes the maximum loss. If a player follows this reasoning, then he can be statistically sure of not losing more than that value called the minimax value. Since (this is the assertion of the theorem) that minimax value is the negative of the one, similarly defined, that his opponent can guarantee for himself, the long-run outcome is completely determined by the rules.

In computer theory, von Neumann did much of the pioneering work in logical design, in the problem of obtaining reliable answers from a machine with unreliable components, in the function of “memory”, in machine imitation of “randomness” and in the problem of constructing automata that can reproduce their own kind (Extract from Enclyclopaedia Britannica, 1997).

Azimov, Isaac (1920 – 1992) did much thinking upon such issues. His long and serious exploration of these ideas are available in his tales of robotics. Robotics is a word coined by Karel Capek, a Czech writer, who introduced the word in a play titled R.U.R. (Rossum's Universal Robots) in 1920.
Asimov developed a set of ethics for robots and intelligent machines, that greatly influenced other writers’ treatment of the subject. He called his rules the three laws of robotics, which are as follows:

1. a robot may not injure a human being, or, through inaction, allow a human being to come to harm.
2. a robot must obey the orders given it by human beings, except where such orders would conflict with the First Law.
3. a robot must protect its own existence, except where such protection would conflict with the First or Second Law.

He went on to develop his ideas in a series of works on the subject.
The following are the relevant titles: I, Robot (1950), The Robots of Dawn (1983), The Rest of the Robots, The Complete Robot, The Caves of Steel, The Naked Sun, and Robots and Empire.

For an Asimov story concerning education, see Profession at this site.
For other science fiction recommendations see recommended reading also at this site.

The preponderance of books on chess concentrate mainly on the cramming great masses of fact and experience: a constant stream of opening variations, endings, combinations, principles, tricks of the masters and so on. All of this is merely memory and calculations, functions much better performed by a computer than a creative human.

The following are some rather special and fundamental books on the subject of chess theory:

Thought and Choice in Chess
Adriann de Groot, Mouton De Gruyter,1978, 2^nd ed., ISBN: 9027979146

Chess Praxis
Aron Nimzovich, Dover Publications, 1936, ISBN: 0486202968

Simple Chess
Stean, Faber & Faber, 1987, ISBN: 0571112579

Secrets of Spectacular Chess
(a commercialised title for what would be better called Aesthetics and Chess)
Levitt and Friedgood, B.T. Batsford, 1995, ISBN: 0713477210
Levitt has a web-site, mainly concerned with his chess work:
http://www.jlevitt.dircon.co.uk

A class site on the life and work of Turing is maintained by the writer Andrew Hodges, a specialist on Turing.

email email_abelard [at] abelard.org

© abelard, 1998


all rights reserved

the address for this document is https://www.abelard.org/turing/tur-hi.php

3770 words
prints as 7 A4 pages (on my printer and set-up)