FROM THE IMITATION GAME TO THE TURING TEST

The Turing Test was introduced by Alan M. Turing (1912-1954) as "the imitation game" in his 1950 article (now available online) Computing Machinery and Intelligence (Mind, Vol. 59, No. 236, pp. 433-460) which he so boldly began by the following sentence:

I propose to consider the question "Can machines think?" This should begin with definitions of the meaning of the terms "machine" and "think."

Turing Test is meant to determine if a computer program has intelligence. Quoting Turing, the original imitation game can be described as follows:

The new form of the problem can be described in terms of a game which we call the "imitation game." It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart from the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either "X is A and Y is B" or "X is B and Y is A." The interrogator is allowed to put questions to A and B.

When talking about the Turing Test today what is generally understood is the following: The interrogator is connected to one person and one machine via a terminal, therefore can't see her counterparts. Her task is to find out which of the two candidates is the machine, and which is the human only by asking them questions. If the machine can "fool" the interrogator, it is intelligent.

This test has been subject to different kinds of criticism and has been at the heart of many discussions in AI, philosophy and cognitive science for the past 50 years.

TT-RELATED REFERENCES (ONLINE)

David Chalmers has prepared a comprehensive bibliography of the philosophy of Artificial Intelligence. Here is the section on the Turing test.

Objections to the Turing test: Is the Brain a Digital Computer? Searle's Chinese Room

Robert French's paper: Subcognition and the Limits of the Turing Test pdf (Mind 99(393) 53-65,1990; reprinted in P. Millican & A. Clark (eds.). Machines and Thought: The Legacy of Alan Turing Oxford, UK: Clarendon Press (1996).) One of the most important arguments against the Turing Test is that it only provides a test for human intelligence.

Stevan Harnad's paper: The Turing Test Is Not A Trick: Turing Indistinguishability Is A Scientific Criterion. (SIGART Bulletin 3(4),9-10, October 1992)

Another paper by Stevan Harnad: Other bodies, Other minds: A machine incarnation of an old philosophical problem. Minds and Machines, Vol. 1, pp. 43-54.

One more selected reading from Harnad Minds, Machines and Turing: The Indistinguishability of Indistinguishables Journal of Logic, Language, and Information 9(4):425-445.

Paper by Larry Hauser: Reaping the Whirlwind: Reply to Harnad's `Other Bodies, Other Minds' Minds and Machines, Vol. 3, No. 2 (May, 1993), pp. 219-238.

Technical Report by Phillip G. Bradford and Michael Wollowski The Formalization of the Turing Test, available in postscript and pdf. 1994.

NEW! By Bruce Edmonds:The Constructability of Artificial Intelligence (as defined by the Turing Test), Journal of Logic Language and Information, 9:419-424.

NEW! By Judith Donath: Being Real, html format. To appear in Goldberg, K. (ed.) The Robot in the Garden: Telerobotics and Telepistemology in the Age of the Internet, MIT Press

Paper by Jason Eisner Cognitive Science and the Search for Intelligence , pdf format. Paper presented to the Socratic Society, University of Cape Town, South Africa, 1991.

Blay Whitby's paper: Why The Turing Test is AI's Biggest Blind Alley Whitby argues that AI should not be too distracted into direct copying of human performance and methods.

Summary of a talk by Pat Hayes: Abandoning the Turing Test (postscript)

A paper by Jean Lessegue: What kind of Turing Test did Turing have in Mind? (Tekhnema, 3, 37-58, 1996)

An essay by David Barberi: The Ultimate Turing Test

WIRED Article by Charles Platt: What's It Mean to be Human, Anyway?

WIRED Article by Richard S. Wallace: The Lying Game

An essay by Warren Sack: Replaying Turing's Imitation Game Paper presented at the panel "Nets and Internets" at Console-ing Passions: Television, Video and Feminism, April 25-28, 1

Stephen J. Cowley and Karl F. MacDorman's paper, Simulating Conversations: The Communion Game (AI&Society, 9, 116-137, 1995)

CogPrints Electronic Archive Many relevant papers on psychology, neuroscience, linguistics and computer science can be found here.

Psycholoquy papers on the Turing Test

You can also take a look at the Simon Laven Page for chatterbot related books and papers (as well as an excellent chatterbot resource, see below)

BACKGROUND READING

The Turing Test and Chinese Room Experiment A nice, concise page by Larry Hauser describing the Turing Test and the Chinese Room of Searle.

Minds, Machines and Searle Stevan Harnad (Minds, Machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence, 1989, 1: 5-25.

Artificial Intelligence: A Philosophical Introduction by Jack Copeland. Oxford: Blackwell, 1993.

Minds, Machines and Godel J.R. Lucas (Philosophy, XXXVI, 1961, pp.112-127)

Minds, Brains and Science John Searle (Harvard University Press, Cambridge, Mass. ,1984)

Thinking Computers and Virtual Persons Eric Dietrich (ed.)

Android Epistemology Ken Ford, Clark Glymour, and Patrick J. Hayes (eds.) Menlo Park, CA: AAAI/The MIT Press.

Alan Turing: The Enigma Andrew Hodges (ISBN 0-09-911641-3 Vintage, Random Century, London.)

Paper by Larry Hauser: Why Isn't My Pocket Calculator a Thinking Thing Minds and Machines, Vol. 3, No. 1 (February, 1993), pp. 3-10.

* top *

LOEBNER PRIZE

In 1991 Dr. Hugh Loebner started the annual Loebner Prize competition. A $100,000 prize is offered to the author of the first computer program to pass an unrestricted Turing test.

The Loebner Prize Homepage

Loebner Prize 2003

Jason Hutchens' MegaHAL page contains some fun transcripts and good information. Hutchens' programs participated in the Loebner contests in 1996, 97 and 98 and one of them won the prize in 1996.

Robby Garner, Robitron has also participated in the contest several times and is the winner of the 1998 and 1999 Loebner prizes. His site contains a lot of cool information and links.

Michael Mauldin's paper: Chatterbots, Tinymuds, And The Turing Test: Entering The Loebner Prize Competition This paper describes the development of a program and its performance on the first three Loebner Prize competitions.

Stuart Shieber's paper: Lessons from a Restricted Turing Test CACM, volume 37, number 6, 1994.

Hugh Loebner's response to Stuart Shieber: In Response

A human confederate's perspective: Tracy Quan tells her story.

Guardian article: Oct 6, 2001 - It's the thought that counts

NY Times article: March 18, 1999 - Look what's talking

* top *

TALK TO THEM

Note: Commercial bots (e.g. help services for special companies, customer service applications) are not listed here, although they can be reached from some of the links.
I realize that I may not have been so good at keeping track of all the chatbots out there. Here are a few links to some sites that may be more frequently updated:

Turing Hub

Alice AI foundation

BotSpot

And you MUST see The Simon Laven Page probably the most comprehensive chatterbot site on the web...

-----

Jabberwock, the winner of Loebner 2003

Jabberwacky, a bot with character

The CHAT Natural Language System (Loebner 94, 95)

ELIZA, the oldest girl around.

JULIA was a participant in Loebner 94.

ALICE is another chatty girl. Beware, this one spreads gossip! Winner of 2000 and 2001 Loebner prizes.

SPENCE'S, a virtual bar... Here, you can talk to Spence or to Erin, his niece who is also the bartender.

MITBOLEL A customizable Java chatbot. You can even put Mitbolel on your own web page!

THOUGHT TREASURE The approach here is a bit different. Thought Treasure has some "world knowledge" (an extensive concept hierarchy). This is an evolving system, latest version available for download.

MeBOT You need a Java compatible browser to talk to MeBot.

BRIAN, an 18 year old college student (3rd place in Loebner 1998)

MIMIC Help him learn talking, he's willing to learn.

Mr. MIND Try to convince him that you are not a computer.

DR ABUSE An ELIZA-based program that talks in Spanish

ELOISA You want to chat in Italian? Visit her.

* top *

PEOPLE
 
Ned Block Online papers. Also check out the homepage of the course Minds and Machines prepared by Block.

Selmer Bringsjord This site contains information on his books and some online papers and presentations.

Rodney Brooks Here you can find out about Brooks' research on building intelligent robots.

David Chalmers A comprehensive site that contains many useful resources on the philosophy of mind as well as information on Chalmers' works on consciousness

Jack Copeland His personal page contains online papers as well as links to his Turing-related projects and conferences

Daniel Dennett Here, you can find a lot of papers by Dennett in html format.

Robert French Contains many papers of the author, downloadable in pdf format.

Stevan Harnad E-Prints on Cognitive Psychology. Many relevant papers online.

Larry Hauser Contains numerous online papers by Hauser. Also an excellent philosophy of mind resource.

Patrick Hayes Information on his research, although not too many papers online.

John R. Lucas Papers on various topics, online.

John McCarthy Personal homepage with various links to online publications.

Donald Michie Personal homepage containing full list of publications, but none are online.

Marvin Minsky Personal homepage with some online publications.

James Moor Personal homepage. No online publications unfortunately. Information on the Loebner Prize 2000 and The Future of the Turing Test Conference

Aaron Sloman His personal page with some online resources.

* top *

OTHER INTERESTING PLACES
 
NEW! AlanTuring.net

NEW! KurzweilAI.net

NEW! The Turing Test Science Contest !

NEW!Turing Tournament at the California Institute of Technology.

Preliminary Turing Test Analysis Interesting paper drawing a parallel between the Turing Test and GIS/spatial technologies research. Not really relevant to our topic, though.

The Automatic Confession Machine, or The Catholic Turing Test

How my program passed the Turing Test by Mark Humphrys.

MAW'97 Turing Test page This page was prepared within the context of Math Awareness Week '97.

Lecture slides This and the following few slides give a nice, concise summary of the Turing Test.

Characters, improvisation and ... Lots and lots of links.

Botspot Go there for all bot-related stuff on the net.

Banai's Turing Test Page

Machine Intelligence Part1: The Turing Test and Loebner's Prize

Psycholoquy A refereed international, interdisciplinary electronic journal.