Did 'Eugene Goostman' really pass the Turing test?

Question

It is being said that 'Eugene Goostman', a computer programme developed to simulate a 13-year-old boy, managed to convince 33 per cent of the judges that it was human, and thus passed the Turing Test.

The computer programme, aka a chatbot, was pretending to be a 13-year-old Ukranian boy for whom English was a second language – something very different indeed.

For me, Eugene sounds exactly like a mediocre chatbot should: repetitive, nonsensical, and littered with non-sequiturs. I don't know how it convinced the judges (which seem unprofessional).

Many people have criticized Eugene like Prof Stevan Harnad who said "It's nonsense" and "We have not passed the Turing test. We are not even close."

Opinions differ, but I would really like to know if it officially passed the test?

It is also being said that:

Two one-time-only prizes that have never been awarded. 25K is offered for the first chatterbot that judges cannot distinguish from a real human and which can convince judges that the human is the computer program. $100,000 is the reward for the first chatterbot that judges cannot distinguish from a real human in a Turing test that includes deciphering and understanding text, visual, and auditory input. Once this is achieved, the annual competition will end.

Does it mean Eugene won $25,000?

Answer 1

There is no "official Turing test" so there's no concept of "officially pass[ing] the test". Turing described a methodology that one might use to evaluate artificial intelligences. The organizers of the event that Eugene Goostman won implemented that methodology in a particular way and the program satisfied the criteria the organizers had chosen. In that sense, it passed the test.

Since there is no "official Turing test", it might be more appropriate to say that Eugene Goostman passed a Turing test or even passed a Turing-style test. It's unlikely that the media would pick up on such subtleties, especially given Turing's fame and the idea of "the Turing test" in the public conscience.

Answer 2

I think the prizes you're referring to are the Loebner Prize. According to the Wikipedia page (see prior link), the winner for 2014 is 'Rose' by Bruce Wilcox. That program did not win one of the one-time-only prizes, but did get $4,000 in prize money. 'Eugene Goostman' competed in 2005 and 2008, finishing second both times.

The competition 'Eugene Goostman' won was organized by Kevin Warwick of Coventry University to mark 60 years since Alan Turing's passing. I'm unaware of any prize money.

Check out Ray Kurzweil's take on both the competition and the program in question. Excerpt:

Professor Warwick claims that the test was “unrestricted.” However, having the chatbot claim to be a 13-year-old child, and one for whom English is not a first language, is effectively a restriction. Moreover, the interactions were reportedly limited to five minutes each. There is a high likelihood of fooling naïve judges in such a brief period of time.

I chatted with the chatbot Eugene Goostman, and was not impressed. Eugene does not keep track of the conversation, repeats himself word for word, and often responds with typical chatbot non sequiturs.

I also found this to be a nice commentary: WIRED--That Computer Actually Got an F on the Turing Test

Answer 3

Adding a Wikipedia link for more depth on the "original" Turing Test.There are many tests called "Turing test". Wikipedia mentions "at least 3 primary versions". The test passed by Eugeene is none of those 3. The test passed by Eugeene was not devised by Alan Turing, but it was a test called Turing test, inspired by Alan Turing, and Eugeene did pass it.

The part mentioning at least 3 primary versions of the Turing test:

Saul Traiger argues that there are at least three primary versions of the Turing test, two of which are offered in "Computing Machinery and Intelligence" and one that he describes as the "Standard Interpretation." While there is some debate regarding whether the "Standard Interpretation" is that described by Turing or, instead, based on a misreading of his paper, these three versions are not regarded as equivalent, and their strengths and weaknesses are distinct.

Version 1

Turing's original game described a simple party game involving three players. Player A is a man, player B is a woman and player C (who plays the role of the interrogator) is of either sex. In the Imitation Game, player C is unable to see either player A or player B, and can communicate with them only through written notes. By asking questions of player A and player B, player C tries to determine which of the two is the man and which is the woman. Player A's role is to trick the interrogator into making the wrong decision, while player B attempts to assist the interrogator in making the right one.

Version 2

The second version appeared later in Turing's 1950 paper. Similar to the Original Imitation Game Test, the role of player A is performed by a computer. However, the role of player B is performed by a man rather than a woman. [...] In this version, both player A (the computer) and player B are trying to trick the interrogator into making an incorrect decision.

Version 3

Common understanding has it that the purpose of the Turing Test is not specifically to determine whether a computer is able to fool an interrogator into believing that it is a human, but rather whether a computer could imitate a human. While there is some dispute whether this interpretation was intended by Turing – Sterrett believes that it was and thus conflates the second version with this one, while others, such as Traiger, do not – this has nevertheless led to what can be viewed as the "standard interpretation." In this version, player A is a computer and player B a person of either sex. The role of the interrogator is not to determine which is male and which is female, but which is a computer and which is a human. The fundamental issue with the standard interpretation is that the interrogator cannot differentiate which responder is human, and which is machine. There are issues about duration, but the standard interpretation generally considers this limitation as something that should be reasonable.

In contrast, the test passed by Eugeene had human judges chatting with a chatbot for 5 minutes, after which they had to decide if it was a bot or not.

Answer 4

Yes, it won "a" Turing Test which followed Alan Turing's original description as closely as possible.
This was not however the $25000 prize money version that is the Loebner Prize, whose specifications differ from Alan Turing's description.
More importantly, Alan Turing never meant this to be a genuine test in the first place, so the outcome is not proof of intelligence, if that's what you actually wanted to know.