How "meaningful" is Turing test?

Because even though it's sometimes presented as a "scientific" way of classifying the goodness of AI, I do view it as a sort of naive, left-handed way for defining or measuring computer intelligence.

Because there are many more aspects to computer intelligence than mere "whether it can trick some people into believing that they talk with a human". Sure if the computer can answer any question similarly to how a humans expert would, then that would be pretty "hard", but computer intelligence is not there yet and it's unsure, whether it'll ever be due to the long-known differences between formal and informal semantics. It could be just a problem of inhuman system design capability. That is, no-one can ever in practice develop a computer program that's that good. It would just be too large.

Nowadays intelligence of computer programs could be measured by e.g. quantities of statements that they can evalute. That is, just the number of possibilities that they're able to process. Because even the goal of the Turing test would be a computer program with large enough database and large enough semantic analyzer and reasoner.

So, are there better measures?

  • $\begingroup$ "Meaningful" sounds like a matter of opinion, and pretty vague as well. (Any community votes?) Opinion-oriented questions typically aren't a good fit here; see our help center. Do you have a specific technical question that can be objectively answered? Can you define "meaningful" in a precise way? Note that we are a question-and-answer site, not a discussion forum, so invitations to open-ended discussion aren't appropriate here. $\endgroup$
    – D.W.
    May 8 '18 at 21:50
  • $\begingroup$ @D.W. It's not opinionated, not more than Turing claiming the Turing test as a "defining" measure. What I'm asking for is, is there perhaps a more meaningful measure for computer intelligence. $\endgroup$
    – mavavilj
    May 8 '18 at 21:58
  • 1
    $\begingroup$ “Nowadays intelligence of computer programs could be measured by e.g. quantities of statements that they can evalute.” Define intelligence! (I know, I'm asking a lot.) It seems to me that you're trying to measure performance, not intelligence. $\endgroup$ May 8 '18 at 22:30
  • 3
    $\begingroup$ @mavavilj You can read the paper introducing the "Turing test" (i.e. the "imitation game") here. I doubt Turing would have claimed it was a "defining measure" (of what?). As jmite has said, it was more of a thought experiment to better frame a discussion of the capabilities of machines. You also have to take into account the context of the times where even most "educated" people would have viewed things like painting a picture, producing a sonnet, or playing chess as utterly beyond "machines" at all. $\endgroup$ May 8 '18 at 22:43
  • 2
    $\begingroup$ Does the Finnish Wikipedia give a citation where Turing claims that it "measures the 'humanity' of AI"? I gave you a link to the primary source. The question that Turing was reframing was "can machines think", not "are machines 'human'". It's also very clear that the purpose of the reframing was to have something more verifiable and to move the question away from ill-defined concerns like whether machines have souls/consciousness/emotions/creativity. You can believe a machine won the imitation game while still doubting it has any of those aspects. I again recommend reading the primary source. $\endgroup$ May 9 '18 at 0:00

Nowadays intelligence of computer programs could be measured by e.g. quantities of statements that they can evaluate.

So, the trick, is that the Turing Test was meant for an evaluation of strong AI: could there be a computer program that was as intelligent as a human? So evaluating statements isn't enough, since humans can do much more than that.

There are some things to keep in mind. First, while some people use the Turing Test as a metric of the quality of chatbots today, that wasn't necessarily its original intent. It was more of a thought experiment, trying to answer the question of "what does it mean for an AI to be intelligent?"

Secondly, the test wasn't mean to strictly evaluate the "question and answer" ability of an AI. It was more saying, "if there's an AI so good we can't distinguish it from a human, is there any meaningful difference?" It was a way to define sentience without bringing in complex philosophical (or possibly even religious) arguments about what is and isn't alive. The Turing Test says "if it looks sentient, we'll say it is, because there's no meaningful way to tell otherwise".

That is, no-one can ever in practice develop a computer program that's that good. It would just be too large.

There are many possible ways around this. One is that we might not write every aspect of a program, but evolve one. We could write a program that writes a program that writes a program etc, each one being slightly more intelligent until it converges on something powerful.

Or, an AI could be developed from machine learning. With enough time and data, something close to human intelligence could grow from statistical observations, having the headstart of hand-coded AI at its core.

If a human brain is enough for human intelligence, a comparably complex program should be as well. But even if no such program is ever developed, the Turing Test still holds meaning as a thought experiment.

  • $\begingroup$ (1st paragraph) But what else does such program do than evaluate statements? $\endgroup$
    – mavavilj
    May 8 '18 at 22:28
  • $\begingroup$ @mavavilj Carry out a conversation, write poetry, give complements, complain about the weather, etc. Anything a normal human does! $\endgroup$
    – jmite
    May 8 '18 at 23:50
  • $\begingroup$ Well, but those are yet more statements. Just add some stochastic "add some random social stuff". I'm pretty sure that that can be modeled as well. Human social behavior can be anything, so the program could fake by randomizing it. $\endgroup$
    – mavavilj
    May 8 '18 at 23:54
  • 1
    $\begingroup$ @mavavilj Right, but an off-topic or odd-sounding statement in a conversation is a huge clue that you're talking to an AI. And there's more to conversation than evaluating statements as true and false, since it requires synthesis of, in addition to evaluation of, text. $\endgroup$
    – jmite
    May 9 '18 at 0:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.