The answers Ray got back are borderline embarrassing. They're designed to deflect the question by ignoring the request and asking another unrelated question or deflecting by humor. These are common ploys in faking AI chat. Hell, what 13 yo calls himself a "little boy?" I was in 8th grade at 13 and looking forward to high school. I can't imagine calling myself a "little boy" at an age when some of my peers were sexually active.
I think this proves that the Turing Test is more or less crap. Humans, who are easily fooled/socially engineered, can't just decide that "this is AI," like its some kind of American Idol-like contest. There should be some rational metric at work here. A bunch of different tests and human judgement as only one part of the testing suite.
Look at what IBM has been doing with Watson. It may never pass this test, but its probably the closest we have to AI (generalist self-learning system). Maybe this event will be the excuse we finally need to lay Turing's test to bed, permanently.
Indeed embarrassing. Actually, I can't believe that they called this chat-bot in any way intelligent. On top of that, this level of chat is the same since 90s, how come nothing has changed? And anyway, text chatting is/will never be a measure of intelligence, the Turing test in this form should never be considered.
You really need to live 13 years as a boy to be able to answer as one.
ELIZA comes from the top of my head, and did impress me at that time ( '94 ? ). Lately I found out it had a psychologist side in it, as per original design and the answering with a question.
Better? Maybe, maybe not. But after 20 years, to show no real progress at all? Baffling.
Not much has changed because the people, companies, and institutions capable of making such improvements are now focusing on things with actual real world application.
I wouldn't go so far as to say the test is crap, but I agree it is given more importance than it deserves. It makes sense that if humans are intelligent and a machine behaves in the same manner, we would have to conclude that the machine is also intelligent. That cleverly punts over the entire discussion of the qualities and mechanism of intelligence. Another thing to keep in mind is although the Turing test could prove an entity is intelligent, it cannot prove that a thing is not intelligent. The most intelligent non-human species on our planet won't pass, but it doesn't mean they are stupid, it just means they aren't human.
"""I think this proves that the Turing Test is more or less crap. Humans, who are easily fooled/socially engineered, can't just decide that "this is AI," like its some kind of American Idol-like contest. There should be some rational metric at work here. A bunch of different tests and human judgement as only one part of the testing suite."""
I think the easiest fix is to crowd source the Turing test. If you can fool 95% of all people chatting with you over a sample of >100k people you're probably pretty good.
It's not crap, it's just a continuum. There are already chatbots that 'pass' sufficiently well to keep lonely people sending them salacious text messages. And presumably when the day comes that the robots are marching on Washington DC to argue for their rights, pretty much everyone will be convinced. For every step in between there will be 'AI's that fool more of the people, more of the time.
Hold on, you may say, we're talking about chatbots progressively fooling people, not 'real' artificial intelligence. But with better chatting comes better functional intelligence. For my bank's chatbot to converse sufficiently well to provide useful customer service requires it to understand all sorts of language and questions. For a personal avatar to take all your messages or make you fall in love with it (like that recent movie) requires even more capability. Whether it can ever transition into fully sapient consciousness verges on philosophy or, at any rate, is a question we can't answer yet.
The only aspect where this chatbot seems better than ELIZA is that it can query the internet for state capitals. That's 50 years of a complete lack of advancement.
Clearly if the criteria for passing the Turing test rests on this 30% of judges "fooled" threshold, we need to add a screen for the judges themselves...
.. and following that line I note that Professor Kevin Warwick appears to have history with both Llewellyn (promoting Warwick's book in 2002 here, http://www.reddwarf.co.uk/news/2002/11/08/roberts-robots/) and Smith .. which might make this a case of him having gathered a few cronies together to generate some publicity?
Is there a record of the event somewhere that could disavow me of the result simply being a case of partial judges being generous towards a friend?
Different tests are a good idea, but the devil is in designing them.
They'd have to both tell us something useful about the possibly-intelligent agent in question and not disallow anyone who we would consider an intelligent agent (more specifically, there's a whole suite of tests we could hypothetically use that would need to be discarded because they'd rule out entire sets of humanity as non-intelligence. Oops :-p)
I think this proves that the Turing Test is more or less crap. Humans, who are easily fooled/socially engineered, can't just decide that "this is AI," like its some kind of American Idol-like contest. There should be some rational metric at work here. A bunch of different tests and human judgement as only one part of the testing suite.
Look at what IBM has been doing with Watson. It may never pass this test, but its probably the closest we have to AI (generalist self-learning system). Maybe this event will be the excuse we finally need to lay Turing's test to bed, permanently.