How Russian-Ukranian Researchers Eugene Goostman chatbot beat Natural Language Turing Test

“Having a computer that can trick a human into thinking that someone, or even something, is a person we trust is a wake-up call to cybercrime”

Kevin Warwick of the University of Reading commenting on the Russian chatbot Eugene Goostman passing the Turing Test for the First time

History has been made for the first time folks. This as a computer has finally passed famed Mathematician’s Alan Turing’s Test for Artificial Intelligence as stated in the article “A Computer Has Reportedly Passed Turing Test For The First Time”, published June 9, 2014 by Stephen Luntz, I Fucking Love Science.

At this point I won’t go into a history lesson as to who Alan Turing was, as that’s easy to Google and he’s quite famous, really. Turing’s Test, however, is his most important contribution to the field of Cybernetics, that is, Computer Systems designed to replace human beings in the Working World. Even more impressive is that this Test was developed at a time when most computers, such as the MARK II and ENIAC were nothing more than gigantic adding and calculation machines.

Defining human thinking is a difficult thing. So he sidestepped that issues and made it simple; fool a human in a double Blind Test that they were taking to another human by mixing up the human interrogators with the computer that was trying to pretend to be human but not revealing their identities to the interrogator. During all this time, the scientists observing the conversations would keep track of who is talking to whom.

After the interrogation, they’d be quizzed. If the interrogator was convinced he was talking to a human, yet at the time the persons was a machine, then in all purposes the machine had passed the Test and could be said to be Artificially Intelligent. To use the specific criteria of Alan Turing’s Test, he predicted that by the year 2000 the average interrogator would be sufficiently skilled to guess correctly only 70% of the time the correct identity of the persons they were interrogating during a quick five minute conversation. That meant 30% of the time, they'd get it wrong and assume that the person they were talking to was human, when in fact it was a machine.

These strict criteria would mean that the interrogator would have to be someone of the highest level of intellect and be a skilled interrogator, able to get information out of anyone, preferably a highly trained Police Interrogator. They'd be conversing over a computer terminal with the person via an IM (Instant Messaging) Platform or other media to simulate human conversation for five minutes. Also, the person they were talking to would have to be hidden in such as a manner that they'd not know if the person they're talking to was human or machine.

Easy for any skilled interrogator, but very difficult for the Average Joe online, who has to contend with 61% of conversations online being potentially with Bot Programs that can already mimic human conversation according to analyst Incapsula in my Geezam blog article entitled “61% of all Traffic on the Internet is generated by bots”.

Russians spoof Turing Test - Eugene Goostman, the 13-y-o Chatbot from Odessa, Ukraine

After 64 years of Turing Tests since the 1950’s in 2014 history was made as detailed in “Computer passes the Turing Test”, published June 9th 2014 by Rexly Penaflorida, PC Magazine. Enter the character Eugene Goostman, a chatbot designed by Russian Researcher Vladimir Vesolov and Ukranian Eugene Demchenko, their program Eugene Goostman was based on idea of a 13 year old boy who spoke very little English.

Clever design for a chatbot; effectively not only is he bad at communication, but his bi-lingualism, which would be evident during conversation, suggests to the interrogator that he'd be fairly westernized. Displaying a lack of knowledge would suggest to the interrogator that he was dealing with a fairly young person, evidence once the interrogator asked him his age and his behavior confirmed the same.

The Turing Test was conducted on Saturday June 7^th 2014 at the Royal Society in London, England in front of 30 Judges. That date should have some resonance with fans of Alan Turning's work; that's his birthday and is also seven months after he had allegedly been caught having sex with another man. For this act, he had been charged with gross indecency, the penalty for which at the time was Chemical Castration.

But his test survives him and now, it's been cracked by Russian Vladimir Vesolov and Ukranian Eugene Demchenko's Eugene Goostman, who managed to deceive the 30 interrogators more than 32% of the time, quite a feat I must say. Many reading this would of course be skeptical, as obviously they'd say the researchers Russian Vladimir Vesolov and Ukranian Eugene Demchenko cheated by making such a character.

After all, if his English is bad, he'd be very difficult to talk to, making his character more believable. Worse, as the interrogator got more into the conversation, he'd realize he's talking to a kid, based on his level of knowledge, meaning that the conversation wasn't evenly matched.

Never mind the fact that the Turing Test wasn't beaten by a supercomputer, as this is human conversation over a computer terminal, not a chess match or Natural Language Communication over the phone as described in my blog article entitled “IBM's Watson soon to be a Watson Engagement Advisor in Call Centers - The Internship of Ask Watson 40% faster search puts Customer Service Agents in Jeopardy at The World’s End” which IBM's Watson, despite its sophistication, would not ace. Human conversation over a computer terminal via text on a screen is something even a basic computer set up as a Server can simulate very well.

Turing Test Turned up for Natural Language - How to spot a chatbot online

You can imagine what this basically means.

The Question on the minds of many at this point is how can you tell if you’re talking to a human anymore online? Even without being fully Artificially intelligent, if a computer program can deceive a highly intelligent and skilled human interrogator, then what of the rest of us in society that casually go about our business online?

We'd basically be sitting ducks for scambots and AI's that would appear to be as human as we are. To the very gullible, they may even be able to get them to solicit personal information about themselves, even send them a link to a website that will ask them to solicit their Credit Card information after pretending to be a pretty female offering them some great financial advice.

Already, Bot programs have been known to have Social Networking profiles on Facebook, even soliciting friendship from people. Heck, they can even beat the reCAPTCHA system that's supposed to discriminate between humans and machines, a discovery made by Google Street View Research Team only recently as stated in my blog article entitled “Google Street View Research indicates CAPTCHA and reCAPTCHA are machine readable – Bots can take over the Earth with Refridgerator Sentinels in Days of Future Past”.

I think the next Turing Test should be done in Natural Language, such as English or Spanish. That way, it’s not only raising the Bar for the Turing Test, but test how well how Natural Language based AI’s have come along.

We already have Real-Time Translation of Voice Conversations no on offer by Microsoft’s Skype Translator as described in my blog article entitled “Microsoft Launches Skype Translator – Star Trek Universal Translator to foster Real-Time Global Village against WhatsApp”. So it shouldn’t be hard to develop a version of the Turing Test that involves actually speaking to someone in a Natural Language over Skype and seeing if it can beat the Turing Test under those conditions.

In the meantime, fancy your chances against Eugene Goostman? Then click on his name in my blog article or just go to the link as shown below:

http://www.princetonai.com/bot/bot.jsp