AI21 Labs, a developer of large language models, has published the results of its latest Turing study, “Human or Not”. Participants were made to chat for two minutes and then asked to guess whether they were chatting with a human or an AI chatbot.
After more than a month of millions of tests, the results showed that 32 percent of people could not tell the difference between a human and an artificial intelligence. “Human or not?” used an AI bot based on leading language models such as GPT-4 and Jurassic-2.
Some more results from the test – 68 percent of people guess correctly when asked if they were talking to a fellow human or an AI bot. It was easier for people to identify fellow human beings – when talking to people, the participants guessed correctly in 73 percent of cases. When talking to the bots, the participants guessed correctly only 60 percent of the time.
France has the highest percentage of correct guesses at 71.3 percent (above the overall average of 68 percent), while India has the lowest percentage of correct guesses at 63.5.
Some of the more popular approaches and strategies used to determine if one was speaking to a human: Humans tend to detect spelling and grammatical errors with humans, use of slang, and politeness were considered artificial intelligence.
Source: AI21 Labs
