GPT-4 was able to trick a human in the Turing test.
Scientists from the University of California, San Diego published a study on the possibility of modern artificial intelligence to bypass the famous Turing test-a criterion that determines the ability of a machine to imitate human behavior in a conversation so convincingly that the interlocutor will not be able to distinguish AI from a real person.
The authors used the methodology proposed by Alan Turing 70 years ago. During the experiment, they organized 1,400 games in which 650 participants had short conversations with either a human or a GPT model. The task of the participants was to determine who is their interlocutor.
As a result, it turned out that the GPT-4 models misled participants in 41% of cases, while the previous version of GPT-3.5 – only 5-14%. People convinced participants that they were not machines 63% of the time.
Thus, the scientists concluded that GPT-4 does not pass the Turing test, but stressed that the test remains an important tool for evaluating the effectiveness of machine dialogue. However, the ability of the GPT-4 model to convince people of their "humanity" in 41% of cases suggests that deception of a person using AI in certain contexts can already be real, especially when people are less attentive to who they are talking to.
The researchers noted that participants who correctly identified the interviewee paid attention to several factors: formality or informality of speech, brevity or verbosity, grammar and punctuation, and the tendency to standard responses.
Scientists suggest that with the growth of "humanity" and the strengthening of individual characteristics in the conversation of AI, it will become even more important to identify factors leading to deception and develop strategies to prevent it. The report raises important questions about the social and economic implications of the widespread use of artificial intelligence, which can successfully mimic human communication, and strategies that can help society adapt to the new challenges associated with the development of AI.
Scientists from the University of California, San Diego published a study on the possibility of modern artificial intelligence to bypass the famous Turing test-a criterion that determines the ability of a machine to imitate human behavior in a conversation so convincingly that the interlocutor will not be able to distinguish AI from a real person.
The authors used the methodology proposed by Alan Turing 70 years ago. During the experiment, they organized 1,400 games in which 650 participants had short conversations with either a human or a GPT model. The task of the participants was to determine who is their interlocutor.
As a result, it turned out that the GPT-4 models misled participants in 41% of cases, while the previous version of GPT-3.5 – only 5-14%. People convinced participants that they were not machines 63% of the time.
Thus, the scientists concluded that GPT-4 does not pass the Turing test, but stressed that the test remains an important tool for evaluating the effectiveness of machine dialogue. However, the ability of the GPT-4 model to convince people of their "humanity" in 41% of cases suggests that deception of a person using AI in certain contexts can already be real, especially when people are less attentive to who they are talking to.
The researchers noted that participants who correctly identified the interviewee paid attention to several factors: formality or informality of speech, brevity or verbosity, grammar and punctuation, and the tendency to standard responses.
Scientists suggest that with the growth of "humanity" and the strengthening of individual characteristics in the conversation of AI, it will become even more important to identify factors leading to deception and develop strategies to prevent it. The report raises important questions about the social and economic implications of the widespread use of artificial intelligence, which can successfully mimic human communication, and strategies that can help society adapt to the new challenges associated with the development of AI.
