ChatGPT (Feb 13 Version) is a Chinese Room
arXiv Preprint Archive February 19, 2023 via arXiv
Summary
AI-generated from the abstractThe current version of ChatGPT (Feb 13 version) is a Chinese Room, not artificially conscious, despite potential evidence of cognitive connections. It exhibits critical errors in causal reasoning, which likely lead to artificial hallucinations—confidently wrong replies. ChatGPT can generate all possible categorical responses to the same question and respond with erroneous examples, questioning its utility as a learning tool. It also generates false references to mimic real publications, so its utility is cautioned.
Study at a glance
| Characteristics | Theoretical or philosophical paper Peer reviewed |
|---|---|
| Keywords | Cs.cl |
| Key finding | Argues that the current version of ChatGPT is a Chinese Room, not artificially conscious, based on its errors in causal reasoning, generation of all possible categorical responses, and production of false references. |
Abstract
ChatGPT has gained both positive and negative publicity after reports suggesting that it is able to pass various professional and licensing examinations. This suggests that ChatGPT may pass Turing Test in the near future. However, a computer program that passing Turing Test can either mean that it is a Chinese Room or artificially conscious. Hence, the question of whether the current state of ChatGPT is more of a Chinese Room or approaching artificial consciousness remains. Here, I demonstrate that the current version of ChatGPT (Feb 13 version) is a Chinese Room. Despite potential evidence of cognitive connections, ChatGPT exhibits critical errors in causal reasoning. At the same time, I demonstrate that ChatGPT can generate all possible categorical responses to the same question and response with erroneous examples; thus, questioning its utility as a learning tool. I also show that ChatGPT is capable of artificial hallucination, which is defined as generating confidently wrong replies. It is likely that errors in causal reasoning leads to hallucinations. More critically, ChatGPT generates false references to mimic real publications. Therefore, its utility is cautioned.