Is 'gpt2-chatbot' Actually GPT-5? This Mystery LLM is Going Viral

🌈 Abstract

The article discusses the rise of a mysterious AI model called "gpt2-chatbot" in the LMSYS chatbot arena. It examines the model's impressive capabilities, the theories around its origins, and the speculation surrounding its potential connection to OpenAI's models.

🙋 Q&A

[01] Everything we know about gpt2-chatbot

1. What is the gpt2-chatbot model?

• The gpt2-chatbot model first appeared on the LMSYS arena, a platform to test and rank large language models (LLMs).
• It has excellent performance in mathematical and logical puzzles, coding, and reasoning.
• The model is available for chatting within the "Direct Chat" and "Arena (Battle)" sections of the LMSYS platform.
• There is no official information about the model on the LMSYS site or elsewhere, making it a "Mystery Model".
• The results generated by LMSYS benchmarks for this model are not publicly available through their API.

2. What are the popular theories about the origin of the gpt2-chatbot model?

• There are two main theories:
  • It is an early version of GPT-4.5 or GPT-5, being stealthily tested by OpenAI.
  • It is a modified version of the old GPT-2 model, fine-tuned on modern assistant datasets.

3. What evidence supports the theory that gpt2-chatbot is an early version of GPT-4.5 or GPT-5?

• The model appears to use OpenAI's tiktoken tokenizer, which is used in their other models.
• The quality of the model's output, including its formatting, verbosity, structure, and overall comprehension, is considered to be at the level of a step from GPT-3.5 to GPT-4.
• The model's structured replies are influenced by techniques like modified Chain-of-Thought (CoT).
• The model appears to utilize the same special tokens as different OpenAI models, such as GPT-4.

[02] Amazing Capabilities of the gpt2-chatbot

1. How does the gpt2-chatbot model compare to other prominent chatbots?

• The gpt2-chatbot is said to be more capable than ChatGPT, Claude 3 Opus, and even GPT-4.
• It has impressive performance in solving complex mathematical and logical puzzles, including problems at the level of the International Math Olympiad.
• The model also excels at tasks like creating ASCII drawings and solving challenging coding problems.

2. What are the limitations in accessing and testing the gpt2-chatbot model?

• The model is accessible on the LMSYS arena, but requests are limited to 8 per day.
• The model's rate limit is only 1000 requests per hour, making it nearly impossible for individuals to extensively test the model.
• Researchers and AI experts are running tests on the model, but the full extent of its capabilities is not yet known.