The Turing Jest

🌈 Abstract

The article discusses the limitations of artificial intelligence (AI) chatbots in generating humorous content, based on a study conducted by Google DeepMind researchers. It explores the reasons why AI-generated comedy falls short, including the reliance on common comedy tropes, the challenges in creating a coherent and engaging comedy set, and the difficulties in capturing the nuances of comedic timing.

🙋 Q&A

[01] Limitations of AI Chatbots in Generating Humor

1. What were the key findings of the study conducted by Google DeepMind researchers?

• The study found that AI chatbots, such as OpenAI's ChatGPT and Google's Gemini, are simply not funny. The chatbots produced "bland" and "generic" jokes, and avoided any "sexually suggestive material, dark humor, and offensive jokes."
• The participants, who were professional comedians, found that the chatbots' overall creative abilities were limited, and they had to do most of the work to generate humor.

2. Why do AI chatbots struggle to generate humorous content?

• The chatbots' humor is based on the extensive data they are trained on, which includes a wide range of comedy from different eras and styles. This leads them to rely on the most common comedy tropes, which can perpetuate outdated and potentially harmful stereotypes.
• Chatbots have difficulty with the key elements of a well-crafted comedy set, such as starting strong, building jokes on each other, maintaining perfect pacing, and delivering a clever and incredible finale.
• Chatbots struggle with creating a coherent narrative and maintaining the nuances of comedic timing, which are essential for effective stand-up comedy.

[02] Measuring Artificial Comedy

1. What is the "Turing Jest" proposed by the author? The author proposes a version of the Turing Test, called the "Turing Jest," to measure the ability of AI to generate humorous content. The key elements of the Turing Jest are:

• 100 up-and-coming or unknown comedians perform their bits in front of a live audience, and their performances are recorded.
• AI systems generate 100 additional 30- to 60-minute comedy sets, with everything (including the comedian, audience, and camera movements) being computer-generated.
• The real and AI-generated comedy sets are released on a streaming service, and the reviews are compared after one year. If the AI-generated sets receive better reviews, then the AI has "conquered" the final hurdle of generating humor.

2. Why does the author believe it's important to explore how to measure artificial comedy? The author believes that we have not yet truly explored how to measure the ability of AI to generate humorous content, and that it is time to start seriously thinking about it. The Turing Jest is proposed as a potential method, but the author acknowledges that it may not be perfect. Nonetheless, the author believes that it is a step towards understanding the capabilities and limitations of AI in the realm of comedy.