Markov chains are funnier than LLMs

🌈 Abstract

The article discusses the differences between Markov chains and large language models (LLMs) like ChatGPT, and how the predictability of LLMs makes them less suitable for generating humor or creative content compared to Markov chains. It also explores the concept of humor and how it is related to unserious surprise.

🙋 Q&A

[01] What is a Markov chain?

• A Markov chain is described as a very simple and naive version of an LLM, which predicts the next word based on the current context, without considering semantics, dimensionality, or other advanced vector math.
• Markov chains are less capable than LLMs at tasks that LLMs are typically used for, but their simplicity can make them better suited for certain applications like next-word suggestions on phone keyboards.

[02] What is the author's definition of humor?

• The author defines humor as being about "unserious surprise" - the more unexpected and surprising a joke or humorous statement is, the funnier it tends to be.
• Good joke writing involves violating patterns and using more descriptive language to create a stronger "snap" or whiplash effect from the surprise.
• Humor is subjective because what qualifies as "unserious surprise" can vary across individuals and cultures.

[03] Why are LLMs not well-suited for generating humor or creative content?

• LLMs are designed to be predictable and produce the most probable next token based on their training data, which makes them unsuitable for generating surprising or creative content.
• Asking an LLM for an "original thought" is considered oxymoronic, as they are built to reproduce common patterns of speech rather than generate truly novel ideas.
• As LLMs become more advanced, the humor in their outputs tends to be lost, as they become better at producing the most average and predictable responses.

[04] What is the author's view on the potential for generating humor algorithmically?

• The author disagrees with the claim that comedy cannot be generated by an algorithm, and believes that with enough research and resources, it may be possible to generate comedy on demand.
• However, the author suggests that the current LLMs are not the right tool for this task, and that a fundamentally different type of language model would be required to effectively generate humor.