LLMs are not suitable for (advanced) brainstorming

🌈 Abstract

The article discusses the limitations of large language models (LLMs) in effective brainstorming and generating truly innovative ideas, especially for frontier problems. It argues that LLMs are primarily trained to mimic existing patterns in the training data, and thus tend to converge towards the consensus in the data rather than being truly creative. The article suggests that this is a fundamental challenge in the current LLM training paradigm, and proposes some potential solutions, such as curating specialized datasets for brainstorming, using reinforcement learning approaches to reward creativity, and exploring alternative training approaches beyond the autoregressive language modeling paradigm.

🙋 Q&A

[01] Limitations of LLMs in Brainstorming

1. What are the key reasons why LLMs are not well-suited for effective brainstorming, especially for frontier problems?

• LLMs are trained to follow existing patterns in the human-produced corpus, and are not natively taught to "brainstorm"
• Their main training goal is to mimic the probabilistic distribution of the input data, which allows them to have some creativity in the sense of "extrapolation" of a pattern, but not true innovation
• LLMs often converge to the consensus in the existing data, which can lead to ideas that are similar to what is already covered in the media (i.e., buzzwords)
• When asked about topics that don't have a clear consensus, LLMs are more susceptible to issues like hallucination and cannot provide much useful insight beyond clichés

2. What observations support the claim that LLMs converge to the consensus in the existing data?

• The idea lists generated by different LLMs are often very similar, even though there is no single source on the internet that directly provides such a list
• The occurrence and preference of ideas generally aligns with the frequency and attention given by the media and main information sources, rather than being based on the actual practicality and creativity of the ideas

[02] Potential Solutions

1. What are the potential solutions suggested in the article to address the limitations of LLMs in brainstorming?

• Curate a specialized dataset of good brainstorming examples on non-conventional topics, potentially by human experts and innovators in various fields
• Use methods like RLAIF to iteratively critique the LLM's responses in terms of creativity, with the assumption that the general standard of creativity should be learnable in regular LLM training
• Explore alternative training approaches that go beyond simply following the existing data pattern, such as seeking out knowledge, thinking, and deductive reasoning skills, potentially by incorporating a world model or moving away from the autoregressive language modeling paradigm

2. What are the challenges and limitations of the proposed solutions?

• Curating a specialized dataset for brainstorming would be much more costly and labor-intensive
• It's unclear whether the RLAIF approach can effectively address the creativity challenge for truly frontier problems, where the creativity judgment may be dynamic and non-conventional
• The alternative training approaches suggested, such as incorporating a world model or moving away from autoregressive language modeling, are still largely unexplored and may require significant research and development efforts