We Need Better Evals for LLM Applications

🌈 Abstract

The article discusses the challenges of evaluating custom AI applications that generate free-form text, particularly in the context of multi-agent research systems. It highlights the limitations of current evaluation tools for large language models (LLMs) and the difficulties in efficiently evaluating the impact of changes to such systems. The article also touches on the cost and time implications of running evaluations, and expresses optimism that the community will develop better evaluation techniques in the future.

🙋 Q&A

[01] Challenges of Evaluating Custom AI Applications

1. What are the key challenges in evaluating custom AI applications that generate free-form text?

• The lack of efficient and standardized evaluation methods for custom AI applications, unlike the standardized tests available for evaluating general-purpose foundation models (e.g., MMLU, HumanEval, LMSYS Chatbot Arena, HELM)
• The difficulty in knowing which changes to keep in a multi-agent research system (e.g., adding a fact-checking agent) due to the inability to efficiently evaluate the impact of such changes

2. What are the two major types of applications mentioned in the article? The article mentions two major types of applications:

• General-purpose foundation models (large language models) trained to respond to a large variety of prompts
• Custom AI applications built using large language models

3. What are the limitations of the current evaluation tools for large language models?

• Leakage of benchmark dataset questions and answers into training data is a constant worry
• Human preferences for certain answers does not necessarily mean those answers are more accurate

4. What are the cost and time challenges associated with running evaluations?

• The monetary cost of running evaluations can quickly add up, especially when iteratively testing multiple ideas
• The time cost of running evaluations on a large number of examples can be significant, slowing down the speed of experimentation and iteration

[02] Potential Solutions and Future Outlook

1. What is the author's perspective on the future of evaluation techniques? The author is optimistic that the community will invent better evaluation techniques, potentially involving agentic workflows like reflection for getting LLMs to evaluate their own output.

2. What does the author encourage developers and researchers to do? The author encourages developers and researchers with ideas for improving evaluation techniques to keep working on them and consider open-sourcing or publishing their findings.