Why we no longer use LangChain for building our AI agents

🌈 Abstract

The article discusses the author's experience using the LangChain framework for building AI-powered applications, and why they ultimately decided to move away from it in favor of a more modular, building blocks approach.

🙋 Q&A

[01] Struggles with LangChain

1. What were the main issues the author faced with using LangChain?

• LangChain's high-level abstractions made the code more difficult to understand and frustrating to maintain, as the team spent more time understanding and debugging LangChain than building features.
• LangChain's inflexibility made it challenging to implement lower-level behavior required for their more sophisticated use cases, such as spawning sub-agents and dynamically changing the availability of tools their agents could access.
• As AI and LLMs are rapidly evolving, designing abstractions that can stand the test of time is incredibly difficult, and LangChain's abstractions often did not align with the team's needs.

2. How did the author contrast LangChain's approach to abstractions with simpler, built-in Python solutions?

• The author provided examples of translating an English word to Italian and fetching JSON data from an API, showing that LangChain's abstraction-heavy approach resulted in more complex and less intuitive code compared to using built-in Python packages or simpler libraries like requests.
• The author argued that good abstractions should simplify the code and reduce the cognitive load required to understand it, but LangChain's abstractions often had the opposite effect.

[02] Moving Away from Frameworks

1. What were the benefits the author's team experienced by moving away from LangChain and using a more modular, building blocks approach?

• The team no longer had to translate their requirements into LangChain-appropriate solutions, and could just code directly without the constraints of the framework.
• The modular building blocks approach allowed the team to develop more quickly and with less friction, as they could focus on the problem they were trying to solve rather than adapting to the framework's limitations.
• The team found that the core components most applications need (e.g., LLM integration, vector databases) can be easily assembled without a heavyweight framework, and the rest can be handled with regular application tasks like file management and caching.

2. Why does the author recommend keeping things simple and avoiding heavyweight frameworks, especially in the rapidly evolving AI and LLM space?

• The author argues that most LLM-powered applications have relatively simple and straightforward usage patterns, and that the majority of tasks can be achieved with simple code and a small collection of external packages.
• Frameworks are typically designed for enforcing structure based on well-established patterns of usage, which the author believes the AI/LLM space has not yet developed.
• Translating new ideas into framework-specific code can limit the speed of iteration and experimentation, which the author sees as crucial for success in the AI/LLM domain.