When LLMs meet DSLs

🌈 Abstract

The article discusses the limitations of current AI agents, particularly "Choose Your Own Adventure" (CYOA) agents and code-generating agents, and proposes the use of domain-specific programming languages (DSLs) as a potential solution to improve the efficiency, complexity, and safety of AI agents.

🙋 Q&A

[01] Instructions is all you need

1. What are the two main types of AI agents discussed in the article?

• The two main types of AI agents discussed are:
  • "Choose Your Own Adventure" (CYOA) agents, which require the user to provide a series of predefined actions for the agent to execute
  • Code-generating agents, which generate and execute code (e.g., Python) to accomplish tasks

2. What are the key issues with CYOA agents?

• CYOA agents are inefficient, as they take several seconds to decide on the best course of action, and performing the same tasks manually is faster
• CYOA agents are not designed for repetition, as executing the same task requires the user to ask the agent the same thing every time, which is a lengthy and expensive process
• CYOA agents don't model complex behavior, as they can only execute a sequence of actions and cannot handle things like loops or control flow

3. What are the key issues with code-generating agents?

• Code-generating agents do not have enough formal guardrails, which presents the possibility of a security disaster if the generated code is executed with little human supervision
• Code-generating agents are inefficient, as the generated code is often boilerplate, unoptimized, and unsafe, requiring a lengthy process of trial and error to correct errors
• As code-generating agents advance, they may produce so much code that manual review becomes all but impossible, even if formal guardrails are implemented

[02] Why custom programming languages might be the missing piece

1. How do domain-specific programming languages (DSLs) address the limitations of CYOA and code-generating agents?

• DSLs can be fast and efficient, as they would iterate on concise, domain-specific code
• DSLs can model complex behavior, as they are code-like and have control flow
• DSLs can have formal guardrails, as the design of the DSL can be controlled
• DSLs are designed for repeatability, as the generated code can be executed later
• DSLs generate code that is easier to review, as it would be more readable and debuggable compared to the verbose, specialized code generated by code-generating agents

2. Why might DSLs be a better solution for AI agents compared to traditional programming languages?

• As computer programming has become more abstract, with a move from low-level languages to higher-level languages like Python, the code generated by AI agents may become too long, verbose, and specialized for human review
• DSLs for AI-written code could provide a new level of abstraction that is more reviewable by humans, allowing for better control and safety of AI agents