Meet the AI Agent Engineer

🌈 Abstract

The article discusses the role of Agent Engineers at Sierra, a company that enables businesses to build their own branded, customer-facing AI agents for customer service, commerce, and more. It highlights the key responsibilities and skills required for this emerging engineering discipline, as well as the technical challenges involved in building and deploying AI agents at scale.

🙋 Q&A

[01] Building and Deploying AI Agents at Scale

1. What are the key differences between building a demo and deploying an AI agent at scale?

• Building a demo showcasing the potential of AI agents can be done relatively quickly, even with a well-directed prompt.
• Enabling agents to operate at scale consistently across millions of conversations is much more challenging and requires a new software development approach and a new type of software engineer.

2. What are the key responsibilities of Agent Engineers at Sierra?

• Agent Engineers work with customers to design, build, and ship AI agents using Sierra's platform, Agent OS.
• They are at the forefront of what's possible with AI and are among the first to productionize large language models (LLMs) to solve consumer problems at scale.

3. What are some examples of complex problems that Sierra's Agent Engineers have tackled?

• Helping Sonos build an AI agent that can troubleshoot speaker connection issues by understanding the details of the Sonos product portfolio and translating that understanding into a reliable AI agent.
• Designing and deploying agents that tackled similarly complex problems for other customers.

[02] The Role of an Agent Engineer

1. How does the role of an Agent Engineer differ from other engineering roles?

• AI agents are nondeterministic and built on tools uncommon in modern software stacks.
• In addition to programming, Agent Engineers must master a modern AI stack, developing expertise in areas like prompt engineering, model selection, and agent orchestration.

2. What technical competencies and personal attributes are important for Agent Engineers?

• Strong software engineering skills, but also an innate curiosity and respect for the complexity of their customers' challenges.
• The ability to understand and apply domain expertise to the development of AI agents.

3. What are the key areas that Agent Engineers at Sierra think about frequently?

• Interacting with systems: Ensuring agents can access the necessary APIs and systems of record to complete transactions, not just answer questions.
• Agent supervision: Layering additional AI models to verify agent outputs and ensure appropriate responses.
• Agent extensibility: Building agents with composable skills and higher-order components to enable flexibility and durability over time.

[03] Technical Challenges in Building and Deploying AI Agents

1. What are the limitations of simplistic AI agents built solely on Retrieval-Augmented Generation (RAG)?

• RAG-based agents can only answer questions using selected customer documents, without the ability to take actions or complete transactions.
• They lack the necessary orchestration and integration with underlying systems to achieve consistent, production-ready workflows.

2. How does Sierra's Agent SDK address these limitations?

• The Agent SDK enables Agent Engineers to construct agents by stacking composable skills and enforcing deterministic API interactions.
• It powers the orchestration necessary to manage agent control flow and interact with the right systems of record.
• It also allows for configuring agent responses based on user-specific rules, such as regional regulations or plan-specific policies.

3. How do Sierra's Agent Engineers ensure appropriate agent responses in the face of unbounded language inputs?

• They layer additional AI models, known as supervisors, to verify agent outputs and revise responses as needed.
• The Agent SDK also allows them to tune agent creativity and determinism based on the context, selectively bypassing the LLM to achieve the necessary consistency.

[04] The Evolving Role of Agent Engineering

1. How does Sierra's approach to agent development differ from fine-tuning an LLM to a particular use case?

• Sierra's Agent Engineers use the Agent SDK to define agent behavior, composing skills expressed using a declarative programming language.
• This composability makes agents quicker to build and deploy, as well as more durable and extensible over time.

2. What are the potential benefits of successfully developed and deployed AI agents?

• Agents can become an integral part of a company's identity, a cornerstone of its brand, and a key element of its competitive advantage.
• They can double or even triple the resolution rate of existing solutions while increasing customer satisfaction, and deliver personalized experiences that were once the realm of science fiction.

3. What backgrounds and experiences are common among Sierra's Agent Engineers?

• The role is not one-size-fits-all, with team members having diverse backgrounds, including infrastructure engineering, entrepreneurship, and business education.
• A combination of engineering expertise, customer obsession, and business curiosity are seen as defining characteristics of successful Agent Engineers.