Engineeringhot

Agent Engineer

Also hired as: AI Agent Developer · Agentic Systems Engineer · LLM Orchestration Engineer

The most-hired AI engineering role of 2026. You build the systems that let models actually do things.

US salary · 2026
$160k – $350k+ total comp
Tracks senior backend bands plus an AI premium; hot agent startups and labs push past the top with equity. Estimates, not offers.
Typical background
Backend and full-stack engineers who built one serious LLM side project, hit the hard parts — state, cost, failure — and decided that was the interesting bit.
This roadmap
5 stages · 24 nodes
5 proof-of-work checkpoints
What is a Agent Engineer?

An agent engineer builds the systems that let a model actually do things: call tools, keep state, recover from failure, and complete multi-step work without a human driving every turn. The model provides the reasoning; you provide everything else — tool interfaces, orchestration, memory, guardrails, evals, and cost controls. It is backend engineering where one component is brilliant, nondeterministic, and occasionally wrong, and the core discipline of the role is designing systems that stay safe, debuggable, and useful anyway.

Demand exploded because agents crossed from demo to deployment. Sierra and Decagon proved customer-facing agents generate revenue; Cognition proved coding agents ship real software; MCP standardized how models reach tools; and seemingly every SaaS company decided its product needs agentic features. That created a hiring wave for people who have actually built this — not watched conference talks about it. In 2026, 'agent engineer' and its aliases show up across AI job boards more than almost any other emerging title, and the interview usually opens with 'show me one you built.'

The role sits between two neighbors. Prompt engineers own the model-facing layer; you own the full runtime around it. MTS engineers build the models; you compose them into products. The distinct skill is systems thinking under nondeterminism: idempotent tools, durable state, spend caps, permission boundaries, and evals that grade an agent's whole trajectory rather than a single output. Engineers who have internalized that mental model are still rare, and companies pay accordingly.

What you'll actually do
  • Design agent loops: when to plan, when to act, when to reflect, and when to bail out to a human.
  • Build and maintain tool interfaces and MCP servers, treating tool schemas as API contracts.
  • Implement memory: session state, long-term stores, and context compaction that survives long tasks.
  • Orchestrate multi-agent systems — subagents, fan-out, review loops — and know when one agent is enough.
  • Build guardrails: permission boundaries, sandboxing, spend caps, and human approval gates.
  • Write agent evals that grade trajectories and task success, and wire them into CI.
  • Instrument everything: traces, token costs, latency, and a failure taxonomy you actually maintain.
  • Operate agents in production, where the incident review sometimes reads 'the model decided to.'
This role fits you if
  • You like backend and systems work but want to be at the frontier of what software can do.
  • Debugging nondeterminism sounds like a puzzle, not a nightmare.
  • Your first question about any demo is 'what happens when it fails?'
  • You already built an LLM wrapper and discovered state, cost, and loops are the real problem.
  • You enjoy API design and product thinking in roughly equal measure.
The toolbox
TypeScriptPythonClaude Agent SDKMCPTool schema designLangGraphDurable execution (Temporal)Postgres + pgvectorRedisStreaming UXTracing (OpenTelemetry)Agent evalsSandboxingCost budgeting
The roadmap — 5 stages, 24 nodes
Stage 1

LLM and Tool-Use Foundations

Weeks 1–4

Understand the model as a component — capabilities, failure modes, and the tool-calling loop everything is built on.

The model as a system componentCore
Tokens, context budgets, temperature, and the failure modes that matter to engineers: hallucinated arguments, truncation, refusal. You are learning a dependency's spec sheet.
What makes a system agenticCore
The loop: model proposes an action, runtime executes it, result feeds back. Learn the autonomy spectrum from single tool call to open-ended task, and where products actually sit on it.
First tool-calling appCore
Define three tools with typed schemas, handle the call-execute-return loop by hand, and log every step. No frameworks yet — you need to see the wires.
Drive an agent dailyElective
Use Claude Code as a power user while you build. Nothing teaches agent UX, permission design, and failure recovery faster than living inside a production agent.
Checkpoint: ship a single-agent app✓ Checkpoint
A working agent with three-plus real tools (search, fetch, file write) and a visible reasoning loop. It must survive a tool failure mid-run without crashing or hallucinating a result.
Stage 2

MCP and the Tool Layer

Weeks 5–8

MCP is the USB port of the agent ecosystem. Building servers — not just using them — is the differentiating skill.

MCP deep diveCore
Servers, tools, resources, and transports — and why a standard protocol beats bespoke integrations. Wire two existing servers into an agent and read their source.
Build your own MCP serverCore
Wrap a real API with auth, pagination, and typed errors. The hard parts are the same as any API client — plus deciding what the model should never be allowed to do.
Tool design is API designCore
Names and descriptions are prompts; schemas are contracts. Learn when to expose one flexible tool versus five narrow ones, and how bad tool docs poison agent behavior.
Checkpoint: publish an installable MCP server✓ Checkpoint
A server for a real API that a stranger can install: repo, README with a working Claude Code config, auth handling, typed errors. One issue filed by someone you don't know is the real success metric.
Stage 3

State, Memory, and Orchestration

Months 3–4

Single agents plateau fast. Durable state and multi-agent patterns are where real products get built.

Memory architectureCore
Session state vs long-term stores, context compaction, and summarization thresholds. Decide explicitly what the agent remembers, forgets, and re-derives.
Multi-agent patternsCore
Roster patterns, fan-out/fan-in, and reviewer loops — plus the honest counterpoint: multi-agent adds cost and failure surface, so learn when one agent with good tools wins.
Durable executionCore
Long-running agents must survive restarts, rate limits, and human-approval pauses. Learn checkpointing and queue-backed execution — Temporal-style thinking, even without Temporal.
Retrieval inside the loopElective
RAG as a tool the agent calls, not a pipeline bolted on front. Build one agent that decides for itself when to search its knowledge base.
Checkpoint: a resumable multi-agent pipeline✓ Checkpoint
A pipeline (research → draft → review) that completes a real task end to end, persists state, and resumes cleanly after you kill the process halfway through. Killing it is part of the test.
Stage 4

Guardrails, Evals, and Cost

Months 4–6

The difference between a demo and a product is what happens when things go wrong — and whether you can prove they usually don't.

The agent threat surfaceCore
Injection now arrives through tool results, web content, and files — not just user input. Design permission boundaries assuming every external string is hostile.
Agent evalsCore
Grade trajectories, not just final answers: did it pick the right tools, in a sane order, at acceptable cost? Build a task suite with an LLM judge and wire it into CI.
Cost engineeringCore
Token budgets per task, model routing (cheap model for triage, frontier for hard steps), caching, and kill switches. Agents without spend caps eventually write very expensive poetry.
Sandboxing and approval UXElective
Execution sandboxes, allowlists, and human-approval gates for irreversible actions. Study how Claude Code's permission model does it — it is the reference implementation.
Checkpoint: harden your pipeline✓ Checkpoint
Retrofit the Stage 3 project: 50+ graded tasks in CI, a hard spend cap, injection test cases delivered via tool results, and a writeup of the three worst failures your evals caught.
Stage 5

Production and the Job

Months 6–9

Operate an agent with real users, then convert the scars into interview material. This is the shortest path to offers.

Deploy and operateCore
Streaming UX, retries with idempotency, timeouts, and traces you can actually read at 2am. An agent you cannot observe is an agent you cannot ship.
Agents in the dev loopCore
Auto-PR pipelines, code-review agents, CI triage. Companies adopt internal agents before customer-facing ones — this is where many first agent-engineer jobs actually live.
Portfolio over resumeCore
Two or three production-grade repos with postmortems beat ten demos. Write up what broke and what you changed — hiring managers read failure analysis as seniority.
The interviewCore
Expect system design ('design a support agent: tools, memory, evals, guardrails') plus deep questions on cost and failure handling. Your checkpoint projects are the answers; rehearse them.
Checkpoint: two weeks of real users✓ Checkpoint
Run one agent with real users — ten is enough — for two weeks. Publish the numbers: task success rate, cost per completed task, and a failure taxonomy. This artifact outperforms any resume line.
Build your portfolio

Nobody hires a Agent Engineer off a certificate. They hire off proof. Ship these and put them where people can click them:

01
An agent that survives failure
A single agent with three-plus real tools and a visible reasoning loop that keeps working when a tool fails mid-run — no crash, no hallucinated result.
Proves: You understand the loop at the wire level, not just through a framework.
02
An installable MCP server
A real API wrapped with auth, pagination, and typed errors, plus a README config a stranger can install from. An issue filed by someone you've never met is the success metric.
Proves: You design tool interfaces as API contracts — the differentiating skill of the role.
03
A resumable multi-agent pipeline
Research → draft → review completing a real task end to end, state persisted, resuming cleanly after you kill the process halfway through.
Proves: You can build durable orchestration — where single-agent demos die and products begin.
04
The hardened pipeline, evals in CI
The same pipeline with 50+ graded tasks gating merges, a hard spend cap, injection cases delivered through tool results, and a writeup of the three worst failures the evals caught.
Proves: You build the guardrails and measurement that separate a demo from a product.
05
An agent with two weeks of real users
One agent, ten-plus users, two weeks — published task success rate, cost per completed task, and a failure taxonomy built from real traces.
Proves: You have operated an agent, not just built one — the scar tissue interviews actually probe for.
Position your profile
Headline formula

I build agent systems that survive production: [agent] ran at [X]% task success and $[Y] per completed task, with trajectory evals gating every deploy.

Resume bullets to earn
  • Shipped [agent] to [N] real users; sustained [X]% task success at $[Y] per completed task over [Z] weeks, failure taxonomy published.
  • Built and published an MCP server for [API]: [N] installs, issues filed and fixed for strangers.
  • Wired [N]+ trajectory-graded eval tasks into CI; caught [N] regressions before deploy, including [the worst one].
  • Cut cost per completed task [X]% through model routing and caching without moving task success.
  • Contained [N] of [N] injection attempts delivered via tool results in red-team testing of the agent's permission model.
Where to be visible
  • GitHub with production-shaped repos: traces, evals, spend caps, and postmortems in the README. Hiring managers read failure analysis as seniority.
  • Publish MCP servers — the ecosystem is young enough that one genuinely useful server gets found, installed, and cited.
  • Agent teardowns: how Claude Code handles permissions, how Sierra-class products do memory and cost. These are the posts practitioners actually share.
  • Contribute fixes to the tools you use — MCP servers and SDKs, LangGraph, agent frameworks. A merged fix beats a thread of takes.
  • Build-in-public logs with numbers: a two-week 'real users' report with success rates and costs is rarer and louder than any opinion piece.
What interviews actually test
  • Live system design: 'design a support agent' — tools, memory, evals, guardrails, cost. Rehearse the whole shape until you can draw it without pausing.
  • Trace debugging: here's a failing trajectory, find the fault. The skill is pattern recognition, and it only comes from reading your own traces daily.
  • Failure and cost interrogation: what happens when a tool times out mid-run? Where's the spend cap? Every answer should come from something you operated, not something you read.
  • Tool and schema design: spec the tools for a scenario — names, descriptions, error shapes. Your MCP server is the direct rehearsal.
  • The portfolio deep-dive: success rates, cost per task, worst failures, what you changed. Numbers from real users outrank architecture opinions every time.
Who's hiring
SierraDecagonCognitionAnthropicOpenAILangChainHarveyRampVercelEvery SaaS adding agentic features
Free courses that map to this role
Related roles
FAQ

What does an agent engineer actually do all day?

Mostly backend engineering with a twist: designing tool interfaces, debugging trajectories from traces, tuning prompts and memory, reviewing eval results, and adding guardrails after an agent did something surprising. Less time prompting than outsiders expect; more time on state, cost, and failure handling.

Do I need ML experience to become an agent engineer?

No. You consume models through APIs; nobody expects you to train them. Strong backend fundamentals — API design, state management, queues, observability — matter far more than ML theory. The model-specific knowledge fits in a few focused weeks.

Agent engineer vs AI engineer — what's the difference?

'AI engineer' is the umbrella for building on foundation models; agent engineer is its systems-heavy specialization focused on autonomy: tools, orchestration, guardrails, agent evals. Job postings blur the titles, so read the responsibilities, not the header.

Which framework should I learn — LangChain, LangGraph, or raw APIs?

Start raw: hand-roll the tool loop so you understand what frameworks abstract, then learn MCP because it is a protocol, not a framework fad. After that, pick up LangGraph or the Claude Agent SDK as your team requires. Frameworks churn; the primitives don't.

Is agent engineering a fad title?

The title may evolve; the work will not. Giving software safe autonomy — tools, memory, guardrails, evals — is a decade-scale project, the way 'webmaster' became frontend, backend, and SRE. Learn the durable primitives and any future title change is a rename, not a restart.

Start building, today

Every stage above maps to free lessons on this site. No signup, no paywall — open the first course and ship your first checkpoint this week.

Browse the courses →