Skip to main content
AI Agents

AI agent failure modes, and how we design around them

Production agents fail in predictable ways. They call the wrong tool with plausible arguments, loop until budgets trip, escalate too late or too early, and hide API errors behind confident language. Production agents need policy matrices, typed tools, traceability, and kill switches. Not a prettier chat UI.

About this piece
Author
Databotiq EditorialAgents and integrations
Published
2026-05-07
Updated
2026-05-07

Builds tool-backed agents for support, ops, and internal platforms with audit requirements.

Failure mode: tool sprawl

If you give an agent twenty tools, you gave it twenty ways to hurt you. We narrow tools to verbs your business already understands: read invoice, create ticket, post credit under policy. Each tool has typed arguments validated server-side, not “stringly typed” JSON blobs interpreted by hope.

Failure mode: runaway loops

Agents retry when they are confused. Retries without caps become storms. We enforce max steps, max tool calls, wall-clock timeouts, and spend caps on paid APIs. When budgets trip, the agent stops and asks for a human with a structured summary. It does not improvise a new plan forever.

Failure mode: silent partial success

APIs return 200 with incomplete payloads. We treat idempotency keys and post-condition checks as mandatory. The agent surfaces receipts: what it attempted, what the system returned, and what changed. Users should never hear “done” when half the write failed.

Failure mode: prompt injection via tools and retrieved text

Untrusted content is not instructions. We isolate system prompts, sanitize tool arguments, and filter retrieval with ACLs. Tools cannot fetch arbitrary URLs unless you explicitly want that risk.

Opinion: the best agent UX is boring receipts

Flashy streaming tokens impress demos. Operators need receipts. Links to tickets, IDs for transactions, and trace IDs for support. We bias UX toward auditability over charisma, because your security team will not fall in love with a typing indicator.

How we evaluate agents before expanding traffic

We run red-team prompts against tool policies, replay production traces in shadow mode, and measure containment without harming customers. Expansion happens when error budgets say so, not when marketing schedules say so.

What you should ask any vendor

Ask for the policy matrix, the kill switch behavior, and the last incident postmortem. If a vendor cannot show how an agent fails safely, they cannot show how it succeeds safely either.

Next step

If you have a queue that should be agent-shaped, start with a Rapid POC that implements two tools and one retrieval source, then measure containment and escalation quality for two weeks. Evidence first, expansion second.

Related reading

Same-topic posts first, then adjacent practices.

Browse all posts
Rapid POC

What is a Rapid POC, and when should you run one instead of an RFP?

A Rapid POC is a sandboxed working build on your real systems and a bounded slice of your real data, designed to answer procurement questions that documents cannot. An RFP still has a role when compliance requires apples-to-apples comparisons, but it is a poor primary tool for AI because the risk is behavioural (models under your traffic, on your documents) and not a feature matrix.

Read the article
Unstructured Data

Unstructured data: the five places it hides in your business

Unstructured data is any payload where meaning is not already in neat rows. Email bodies, PDF contracts, call recordings, images from the field, and the long tail of notes fields your teams misuse because your structured schema never matched reality. If you only warehouse structured tables, you are flying half blind on what actually happened in operations.

Read the article
RAG / Chatbots

When to use RAG versus fine-tuning versus an agent in May 2026

RAG answers questions from a corpus you control and can cite. Fine-tuning shapes model behaviour and small specialised tasks when you own training signal. Agents plan steps and call tools under policies. Most production systems compose two of these. The failure mode is picking the buzzword instead of naming the decision the software must make.

Read the article
FAQ

Questions buyers actually ask.

Honest, specific answers tied to the thesis above. Not generic FAQ filler. If something isn't covered here,ask us directly.

Should every chatbot become an agent?

No. If the task is informational and read-only, RAG without tools is simpler and safer.

What is the minimum observability?

Trace IDs, structured tool logs, and redacted transcripts with retention policies your legal team approves.

How do we roll back?

Feature flags, traffic throttles, and instant model-off switches tied to error rate alarms.

What is the fastest proof path?

A bounded Rapid POC on a narrow queue with explicit policies and weekly eval reviews.

Want this thinking on your problem?

A short note is enough. We will reply within one business day with a Rapid POC scoping call.