What Does Production-Ready Mean When AI Writes the Code?

Coding agents have made one kind of software demo extraordinarily easy.

Describe a feature. Wait. Refresh the browser. Something appears to work.

That is useful. It is also where a dangerous amount of software confidence can come from.

The tools themselves are not the problem here. Anthropic's framework for developing safe and trustworthy agents is explicit that agent autonomy is something to be paired with human oversight rather than a replacement for it, and OpenAI describes Codex as proposing changes for review. Both vendors position their agents inside an engineering process. The confidence problem arises when teams quietly remove that process because the output arrives so quickly and looks so finished.

"Works in the demo" and "ready for production" have never meant the same thing. Coding agents make the distance between those two states easier to overlook because the first state arrives so quickly.

So what should production-ready AI code actually mean?

Not perfection. No serious engineering team can promise that.

A more useful definition is:

The implementation has explicit behaviour, known failure modes, appropriate safeguards and enough evidence that the team understands what will happen outside the happy path.

Start with behaviour, not files

Production readiness is not a property of the amount of code written.

It starts with behaviour.

For a recurring software capability, we should be able to describe:

• valid states;
• invalid states;
• transitions between states;
• who is allowed to trigger them;
• what external systems can change them;
• what happens when operations partially fail;
• how recovery works.

If those answers are fuzzy, a beautifully structured implementation is still risky.

Coding agents can help discover these questions, but the questions need to become explicit.

Failure behaviour matters as much as successful behaviour

Imagine a webhook endpoint.

The happy-path requirement is simple:

Receive event. Update database. Return success.

Production introduces different questions:

• What if the event is delivered twice?
• What if two copies arrive concurrently?
• What if the database write succeeds and the process crashes before acknowledgement?
• What if the first attempt records the event but fails before applying the business effect?
• What if an older event arrives after a newer one?
• Which failures should cause the provider to retry?

Those are not obscure theoretical cases.

Distributed systems produce these situations naturally.

A production implementation must make its behaviour under these conditions deliberate rather than accidental.

We look at one example in Stripe Webhook Idempotency Is Not Just Deduplicating Event IDs.

What "accidental" looks like in practice

Two examples from Udria's own codebase, both found by writing tests rather than by reading code. Neither was exotic, and neither produced an error message.

The first was an event ledger that recorded that a webhook had been seen rather than that it had been processed. A transient database failure after the ledger insert meant the retry was greeted as a duplicate and acknowledged, so the provider stopped retrying and the event was applied by nobody. Every component behaved as specified. The specification was wrong.

The second was quieter still. A handler for a failed card payment matched rows on setup_intent_id AND status = 'started'. But setup_intent_id was only written when a reservation completed — so the two conditions could never be true at once. The handler matched nothing, ever. An entire status was unreachable, and because its job was to mark failures, the absence of failures looked exactly like success.

Neither bug would be caught by asking whether the code looks correct. Both were caught by asking what must be true after this runs, and then checking.

That is the practical difference between reviewing code and verifying behaviour.

Security needs explicit boundaries

An AI-generated feature can be functionally correct and still unsafe.

Production readiness requires questions such as:

• Where is authorisation enforced?
• Are privileged operations checked server-side?
• What user-controlled input crosses a trust boundary?
• Which secrets exist?
• Are secrets exposed to the client?
• Are signatures verified?
• Are database policies consistent with application assumptions?
• What data should never be logged?

"Use best practices" is not a security specification.

The relevant boundaries should be named.

The failure worth guarding against is subtler than a missing check. It is a check that exists in the wrong place. An agent asked to add an admin feature will frequently produce a client component that hides the admin controls unless the user is an admin — which is correct, looks correct, and demos correctly. The server action behind those controls may have no check at all, because from the agent's point of view the requirement "only admins can do this" was satisfied by the interface.

The specification that prevents this is one sentence: authorisation is enforced on the server, in the same function that performs the mutation, and the interface merely reflects it. Given that sentence, an agent implements it every time. Without it, roughly half the time you get a beautiful, entirely decorative permission system.

The same applies to safety gates. A gate that only the interface respects is not a gate. One of Udria's own — the flag that blocks live card collection until the terms have had legal review — originally tested whether the Stripe key began with sk_live_. Restricted keys begin with rk_live_, and restricted keys are the recommended production credential. The gate was real, enforced server-side, and would have waved through exactly the configuration a careful team is most likely to deploy. It now asks Stripe which mode the key belongs to rather than pattern-matching one key shape.

Migrations are part of the feature

A greenfield demo starts from an empty database.

A real product already contains:

• users;
• organisations;
• historical records;
• partially complete data;
• deployment processes;
• live traffic.

A production-ready capability should consider:

• schema changes;
• defaults;
• nullability;
• indexes;
• unique constraints;
• backfills;
• rollback;
• compatibility during deployment.

If a migration can leave an existing customer unable to use the product, the migration is part of the product behaviour.

There is an ordering constraint here that agents will not infer, because it is a property of your deployment process rather than of your code. During a rolling deploy, old code and new code run against the same database at the same time. A migration that renames a column, tightens a constraint, or adds a not null without a default will break the instances that have not been replaced yet, and it will do so in the window when you are least able to reason about what is happening.

The pattern that survives this is unglamorous and worth specifying explicitly: add columns as nullable, backfill, then add the constraint. Never rename in a single step — add, dual-write, migrate readers, drop. Make every migration safe to run twice, because at some point one will be. A specification that says only "add a migration" gets you code that works perfectly against an empty test database.

Observability is not optional after failure

A system can fail correctly and still be operationally useless if nobody can tell what happened.

Depending on the feature, useful observability may include:

• structured errors;
• event identifiers;
• correlation IDs;
• audit records;
• retry counts;
• status transitions;
• metrics;
• alerts.

You do not need enterprise telemetry for every small feature.

You do need enough information to answer:

Why did this user end up in this state?

In practice that question is almost always answered by state transitions rather than by error logs. An error tells you that something failed once; a transition history tells you the sequence that produced the state somebody is complaining about, which is usually the actual question.

For anything driven by external events, the minimum useful record is: which event, when it arrived, how many attempts it has had, what the last error was, and what state it is in now. Those five columns turn "billing is broken for this customer" from an investigation into a query. They are also almost free, because the table recording them already has to exist for idempotency.

Tests should verify invariants, not screenshots

Coding agents are particularly good at generating tests.

That only helps if the tests represent meaningful behaviour.

For example, a subscription implementation might have an invariant:

Re-delivering the same successfully processed provider event must not grant access twice.

A permissions system might have:

Removing a role must remove every permission derived only from that role.

A scheduling system might have:

Two confirmed bookings cannot occupy the same exclusive resource at the same time.

These are valuable because they describe what must remain true across implementations.

That is much stronger than testing that a button exists.

A test that has never failed has not been tested

There is a failure mode specific to agent-written tests, and it is worth naming precisely because the tests look thorough.

If the same misunderstanding produces both the implementation and its tests, the tests pass. They will keep passing. They encode the bug as the expected behaviour, and they do it in the confident vocabulary of a well-written test suite. The ledger bug above had tests. They were green. They tested that a duplicate delivery had no second effect — which was true, and which was not the property that mattered.

The check that separates a real regression test from a decorative one takes about a minute: run it against the broken implementation and confirm that it fails. If it passes against both the broken and the fixed version, it is testing something other than the thing you were worried about.

This is worth doing once, by hand, for every invariant that protects money, access or data. It is also the single cheapest quality practice we know of in agent-assisted work, because it verifies the specification rather than the code — and in the coding-agent era the specification is where the defects live.

Evidence the team can inspect later

One more property separates production-ready from merely working, and it is the one that is hardest to retrofit: somebody other than the author has to be able to establish that the thing is correct.

With human-written code this happened implicitly. The author could explain the reasoning, and the reasoning lived in somebody's head for as long as they stayed at the company. With agent-written code there is no such head. The agent that wrote it has no memory of the session, and the person who prompted it may have read the diff once.

So the artefacts have to carry the reasoning. In practice that means three things travel with any critical feature:

• The invariants, written down, in the repository rather than in a ticket. These are the claims the implementation is making about itself, and they are what a future reader checks the code against.
• Tests that map to those invariants by name, so that when one fails, the failure says which promise was broken rather than which function threw.
• A short note on what was deliberately not handled. The most dangerous thing in an agent-assisted codebase is a gap that looks like an oversight but was a decision, or an oversight that everyone assumes was a decision.

That last one costs two sentences and saves an afternoon. The webhook ledger described earlier now carries a comment explaining why a failed handler must not delete its own claim row — which is the fix somebody will otherwise reach for, because it is the obvious one, and it reintroduces the original bug by a longer route.

The agent should be allowed to adapt

Production-ready does not mean prescribing every function name.

An existing repository already has conventions.

A coding agent should inspect:

• data-access patterns;
• authentication;
• organisation model;
• component structure;
• API style;
• error handling;
• test framework.

Then it should implement the required behaviour in a way that fits those conventions.

That is one reason we think capabilities should be defined more by outcomes, constraints and acceptance tests than by fixed source-code templates.

"It compiled" is evidence of almost nothing

Compilation is useful.

Passing unit tests is useful.

Neither tells the whole story.

A stronger verification ladder looks something like:

1. code compiles;
2. static checks pass;
3. unit tests pass;
4. integration tests pass;
5. acceptance tests exercise required behaviour;
6. failure cases are deliberately tested;
7. a human or trusted review process inspects security-sensitive logic;
8. deployment and rollback paths are understood.

Not every feature needs the same level of scrutiny.

Billing deserves more than a colour-picker.

The important point is that confidence should come from evidence appropriate to the risk.

A ladder, applied by risk

Putting the pieces together, the practical form of this is not a checklist applied uniformly but a ladder where the risk of the feature decides how far up you climb.

Tier	Examples	What production-ready means
Cosmetic	Layout, copy, a colour picker	It compiles, it type-checks, someone looked at it
Functional	Internal dashboards, filters, exports	The above, plus unit tests over the logic that is easy to get wrong
Stateful	Anything writing data a user will later rely on	The above, plus migration safety, integration tests, and a defined rollback
Critical	Money, access control, external events, anything irreversible	The above, plus named invariants, tested failure paths, idempotency, observability of state transitions, and human review of the security-sensitive parts

The tier is decided by blast radius, not by how difficult the feature was to build. Billing that took twenty minutes to write is still critical. An intricate internal report that took two days is still functional.

This matters more in agent-assisted work than it used to, for a simple reason: the effort a feature took is no longer a signal of how much it matters. That correlation was always rough, but teams leaned on it heavily and mostly got away with it. It is now close to useless, and something explicit has to replace it.

Where this standard should be relaxed

A definition of production-ready that applies equally to everything gets ignored, so it is worth saying plainly where this is too much.

Internal tools with a handful of trusted users, throwaway prototypes, and features whose worst-case failure is a cosmetic annoyance do not need a failure taxonomy or an acceptance suite. Applying the full ladder there is not rigour; it is a tax that trains the team to treat the whole standard as ceremony.

The distinction that has held up for us is not how complex a feature is, but what it touches. Anything that moves money, grants or revokes access, mutates data that cannot be regenerated, or talks to a system that retries is worth the full treatment. Almost everything else is not.

That test is also easy to apply to an agent's output before reviewing it, which matters when there is more output than there is review capacity.

The question to ask before shipping

All of this reduces to one question that is worth asking out loud before a critical feature goes out, and it is deliberately not "is this correct?"

If this is wrong, how will we find out?

The answers sort themselves quickly. A customer will email us is a bad answer. An alert fires is a good one. Nothing — it will fail silently and the numbers will just be slightly wrong is the answer that should stop the deploy, and it is the honest answer far more often than teams expect.

Both Udria bugs described earlier had that third answer. A webhook event applied by nobody produces no error. A status that can never be set produces no error either — it produces an absence, and absences do not page anyone. In both cases the system's response to being wrong was to look exactly like a system that was right.

That is the property worth designing against, and it is why observability of state transitions earns its place on the critical tier. Not because failures are common, but because the failures that matter are the ones that do not announce themselves.

AI makes this more important, not less

When code was slow to create, implementation effort naturally forced teams to spend time with a problem.

When code appears in minutes, it becomes easier to skip straight from intention to deployment.

The answer is not to slow coding agents down artificially.

It is to move discipline into:

• specification;
• constraints;
• tests;
• review;
• verification.

That is what production-ready increasingly means when AI writes the code.