How to Stop AI Agents From Repeating the Same Mistake

What actually stops an AI agent from repeating a mistake?

The reviewer can add a note that says, Never show an empty sources section. The note may help. The output path does not have to obey it.

The repeat stops when a named checkpoint reads an approved rule before the risky action. The team must prove that it blocks the known bad case.

This guide uses a six-step path from correction to enforcement. A correction makes the lesson visible. A failure record preserves one event. A candidate proposes a response. A person approves a bounded tradeoff. A named checkpoint changes the next decision. Tests of the bad and valid cases, plus a saved decision record, show what happened.

First, decide which failures belong together. Three summaries may look wrong for different reasons. One has an empty sources section. Another cites the wrong source.

Those outputs look similar to a reviewer. They do not prove one shared cause.

The synthetic example is narrower. The same zero-source input produced the same empty sources section three times. That gives the team one bounded pattern to investigate.

I call that a failure class: the smallest repeatable pattern with a stable input condition and observed bad outcome. It groups evidence without claiming a settled cause.

Use the questions below to group evidence. Do not use them as an automatic rule-approval test.

This check catches common shortcuts. Do not count one event twice, group everything under bad quality, or treat the AI's explanation as the record.

The operating rule is blunt: a written lesson is not changed behavior.

Once the failure class is clear, the team still has to decide whether the evidence deserves an active rule.

When is a repeated failure ready to become a rule?

A repeat is ready for review when the team can name one stable failure class, distinct records, and the smallest condition to stop.

Admission needs more answers. What valid case must pass? Who owns and approves the rule? Which step reads it? What happens when it is old or removed?

No fixed count answers those questions. The three synthetic events did not provide statistical proof or automatic approval.

The table below is the review record I would use before a candidate becomes active.

A valid near-match is a similar case that should pass. Here, zero sources should trigger the rule. One real source should not. A runtime consumer is the named workflow step that must read the rule. The rollback state is what the workflow does safely after removal.

An approved record is still just a record. Before prescribing the checkpoint, it helps to inspect where my own earlier system stopped short.

What did WarpOS teach me about the gap between a lesson and a rule?

WarpOS is my agentic operating-system project for coordinating and testing long-running AI work.

I inspected its public code and documents at frozen commit 8b083d7. This review of fixed public bytes did not prove that each procedure ran in a real session.

The frozen design had a useful learning lifecycle. It separated logged, validated, and implemented, required evidence for promotion, and preserved candidate lessons.

The gap appeared inside implemented. That label could point to a blocking check, a lint check, or another instruction. It did not prove that one step stopped the bad case.

The comparison below separates what I observed from the hardening step I would add now.

The memory guard was useful and deliberately advisory. A reminder after the write can help. It is also late. The guard could request an enforcer, but it could not prove one was installed.

The frozen procedure did include a real instruction to undo the code change. The missing piece was a tested rule version, a named consumer, and a safe state when no prior version exists.

Written instructions may still help. This inspection did not measure real hook runs or a production failure rate. The record did not distinguish influence from a hard block.

That makes the next question concrete. Once a rule is approved, where must it run to change the outcome?

Where must the approved rule run?

The zero-source check belongs before the summary is accepted or published. A later check can report the problem. It cannot prevent the publication.

The runtime consumer is the named checkpoint that reads the active rule. It checks the action, input, and time, then decides before work continues.

Current Claude Code hook documentation gives one vendor-specific example. A PreToolUse hook can block a call. A PostToolUse hook runs after success and cannot undo the completed call. Other systems expose different checkpoints, so find the matching decision point in your own workflow.

A written instruction may guide the AI. A later reminder may record a problem. Neither proves a stop happened before the action.

A receipt is the saved record of one workflow decision. A reviewer needs enough detail to reconstruct that decision without trusting the final AI message.

One receipt proves one recorded consumer decision. It does not prove the rule is correct or that every workflow path uses it.

The team now needs a harder test. Did the checkpoint stop the known failure, preserve the valid case, and leave a safe way out?

How do you prove the rule runs and can be reversed?

I built a repeatable local fixture with made-up data. It used no production logs, customer data, network calls, AI provider calls, or outside actions.

Three frozen events shared one input and one error: the input had zero sources, and the output contained an empty sources section. A local validator served as the named checkpoint.

The first test had to be red. It expected a block, but the unguarded workflow allowed the bad synthetic output once.

After approval, the same input had to stop while the one-source near-match passed. The fixture also checked missing data, review time, and rollback.

The table shows the observed sequence. It records local control-flow decisions, not a claim that an AI model learned.

An independent checklist passed all 12 expected cases. Fourteen wider checks covered admission, receipts, review, rollback, file identity, and privacy. A deliberate BLOCK to ALLOW change was caught. Two fresh runs produced the same evidence file, byte for byte.

Those figures are integrity checks for one local artifact. They are not performance metrics.

The machine did not have an epiphany. A validator changed a decision. It blocked the known synthetic bad case, allowed the near-match, and failed safe during review and rollback.

The fixture met its bounded contract. A real rollout still has several unanswered questions.

What does the synthetic proof still not establish?

The local result is useful because its boundary is clear. Before moving it into production, a team still has seven questions to answer.

The safe reading is precise. A local validator blocked one known synthetic bad case, allowed one near-match, and stopped safely at review and rollback boundaries.

The model did not train itself. The test did not prove production readiness, general reliability, or the absence of every valid case the rule might block.

With those limits visible, a team can build the smallest useful version in one workflow.

How can you add one tested rule without building a giant policy system?

Choose one consequential but bounded failure that you can recreate with synthetic or safely redacted inputs. Do not begin with customer data, credentials, payments, account state, or outside actions.

The checklist below keeps the first implementation small while preserving the full decision.

Current guidance supports parts of this method. OpenAI recommends workflow-specific evaluation and shows how weak or strict coding tests can mislead. My two-sided test is an inference, not its standard.

NIST's voluntary AI Risk Management Framework supports ownership, review, recovery, and deactivation. It is being revised and does not require my rule record.

Google's site reliability guidance supplies an analogy: give corrective work an owner and a verifiable end state.