What Should AI Need Before It Can Change the Real World?

What changes when an AI agent can act?

Drafting is cheap to undo. Sending, purchasing, publishing, deleting, deploying, or changing a customer record creates a second system whose state may outlive the agent session.

That outside state introduces four separate moments: proposal, approval, execution, and verification. Collapsing them into one approved flag is how drift hides.

The target can change after review. A payload can be regenerated. A price can move. A permission can become broader. A precondition can expire. A retry can repeat an action that already succeeded. The agent can also report success when the downstream system timed out, rejected the request, or accepted only part of it.

The operating rule is simple: approval must match the action. Execution must produce a receipt. The target must prove the postcondition.

Human approval handles judgment. It does not replace authorization enforcement, transaction integrity, idempotency, downstream receipts, or rollback.

Which actions should run automatically, require a person, or stay blocked?

Start with impact and recoverability, then account for data sensitivity, permissions, cost, scope, and whether the result can be verified.

Do not turn REQUIRE_HUMAN into the default for every tool call. That creates a queue of low-context prompts until people click yes on instinct. Automate the safe class with narrow policy. Reserve attention for actions where human judgment changes the risk.

Also keep the decision action-level. Allow email is a capability grant. Send this digest to these three recipients with this subject and body before 5 p.m. is an action.

What should an action-level approval contract contain?

The contract needs enough information to reconstruct what was proposed, what was authorized, what executed, and what happened next.

Canonicalization matters. Two equivalent JSON objects should not produce different fingerprints because their keys were reordered. Two materially different recipient lists should not produce the same fingerprint because the approval display hid the recipients.

Use the significant data a reviewer actually needs. OWASP calls this what you see is what you sign in transaction authorization. The target and amount of a transfer are the classic example. For an AI action, significant data may include recipients, public visibility, revision, environment, price, permissions, or deletion scope.

Recompute the fingerprint at the final control gate. Do not trust the proposal-time hash if the executor rebuilt the payload later.

What does the local mutation experiment prove?

I built a side-effect-free Node.js fixture around one temporary local JSON file. No account, API, customer, deployment, purchase, message, or remote system was touched.

The starting file is an internal-review draft. The proposed action replaces it with an internal-review published record. Its fingerprint covers the target, operation, payload digest, zero-dollar cost ceiling, starting file digest, local environment, and narrow write permission.

The fixture approves that fingerprint, then changes one field: audience becomes public. The changed action is rejected as BLOCKED_ACTION_MISMATCH. The target digest remains unchanged.

The fixture then submits the exact approved action, rechecks the starting digest immediately before execution, writes the file, records a receipt, reads the target independently, verifies the after-digest, restores the original bytes, and verifies the starting digest.

That result proves a narrow mechanism. It does not prove production safety, distributed rollback, credential security, downstream idempotency, concurrent-write handling, or approver identity. Those controls must exist at the real boundary.

The useful part is the failure shape. A one-field material change invalidates the old approval before the write. That is the behavior to plant in a test.

What do current agent tools provide, and what still belongs to you?

Current agent products expose useful approval and permission surfaces. They do not remove the need for your application contract.

The OpenAI Agents SDK can pause a run when a tool call needs approval. Its documented interruption contains the tool name and arguments, and approval is scoped to a specific call ID unless a developer deliberately chooses a sticky decision. The SDK also supports serializing run state across a long approval wait. That is valuable orchestration. Your application still decides which arguments are significant, how they are canonicalized, when approval expires, what downstream credential is used, and what receipt proves success.

Claude Code's current permission model separates allow, ask, and deny rules, with deny taking precedence. Its documentation says the runtime enforces those rules rather than relying on the model. That is a real boundary. A shell permission rule still describes a class of commands. It is not a business approval for one deployment target, revision, cost, and rollback plan.

OWASP's Excessive Agency guidance recommends minimizing tool functionality and permissions, executing in the user's context, requiring human approval for high-impact actions, and enforcing authorization in downstream systems. Its transaction authorization guidance says significant data should be visible to the approver, changed data should invalidate authorization, the final execution gate should recheck authorization, and authorization should be unique and time-limited.

What did frozen WarpOS evidence teach me about approval?

WarpOS had sensible hard ceilings. The frozen autonomy document reserved remote pushes, paid-service signups, production deploys, destructive migrations, and secrets sent to remote systems for an operator decision.

Its external-service gate blocked a descriptor that required approval unless the recorded state was approved or waived. Its approval hook also checked that a release had an approval reference pointing to an approved or waived record before deploy.

Those controls were better than asking the model to be careful. They also exposed the next problem.

The approval hook checked a pointer and state. It did not bind target, operation, payload, cost, permission, preconditions, expiry, receipt, or rollback. Its own header said parse errors failed open, and an environment variable could disable it.

The frozen release records made the evidence problem concrete. RL-20260611-045.yaml says deployed to internal-canary, names an approval reference, timestamp, and actor. Its companion report says preparing, not deployed, no approval recorded, and deployment not performed.

I cannot infer the real deployment state from that disagreement. The YAML may be newer. The report may be stale. The external target may tell a third story. That is precisely the point.

A release record is not a deployment receipt. A rendered report is not a target postcondition. If two internal artifacts disagree, stop pretending a status label settles reality. Query the deployment target, bind the receipt to the action, regenerate or invalidate stale reports, and keep the mismatch visible until reconciled.

What should happen after approval?

Approval is the start of controlled execution, not the end of review.

Unknown is a first-class result. If the network times out after a send request, retrying blindly can send twice. The next action is not try again. It is query by idempotency key or receipt, determine whether the first action happened, then decide.

How do you implement the first exact-action gate this week?

Pick one reversible action with a real consequence but a contained blast radius. Do not start with money movement, broad deletion, or an irreversible production migration.

Write its canonical schema. Name the significant fields. Create the fingerprint. Display those same fields at approval. Make any material change invalidate the decision. Use a credential that can perform only that operation on that target. Capture the downstream receipt. Query the target. Exercise rollback in a safe environment.

Then plant three failures: change the target after approval, change one significant payload field, and let the approval expire before execution. All three should block before the outside call.