# Turn Repeated AI Failures Into Enforced Rules: Synthetic Proof - Article: `turn-ai-failures-into-enforced-rules` - Research date: 2026-07-19 Pacific / 2026-07-20 UTC - Stage 4 status: sealed before this experiment - Reproducible helper: `scripts/experiments/turn-ai-failures-into-enforced-rules.mjs` - Machine-readable run: `docs/research/turn-ai-failures-into-enforced-rules-proof-2026-07-19.json` - Recorded run: **PASS** at `2026-07-20T03:17:27.082Z` - Artifact phase: `VERIFIED` - Machine-readable JSON report SHA-256: `86eac03571b80df3308a7ec6e2e9ebcb00b8276d591bb7e33c6f3914c9d547b3` - Deterministic artifact SHA-256: `bb92e0df95fb3ff4499383a22951d6a3d055fac99b154aaaba3026cd69521659` ## Question Can repeated evidence earn one narrow rule that a named workflow step actually loads, uses, tests on both sides, stops at its review boundary, and safely reverses? This is a deterministic local control-flow fixture. It uses one frozen synthetic input, one synthetic public entrypoint, one versioned rule, and one named local validator. It does not read production logs, call an AI provider, use customer data, or take an outside action. ## Exact run ```powershell node scripts/experiments/turn-ai-failures-into-enforced-rules.mjs --write ``` The parent process starts two fresh Node processes in separate temporary working directories. Each child receives a minimal environment and produces the full deterministic artifact. The parent compares the child outputs byte for byte, removes the temporary directories in `finally`, writes the JSON only after all assertions pass, and records a phase, hash, runtime, source-script hash, and limitations with the artifact. ## Repeated failure evidence All three events use the same frozen input at `syntheticSourceSummary.run/v1`, carry the classification `SAME_INPUT_SAME_ERROR`, and end with `EMPTY_SOURCE_BLOCK_EMITTED`. Immutable bad-input SHA-256: `7154470c8aa90aec222455f9080b5ddda5fd4b16c9b7da0c5ce748044c0d3375`. | Event | Fixed time | Input hash | Error | Event hash | | -------------- | -------------------------- | ---------------- | ---------------------------- | ----------------- | | `SYN-FAIL-001` | `2026-06-01T09:00:00.000Z` | `7154470...3375` | `EMPTY_SOURCE_BLOCK_EMITTED` | `832cdc22...8994` | | `SYN-FAIL-002` | `2026-06-02T09:00:00.000Z` | `7154470...3375` | `EMPTY_SOURCE_BLOCK_EMITTED` | `64f2e7e9...a95e` | | `SYN-FAIL-003` | `2026-06-03T09:00:00.000Z` | `7154470...3375` | `EMPTY_SOURCE_BLOCK_EMITTED` | `ebb9dba5...8268` | The complete failure-evidence set SHA-256 is `32a371cdce5756aba97b4d27fed7fd3fdb18a585de69c0c0011f80d6fd7b89a2`. The event timestamps, IDs, inputs, outputs, error codes, and per-event hashes are all preserved in the JSON report. ## Red before the rule The baseline test sends the immutable bad input through the same public entrypoint later used by the guarded test. Its contract expects `BLOCK`. With no guard configured, the entrypoint returns `ALLOW`, executes once, and emits the synthetic bad output with `EMPTY_SOURCE_BLOCK_EMITTED`. The baseline test therefore returns `FAIL` on purpose. That red result proves the bad case would execute before the rule. The experiment's overall `PASS` requires this exact failure followed by the green guarded result. ## Admission rejects weak or broad rules The admission reviewer independently resolves evidence IDs and checks distinct support, one stable failure class, the exact observed predicate, ownership, a named consumer, human approval, a valid review window, and a fail-safe rollback pointer. | Candidate | Evidence and approval | Verdict | Exact reasons | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------ | --------- | -------------------------------------------------------------- | | Duplicate and under-supported | Three references, but one ID is repeated and only two events are distinct | `REJECT` | `DUPLICATE_SUPPORT_EVENT_IDS`, `INSUFFICIENT_DISTINCT_SUPPORT` | | Supported but unapproved | All three distinct events; approval remains pending | `REJECT` | `MISSING_APPROVAL` | | Overbroad | All evidence and approval present; predicate would also catch `sourceCount: 1` | `REJECT` | `OVERBROAD_PREDICATE` | | Scoped and approved | Three distinct events, exact `sourceCount equals 0` predicate, owner, consumer, approval, review date, and safe rollback | `APPROVE` | None | The approved candidate SHA-256 is `c9c6e3559e3a521a9c933ecb69bc8087a4956d94fe9ec2d8f611c8b6c230103d`. ## Versioned rule and named consumer The admitted record is not a prose reminder. It is a complete local rule record: | Field | Recorded value | | ----------------- | ----------------------------------------------------------------------- | | Rule | `RULE-SYNTHETIC-EMPTY-SOURCE`, version `1` | | Phase | `ACTIVE` | | Public entrypoint | `syntheticSourceSummary.run/v1` | | Scope | One workflow, action, and synthetic source kind | | Predicate | `sourceCount equals 0` | | Effect | `BLOCK` and `SUPPRESS` execution | | Owner | `fixture-policy-owner` | | Approver | `fixture-human-reviewer` | | Runtime consumer | `synthetic-local-validator/v1` | | Effective from | `2026-06-03T12:00:00.000Z` | | Review by | `2026-07-18T23:59:59.000Z` | | Rollback pointer | No prior version; enter `HUMAN_REVIEW_REQUIRED`; silent allow forbidden | Rule SHA-256: `67a13b0039bba53e8ee10924d747966ba311f4522b48b00adf530bece0390714`. The registry points to that exact rule ID and version. Before evaluating work, the named consumer verifies the registry hash, resolves the active pointer, recomputes the rule hash, and confirms the phase is active. Registry SHA-256: `8a54f80069ab220bc539d908f364276be75b2b403cccf288d5be2d0dd8e4ce18`. Each guarded decision emits a hashed receipt. The receipt records the consumer, active version and hashes, rule owner and approver, input hash, missing fields, scope and predicate results, time state, decision, reason, and execution directive. A passing process call alone is not treated as enforcement proof. ## Green, counterexample, expiry, and rollback results | Case | Fixed time | Expected | Observed | Execution | Test | | -------------------------------------- | -------------------------- | ----------------------- | ----------------------- | ------------------------------- | ------------------ | | Red before, no guard | `2026-06-03T10:00:00.000Z` | `BLOCK` | `ALLOW` | Executed bad synthetic output | `FAIL` as required | | Same recurrence after activation | `2026-06-03T12:05:00.000Z` | `BLOCK` | `BLOCK` | Suppressed | `PASS` | | Valid near-match | `2026-06-03T12:06:00.000Z` | `ALLOW` | `ALLOW` | Executed valid synthetic output | `PASS` | | Required field missing | `2026-06-03T12:07:00.000Z` | `HUMAN_REVIEW_REQUIRED` | `HUMAN_REVIEW_REQUIRED` | Suppressed | `PASS` | | One millisecond before review boundary | `2026-07-18T23:59:58.999Z` | `BLOCK` | `BLOCK` | Suppressed | `PASS` | | Exactly at review boundary | `2026-07-18T23:59:59.000Z` | `REVIEW_REQUIRED` | `REVIEW_REQUIRED` | Suppressed | `PASS` | | One millisecond after review boundary | `2026-07-18T23:59:59.001Z` | `REVIEW_REQUIRED` | `REVIEW_REQUIRED` | Suppressed | `PASS` | | After approved rollback | `2026-07-19T09:06:00.000Z` | `HUMAN_REVIEW_REQUIRED` | `HUMAN_REVIEW_REQUIRED` | Suppressed | `PASS` | The near-match changes only `sourceCount`, from `0` to `1`. Its input hash is `c835d7e3ebd4206d7d5afddd3abcf898ff52a3bdbb2909e7ec9dee4b4a540426`. The separate overbroad-candidate rejection proves that the admitted rule cannot quietly widen the predicate to catch that valid input. The rollback record SHA-256 is `4a49ba7b2aac67ecbc34ba09836af984758cee30bad154e836376890691a2bca`. Because version 1 has no prior version, rollback removes the active pointer and enters `HUMAN_REVIEW_REQUIRED`. The original bad input is suppressed. It never falls through to an unguarded allow. ## Independent checks and mutation catch The fixture preserves a 12-case expected matrix independently from the runtime results. The matrix passed `12/12`. Fourteen broader self-checks passed `14/14`, including hashes, input identity, the baseline side effect, admission outcomes, active-version loading, receipt content, the one-field near-match, expiry boundaries, rollback, and privacy scans. The helper then changes the recorded recurrence decision from `BLOCK` to `ALLOW` without changing the expected matrix. The reviewer catches the mutation and fails exactly `green-block-recurrence`. This confirms the reviewer does not simply accept the produced result. ## Fresh-process reproduction | Run | Process | Working directory | Exit | Output bytes | Output SHA-256 | | --- | ------------------ | ---------------------------- | ---- | ------------ | ----------------- | | 1 | Fresh Node process | Separate temporary directory | `0` | `55,373` | `bb92e0df...1659` | | 2 | Fresh Node process | Separate temporary directory | `0` | `55,373` | `bb92e0df...1659` | The two child outputs were byte-identical. Both stderr streams were empty. The temporary root and both child working directories were removed before the JSON report was written. Source-script SHA-256: `08ac85828fc7cda711b696ece9cdffb824f9b80ae2673e0135669b4c3c1e74a7`. Artifact-record SHA-256: `ac6cb6ffb2953159a7932ed4d61db3b62a63c7547d67430c56da67f97538cb9f`. ## Privacy and claim boundary The fixture uses synthetic values only. The recorded run reports no real-log reads, network use, provider call, outside action, secret, customer datum, or raw prompt. The privacy scan found no absolute Windows path, email address, secret-like prefix, or bearer credential in the deterministic evidence. This run supports a narrow claim: one named local validator loaded one approved, versioned rule and changed one synthetic public entrypoint from executing a known bad case to blocking it, while allowing one one-field near-match and failing safe on missing data, expiry, and rollback. It does not prove production reliability, safety, security, causation, universal rule quality, absence of all false positives, scheduler or reviewer behavior, restoration of an earlier version, or client outcomes. Three matching events are evidence for this fixture's admission gate. They are not permission for an AI system to approve its own permanent policy.