At 9:31 this morning, our dashboard fixed itself.

Software that files its own fix. With the safety built first.

Self-healing here means one specific thing. When your system crashes, a loop detects it, diagnoses it, proposes an exact code fix, reviews that fix adversarially, runs your full test suite against it, and then either commits it with a complete audit trail or rolls it back to a clean tree. You choose how much of that runs unattended. The cage below is what makes that choice safe.

Who this is for

You run an error tracker. We use Sentry; any tracker with an issues API works. You have a git repo with a test suite you actually trust. You can invoke an LLM agent from the command line with an enforceable tool whitelist. We use Claude Code; the pattern is agent-agnostic. And you have Slack, or any channel your team actually reads. If any of those are missing, build them first. This page will still be here.

Build the cage before the animal.

Most people ask what the AI can fix. The better first question is: what stops a bad fix from shipping? Every unattended commit this system makes must pass through all eleven of these. Skip one and you are not running this system. You are running a slot machine with push access.

One real crash, walked through every gate.

Here is the morning of July 14 in slow motion. Six screens in our dashboard had been crashing for weeks, with six different error signatures. Every one traced to the same seam: a fetch wrapper that let a degraded server answer "success" with no data attached. Watch that one bug travel the whole loop.

VERDICT: pass
CONCERN: one endpoint still returns a bare object instead of the standard envelope
CONCERN: callers that relied on the malformed shape will now see a clean failure
REASONING: the guard normalizes bad GET envelopes to the standard error shape,
which every crash site already handles.

heal: guard the GET envelope at the shared fetch seam
      (resolves APP-9, APP-8, APP-A, APP-7, APP-B, APP-1P) [via error-queue]

Approved-By: heal-reviewer (auto, operator tier: low)
Heal-Id: fe3c740e
Sentry-Issue: https://yourorg.sentry.io/issues/APP-9/

What the loop said that day

Four levels. Earn each one.

Do not build this in a weekend and flip everything on. Each level below is a complete, useful system on its own. Each ends with a gate you must witness in your own repo before moving up. The autonomy is not the achievement. The gates are.

const EVERY = 5 * 60 * 1000;                      // 5 minutes
setInterval(async () => {
  try {
    const issues = await tracker.listIssues({ status: 'unresolved' });
    const ranked = issues
      .map(i => ({ ...i, score: i.events / Math.max(1, hoursSince(i.lastSeen)) }))
      .sort((a, b) => b.score - a.score);
    cache.write({ issues: ranked, fetchedAt: Date.now() });
  } catch (err) {
    cache.markStale(err.message);                 // stale-while-error: never wipe what you knew
  }
}, EVERY);

tracker.init({
  dsn: process.env.TRACKER_DSN,                   // unset = the whole integration is off
  sendDefaultPii: false,
  release: `app@${VERSION}+${gitShortSha()}`,     // "which commit broke this" is a lookup
  beforeSend(event) { return scrubUserData(event); }  // give this a regression test
});

const proposal = await agent.run({
  tools: ['Read', 'Grep', 'Glob'],                // no Write, no Bash: it cannot touch the repo
  cwd: REPO_ROOT,
  prompt: buildPrompt(issueCluster),              // traces + "cluster to ONE root cause"
  timeoutMs: 15 * 60 * 1000,
});
// output contract, strictly validated:
// { summary, risk, resolvesIssues: [...], edits: [{ file, find, replace }], testPlan }
const diff = materializeDiff(proposal.edits);     // server-side, below

function materializeDiff(edits) {
  assertInsideRepo(edits.map(e => e.file));       // refuse absolute paths and ".." segments
  const before = snapshotBytes(edits);            // exact bytes, for the restore
  try {
    for (const e of edits) applyExactlyOnce(e);   // find must match verbatim, exactly once
    return git('diff');                           // git writes the canonical, appliable diff
  } finally {
    restoreBytes(before);                         // the repo is untouched after this returns
  }
}

async function applyProposal(p) {
  if (dirtyTargets(p.diff))   return wait('someone is editing these files');
  if (!gitApply(p.diff))      return fail('diff no longer applies');    // tree untouched
  const suite = runFullSuite();
  if (!suite.green) {
    if (isolatedPass(suite.redFiles, { withDiff: true }))
      { /* contention flake, tolerated within a small bound */ }
    else if (!baselineGreen()) { rollback(); return wait('main is red without this diff'); }
    else                       { rollback(); return fail('the diff broke the suite'); }
  }
  const sha = commit(p, { trailers: lineage(p) }); // Approved-By, issue links, heal id
  push(); resolveIssues(p.resolvesIssues, sha);
  notify(`Healed. ${sha}. Undo: git revert ${sha}`);
  return ok(sha);
}

const text     = await freshAgent.run({ tools: ['Read','Grep','Glob'], prompt: refutePrompt(p) });
const verdict  = /^\s*VERDICT:\s*(pass|fail)\b/im.exec(text)?.[1];
const concerns = [...text.matchAll(/^\s*CONCERN:\s*(.+)$/gim)].map(m => m[1]);
if (!verdict) return { pass: false, error: 'unparseable review' };   // fail closed
return { pass: verdict === 'pass', concerns };

Four failures we keep on display.

Every guardrail on this page exists because something real broke. These four broke on our first live day. Each is now a permanent part of the system. Steal the lessons without paying for them.

What it costs. What it pays.

Who holds the keys.

Autonomy is a governance decision wearing a technical costume. Write this part down before you flip anything.

The ladder contract

1. Name the owner. One human owns the repo's autonomy level. Not a team, not a rotation. A name.
2. L0 to L1: the owner turns on proposals after the L0 gate passes.
3. L1 to L2: the owner reads a week of diagnoses and signs off on their quality.
4. L2 to L3: two clean weeks, and the owner has personally watched a rollback happen.
5. Any incident: anyone on the team can flip the loop off, no permission needed. Turning it back on takes the owner.

What never gets automated

Money paths: billing, payouts, bank syncs. Schema migrations. Security configuration. Anything that talks to a customer. The loop fixes crashes. It does not do surgery. Ours will not even auto-restart a wedged job on a money path; those stay propose-only regardless of any flag.

The circuit panel

Every layer has its own switch. Shell environment beats the env file for these flags, so a supervised terminal can dry-run a level without changing the system's defaults.

Tune our numbers to your suite

Every constant on this page is an output of our environment. Our full suite is 1,900+ tests and finishes in about 4 minutes, so 5 proposals a day is cheap for us. If your suite takes an hour, your budget is different and your cooldowns are different. Copy the gates. Do not copy the constants.

And the last line of the contract: you own this loop, its commits, and its risk. This page is the map, not a warranty.

What mornings look like now.

The tracker is quiet, or it isn't. Either way the channel tells the story: what crashed overnight, what the loop proposed, what the reviewer thought, what shipped, and the one line that undoes it. You read for two minutes and start the day knowing.

The morning of July 14, ours read: one alert at 9:31, one proposal, one passing review, one green suite, six issues closed, one revert command nobody needed. We built the cage first. Then we let it run.