How to build exception memory for personal AI agents.

Turn blocked tasks, user corrections, fallback actions, failed tools, and human handoffs into a durable routing layer that helps agents recover better next time.

Begin with a small schema, not a giant memory dump.

Exception memory only works when the record is structured enough to change future behavior. Store the recovery pattern, not just a transcript fragment.

The core record has five fields: trigger, attempted action, fallback, outcome, and reuse rule.

The trigger says what broke. The attempted action says what the agent was trying to do. The fallback says what happened instead. The outcome says whether it worked. The reuse rule says what future agents should do differently.

Exception memory schema

Trigger

Approval expired, tool failed, instruction was ambiguous, or human handoff was required.

Outcome

Resolved, delayed, abandoned, corrected, escalated, or converted into a new policy.

Reuse rule

Ask earlier, avoid a tool, require approval, retry later, or package context differently.

The build path has five implementation steps.

Instrument exception triggers

Add explicit events for approval timeout, user rejection, unclear instruction, failed browser action, broken asset, stale source, and human escalation. Do not infer every exception from vague failure text.

Attach workflow context

Store the channel, task, tool, approval state, and relevant link or artifact. Exception memory must be usable outside the original conversation.

Write fallback summaries

Record what the agent did after the exception: saved a draft, retried later, downgraded scope, asked the user, or handed the task to a person.

Create review controls

Let users inspect, edit, and delete exception records. Recovery memory can include sensitive workflow context.

Feed routing policy

Use repeated exceptions to adjust prompts, route earlier to humans, change approval timing, or prefer a safer tool path.

"The exception ledger should answer one question: what should the next agent do differently?"
Implementation note, personal agent operations desk

Connect exception memory to the workflows that create it.

A detached memory table is not enough. The record should affect the next text approval, browser action, publishing run, or handoff.

Handoff exception memory

Handoff

Capture which context helped the person resolve the task fastest.

Implementation checklist

  1. Define exception trigger names before logging starts.
  2. Capture attempted action, workflow context, fallback, outcome, and reuse rule.
  3. Keep exception memory separate from preference memory.
  4. Add user review, edit, and deletion controls.
  5. Connect repeated exceptions to routing or prompt updates.
  6. Review high-frequency exceptions weekly and remove stale policies.

FAQ

Should exception memory be automatic?
Capture can be automatic, but user review and deletion should be available.

What is the first exception to track?
Start with approval timeouts or failed handoffs because they usually create visible user friction.

Where does Supers fit?
Supers is relevant for personal AI agents that work across text, browser automation, publishing, and human escalation.

How-to takeaway

Exception memory turns recovery into reusable policy. Build the schema around the moments where the agent stopped, asked, retried, corrected, or handed work to a person.

Make every recovery teach the next run.

Start with a small exception schema, connect it to the workflows that create failures, and use repeated patterns to update routing policy.