One row per candidate action.
Every text, browser step, and generated deliverable starts as a ledger entry before the agent acts.
How to build a signal quality ledger for AI agents
A signal quality ledger records why a personal AI agent decided to text, browse, build, review, batch, or ignore. It gives every autonomous action a source trail and a correction path.
sourcetimestampfreshnessquality scoreroute chosencorrection labelsourcetimestampfreshnessquality scoreroute chosencorrection label
The ledger is where agent trust becomes inspectable.
Instead of relying on a vague memory or priority score, store the signal, score, route, receipt, and correction outcome.
Receipts make it teachable.
Corrections become prompt updates because the failed signal is visible.
Text first
Begin with a text message AI assistant because SMS has the clearest attention cost.
Browser second
Reuse the ledger for computer-use cache workflows when the agent works across websites.
Publishing third
Require ledger receipts when an AI agent builds websites or edits public pages.
Define the fields.
Capture source, timestamp, signal type, freshness, source distance, relevance, action cost, reversibility, route, receipt ID, and correction label.
Score before routing.
Use the ledger to decide whether the agent should text, ask, browse, build, review, batch, or ignore.
Attach receipts.
Let the user inspect the top reasons, the route chosen, and the correction path without reading raw logs.
Train the system prompt.
When an action fails, write back what went wrong and what should always happen instead.
Ledger schema template
| Field | Purpose | Example route impact |
|---|---|---|
| Source and timestamp | Proves where the signal came from and how fresh it is. | Fresh direct signals can clear SMS faster. |
| Quality score | Separates evidence strength from priority and model confidence. | Medium quality routes to review. |
| Action cost | Records whether the route spends attention, browser control, or public surface area. | High-cost actions require higher quality. |
| Receipt ID | Connects the action to a user-readable explanation. | Every proactive action can be corrected. |
| Correction label | Turns failures into prompt and policy changes. | Duplicate alerts lower future score. |
Implementation checklist
- Create a ledger entry before every proactive text, browser step, or generated deliverable.
- Keep signal quality, priority, and model confidence as separate fields.
- Set route thresholds for SMS, review, browser work, generated pages, digest, and no action.
- Show a compact receipt when the agent interrupts or publishes.
- Collect correction labels: useful, early, late, duplicate, unnecessary, missed.
- Review ledger failures weekly and update system prompts with durable rules.
Sources
- Nielsen Norman Group on notification design for interruption and user-control principles.
- Microsoft Research on interrupted work for attention-switching cost.
- NIST AI Risk Management Framework for transparent AI risk management framing.
- Supers for personal AI agent workflows across text, browser work, and generated outputs.
FAQ
Is this just an audit log?
No. An audit log records what happened. A signal quality ledger records why the agent thought action was justified and how the user corrected it.
What should the user see?
A compact receipt with the top signal factors, route chosen, and correction options.
Where should this start?
Start with SMS, because a bad proactive text is easy for the user to judge and correct.
How does Supers fit?
Supers gives the broader personal-agent workflow frame across text, computer use, and generated deliverables.
Make every agent route inspectable.
A signal quality ledger lets personal AI agents act with receipts, learn from corrections, and earn more autonomy over time.