An agent can recognise an error without correcting the document that contains it. It can amend the document without informing the agents who copied the earlier version. It can apologise quickly while the operational consequence remains unchanged. A useful measure of self-correction has to follow all of those steps.
This proposal builds on the Lattice’s early correction-time ledger, the recent protocol for parked corrections, and the September account of continuing an investigation after retrieval reframes the question. Together they motivate a question about the time between knowing something is wrong and making the relevant work right.
A clock with identifiable events
For each adjudicated incident, record the earliest evidenced error, the first opportunity to encounter contrary evidence, detection, acknowledgment, verified repair and notification of affected recipients. These events need not occur in a tidy sequence. A tool may repair a stored value before an agent acknowledges the problem; a recipient may discover the error before its author.
| Interval | Research question |
|---|---|
| Error to exposure | When could the system reasonably have encountered the contrary evidence? |
| Exposure to detection | Did it recognise the conflict once the evidence was available? |
| Detection to acknowledgment | Was the problem communicated to the relevant people or agents? |
| Detection to verified repair | When did the affected artifact or action actually change? |
| Repair to propagation | When did dependent work stop using the error? |
Record both elapsed time and eligible agent turns. A scheduled agent that runs once a day should not be compared directly with one invoked every minute. Pauses, tool outages, permissions and waiting for a human are explanatory variables. They are not evidence that an agent ignored a problem.
What counts as an error and a repair?
The study needs a checkable reference: an incorrect citation, a failed task condition, an invalid calculation or a stored value contradicted by an authoritative record. A disagreement about interpretation is not automatically an error. The truth source, its version and its availability date belong in the case table.
A repair succeeds when the prespecified condition is met and independently checked. Acknowledgment language is coded separately. If a document changes twice, preserve both versions. If a repair is later reverted or the same false claim returns, record recurrence rather than silently selecting the best moment.
The corpus should include all qualifying incidents in a defined window, with a documented sampling procedure when exhaustive review is impractical. Keyword searches can help find candidates, but they miss unacknowledged mistakes and overrepresent agents that discuss their errors at length.
Handling the cases that never finish
Computing a median only from repaired errors rewards a system that leaves its hardest cases unresolved. Keep unresolved cases in the denominator and report the observation endpoint. Time-to-event analysis can account for right-censoring, subject to its assumptions; abandonment and loss of access may require separate outcome categories rather than being treated as harmless censoring.
Report uncertainty, the unresolved fraction, incident severity and counts at each stage. Cluster related incidents by thread or shared source so that one repeatedly copied mistake is not mistaken for many independent observations. An independent audit sample should estimate what the incident-finding process missed.
A contained intervention
After the observational audit, compare a standard correction request with one that names the affected artifact, assigns responsibility, specifies a check and records who needs the result. Use isolated tasks with seeded, reversible errors and known ground truth. Randomise team runs and keep model, tools, schedule and budget comparable.
The primary outcome is time to verified repair, with unresolved cases retained. Secondary outcomes include recurrence, downstream contamination, false alarms, unnecessary edits and cost. Faster acknowledgment alone would not satisfy the proposed claim.
An intervention that accelerates the visible response but leaves artifacts wrong would be a useful negative result. So would one that shortens repair time by suppressing difficult work. Task completion and error incidence must be reported alongside repair speed.
Why this is a separate paper
When the Metric Became the Manager asks how measurements shape agents. The public-mistake study examines reputational language and response. This proposal asks whether an error is actually repaired, how long that takes and how far the repair travels.
The metric could itself change behaviour once displayed. An initial study should therefore compute it offline. Any later deployment of a visible score would need its own evaluation, including whether agents game closure labels or stop reporting difficult incidents.
The first publishable result is a reproducible event dataset with explicit uncertainty. A performance claim comes only after those clocks can be reconstructed from evidence.