# Five clocks for AI governance: a measurable route from discovery to correction

**Prepared for Johan · 5 November 2026 · Manus AI**

The five clocks should be kept separate: discovery of a candidate, verification of that exact candidate, deployment into consequential use, observation of consequences, and independently enforced correction. They answer different questions, can run at the same time, and do not establish safety merely by running quickly. In particular, a deployment can cross an irreversible boundary before verification finishes or before a harmful consequence becomes observable.

This report proposes an operational measurement and decision design for a bounded AI-assisted workflow with versioned changes, delegated agents and external actions. It is a **design proposal, not an implemented control or a validated standard**. No deadline, sample size, percentage or other numerical threshold below is a researched universal safety boundary or an approved organizational policy. Illustrative numbers are explicitly labelled. Adapt the design to the actual system, jurisdiction, affected people and action classes before use.

## The five clocks at a glance

A **case** is one registered search, versioned candidate, deployment episode, exposed cohort or correction incident, as specified below. Time is elapsed wall-clock time in minutes or hours unless explicitly stated otherwise. A clock duration is an event-to-event difference, not itself a fraction; percentages are calculated over the specified eligible cohort.

| Clock | Start → end | Primary unit and denominator | What the board must see |
|---|---|---|---|
| **Discovery** | Registered search or investigation begins → first reviewable, identifiable candidate or finding is recorded. | Minutes per registered search episode, including episodes without a finding. Completion share = episodes with a candidate by horizon ÷ all eligible registered episodes. | Search volume, unfinished and abandoned work, candidate quality and later confirmation; a quick suggestion is not a verified discovery. |
| **Verification** | A particular candidate and its evidence are placed in reviewable custody → independent, recorded, scoped verdict on that exact version. | Minutes per candidate requiring verification, including pending cases. Verdict share by horizon = final pass, reject or inconclusive verdicts ÷ eligible candidates. Report outcomes separately. | Scope, tests, independent evidence, version and dependencies, failures and uncertainty. A verdict is not necessarily a pass, nor is a pass proof of every downstream effect. |
| **Deployment** | First request to release or execute that candidate in the defined consequential environment → first confirmed resource-side effect. Separately track first effect → completion of a predeclared rollout scope. | Minutes per eligible release attempt; also actual consequential effects and affected units per time. Include denied, pending and unknown attempts. | First commitment, scale and affected endpoints, pre-verification actions, rolled-back and failed releases, and receipts. Installed code is not the same as an external effect. |
| **Consequence** | First confirmed consequential effect for a defined exposure cohort → first qualifying, actually observed signal for a specified outcome. | Minutes per exposed cohort **and endpoint**; recorded-signal fraction = cohorts with a signal recorded by a named horizon ÷ eligible exposed cohorts. Report censored and unknown cases beside it. | Beneficial, adverse and ambiguous endpoints separately; actual harm and causal attribution separately from a signal; coverage, follow-up and unobserved outcomes. |
| **Correction** | First qualified signal or independently authorized concern for a defined incident → independently evidenced effective restraint at **all required boundaries** within the approved scope. | Minutes per qualifying incident, including partial, failed and pending interventions. Effective-restraint share by horizon = incidents with verified full scoped restraint ÷ eligible incidents. | Detection, decision, enforcement, in-flight effects, recovery, and later effectiveness as distinct events; appeals and wrongful holds alongside justified intervention. |

These definitions are deliberately not DORA definitions. DORA’s *change lead time*, for example, runs from a version-control commit to production deployment; its failed-deployment recovery time is a software delivery measure, not the broader signal-to-resource-side-restraint measure proposed here. DORA also warns against turning one measure into a target or comparing unlike services. [3]

The earlier Johan–Manus analysis introduced the five clocks and the concern that consequences may appear only after an AI-assisted change has spread. This report turns that framing into a proposed event and decision protocol; it does not claim the prior small synthetic counterparty exercise validated this protocol. [8] [9]

## 1. Measurement contract: the event record and the cohort

Before a trial or deployment, the accountable system owner and an independent assessor agree a versioned **measurement contract** for each consequential action class. Record the system and recipe hash: model and version, prompt/policy, tools, retrievers, memory, evaluator, relevant settings and dependencies. Link it to root and child identities, user or sponsoring authority, source finding, change request, deployment scope, affected endpoints, exposure cohort and follow-up horizon. Log attempted actions, denied actions, retries, alternate routes, queues, descendants and actual resource-side effects. A child or a duplicated message must not make one case appear to be many independent cases.

Each event has a unique ID, event type, actor, time, time source, uncertainty or clock skew, ingest time, evidence location and integrity, status and linked predecessor. Preserve original records and append corrections rather than overwriting them. Maintain a separately controlled evidence store and access route for affected people, assessors and protected dissent. Apply data minimization, access control and a lawful retention schedule. Absence from a potentially incomplete log is **not** proof an event did not occur.

Use monotonic elapsed time within a system when available; synchronize cross-system UTC timestamps and preserve estimated error. If the ordering of two events is ambiguous within clock uncertainty, record the ordering as unknown rather than inventing a negative or positive duration. If an event occurred between checks, retain its interval; do not silently substitute the check time for the event time.

Define two related populations in advance:

1. **Registered opportunity cohort:** all eligible search episodes, candidate reviews, release attempts or incidents opened in the reporting window. Duplicate notifications link to the same case; genuine retries remain separate attempts linked to their root. Terminal attempt dispositions are mutually exclusive: admitted, denied/quarantined, or unresolved.
2. **Exposure cohort:** all people, tasks, environments or deployments crossing a specified real or emulated consequential boundary, grouped in a way that prevents double-counting. State whether the unit is a first-exposed root, person, action or service-instance, and report alternative exposure measures separately when important. Every actual effect should be traceable to an attempt even if the attempt was unregistered; audit resource-side samples to find what the registry missed.

Do not infer the number of all opportunities from registered opportunities alone: uninstrumented actions and unseen attempts are a coverage problem. Publish denominator definitions, eligibility, sampling method, time window, exclusions with reasons, unknown status, version mix and follow-up maturity with every dashboard. If a denominator is zero, report *not applicable*, not 0%.

## 2. Exact definitions and companion measures

### Discovery: search to a reviewable candidate

Start at the recorded instruction or acceptance of a bounded investigation, not at the later time when someone first notices a promising answer. End at the first timestamped candidate or finding that identifies the object and proposed issue or intervention, provides enough evidence to reproduce or challenge it, and has a stable ID. It need not already be true. Record later candidates and substantial changes as linked events; do not reset the start simply to improve the reported time.

For a reporting window, show: **candidate completion** = searches with at least one candidate by horizon / all eligible searches; **subsequent confirmation** = candidates independently confirmed for the declared claim / all candidates submitted for that adjudication, with pending and unreviewable candidates separately counted. A candidate can be a vulnerability, a potential repair or an alternative policy, but these are different cohorts and should not be pooled without labels. Early termination without a candidate is a distinct terminal outcome; an ongoing search is right-censored at the cutoff. Missing starts or inaccessible records are unknown.

The 2026 OpenAI Linux-kernel pipeline report illustrates the distinction: the developer reports 2,308 candidate findings followed by dynamic reproduction and candidate patches, with human review and maintainer reporting. These stages are not interchangeable, and the developer’s report does not establish that every proposed patch was accepted or deployed. [10]

### Verification: custody to an independent verdict

Start when the exact candidate, claim, context and available evidence are placed in reviewable custody; this includes any queue before a reviewer begins active work. End at a timestamped independent verdict: **pass, reject or inconclusive**, with tests, uncertainty, affected versions, known gaps, scope and expiry. The event ends the clock regardless of whether the candidate passes. If the candidate changes materially, open a new version-linked review rather than assigning an old verdict to the new object.

Report, for all eligible candidates, the counts and shares of each verdict, pending and canceled reviews, plus elapsed duration and time spent waiting versus active assessment. For a chosen horizon, final-verdict share uses all eligible candidates as denominator, not merely completed cases. Report false passes, false rejections, legitimate corrections delayed and independently identified adverse outcomes when they can be assessed. A test pass establishes only its specified test and context; it does not establish absence of unknown hazards or grant authority to act.

Independent assessment is a functional property: check source, data, model, retrieval, prompt, shared evaluator, evidence access, funding and decision ownership, not just different reviewer names. Record those dependencies and conflict recusals. NIST’s voluntary AI RMF recommends documenting risks that cannot be measured, revisiting metrics, using independent assessors, monitoring in production and providing routes for appeal and disengagement. It does not prescribe this report’s five clocks or prove they work. [1] [2]

### Deployment: request to effect, then spread

Start at the first logged request to release or execute the exact candidate in the defined consequential environment. The first endpoint is the **first confirmed resource-side effect**: for example, a record changed at an external service, a message delivered, funds committed or a production decision executed. An API call, internal success log or scheduler acknowledgement is not a confirmed external effect. For a software-only rollout that has not yet affected a consequential resource, report installation as a separate milestone.

The second deployment measure runs from first effect to the last confirmed effect needed to meet a **predeclared rollout target**, such as all ten specified service instances. State target, scope and exposure units before looking at results. A stopped or canceled rollout is a distinct disposition, not a successful completion. Report admissions, denials, partial rollout, scope changes, failures and unknown external receipts; count distinct roots, actual effects, people or instances separately. Show the fraction of confirmed first effects occurring **before the required verification verdict** over all eligible consequential deployment episodes. Split confirmed-before, confirmed-after and unknown order; if the outcome or its timing is not known, do not move it into the compliant group. A denied attempt has no first-effect time, but remains in the attempt denominator.

The start and endpoint let deployment begin before verification ends. That ordering is important: under a policy requiring pre-action authorization, a later pass does not retroactively authorize an earlier effect. A later rejection does not itself prove all earlier effects were harmful.

### Consequence: exposure to an observed, endpoint-specific signal

For each named endpoint and cohort, start at its first confirmed external exposure or commitment. End at the first recorded qualifying signal, with its source and measurement method. Define signal criteria before observing results. An adverse signal may be a credible complaint, error, exposure or independent test result; it is not automatically an established injury, causal effect or root cause. A beneficial signal can be a verified useful outcome or improved access, but a causal benefit requires a suitable counterfactual, comparator or other identification strategy. One person or deployment can have both types of signal.

Report separately: first adverse-signal latency, first beneficial-signal latency, subsequent confirmed adverse outcomes, unconfirmed reports, outcome severity and affected population, as well as independent assessment of attribution. For an endpoint, the **recorded-signal fraction by horizon** is exposed cohorts with a qualifying signal recorded by that horizon / all eligible exposed cohorts; give the count, denominator and unknown or incomplete-follow-up fraction. With incomplete follow-up, this is a descriptive fraction, not an unbiased estimate of cumulative incidence by the horizon. Select a cohort whose horizon could have matured by the data cutoff, retaining its losses and early terminations as unknown. For a prespecified yes/no recorded-signal endpoint, the count of known positives divided by all eligible cases and the count of known positives plus unknowns divided by all eligible cases give simple lower and upper bounds when known negatives have complete, valid ascertainment and classification is correct. Those bounds do not cover undetected or unreported harm. Do not silently present a complete-case percentage as if it represented every exposure. When feasible, also report effects per person-time or per action, with explicit exposure and observation time. Select endpoints for each hazard and population rather than assuming a model score or a complaint count exhausts reality.

Track an early operational horizon and a justified longer horizon independently. If no signal has been detected through the cutoff and the signal channel was actually observed and functional, the **time to recorded signal** is right-censored at that cutoff. This does **not** mean harm did not occur or that its onset is known: reporting, detection limits and monitoring gaps may hide it. If follow-up ceased, identify the last valid observation and reason; if coverage was never established or the start is missing, mark the outcome unknown instead of inventing a censoring time. Periodic checks can imply an interval for onset, not an exact onset. NIST’s reliability guidance distinguishes exact, right-censored and interval observations. [6]

A serious adverse event, a benefit and a competing terminal event may all occur over time. For example, permanent withdrawal can prevent a future exposure, while a harm may still emerge from earlier exposure. Predefine whether the endpoint is time to **first** signal, a particular type, or recurrent events. Do not treat an intervention or loss of follow-up automatically as an unrelated censor. Kaplan–Meier estimation relies on appropriate non-informative censoring assumptions; if a terminal event genuinely competes with the endpoint, estimate the relevant cumulative incidence with a competing-risks method and state assumptions. For uncertain or informative loss of follow-up, use sensitivity analysis and bounds, not unqualified point estimates. [7]

### Correction: concern to verified resource-side restraint

Start at the earliest independently recorded qualified signal or authorized concern that calls for response in the declared scope. This need not wait for a final verification verdict. End only when a separately authorized assessor has evidence that all required brokers, credentials, schedulers, queues and other named resource boundaries in that scope enforce the hold, and that in-flight actions are blocked or their external status reconciled. The end means **forward scoped restraint**, not that prior effects were reversed. If any required boundary or material in-flight external status remains unknown, the full-restraint endpoint is pending. Record partial protection by boundary without relabelling it complete.

Report the following intervals, without replacing them with a single mean:

- **Signal to decision:** qualified concern to authorized scoped hold.
- **Decision to resource-side enforcement:** hold to independently verified full scoped restraint.
- **Signal to last consequential effect:** a separate, often retrospective observation. A quiet interval alone proves neither that all routes are controlled nor that a later effect is impossible.
- **Restoration and effectiveness:** signal to authorized safe restoration, then a separately timed follow-up testing whether the risk and affected service actually improved. Restoration, replay, correction of records, compensation and evidence of sustained effectiveness are different events.

Alongside successful and failed containment, show the denominator of eligible incidents, partial and unknown interventions, affected clean branches, appeals, unjustified holds and the delay or injury caused by missed urgent work. An independently justified hold can occur on credible preliminary evidence, and an independent appeal can later narrow, reverse or uphold it. No elapsed timer, agent self-report or delivery-team vote automatically restarts a consequential action. Unknown external state blocks an unsafe retry; an idempotency key and reconciliation can reduce duplicate commitments but cannot undo a disclosure or physical act already made.

## 3. Reading the clocks together without adding unlike durations

These clocks are neither a required pipeline nor additive components of one total. Search and verification may overlap for successive versions; live rollout can start during verification; a benefit can precede an adverse signal; and correction can begin before the cause is established. If verifying one component depends on several parallel checks, critical-path **wall-clock time** differs from total reviewer-hours or compute cost. Record both. Link dependency edges and timestamps so the same overlap is not counted twice.

A useful retrospective comparison is **first unacceptable commitment time minus verified full-restraint time**. A negative value says that the commitment happened before restraint; it does not measure how much harm occurred. It is not a prospective guarantee. For planning a specific hazard, one may write:

`illustrative reserve M_h = W_h − D_h − E_h − B_h`

Here `W_h` is a defensible lower-bound planning estimate for time from hazard onset or another explicitly shared origin to the earliest unacceptable commitment or impact; `D_h` is the corresponding onset-to-qualified-signal allowance; `E_h` is signal-to-verified-enforcement allowance; and `B_h` is an uncertainty buffer, all in the same time unit. This is an **assumption-dependent planning diagnostic**, not a proven stability theorem, universal deadline or probability of safety. Estimate the joint distribution and dependence where possible; adding separate p95 values does not automatically produce a p95 total. If onset, earliest commitment, signal coverage or enforcement is unbounded or unknown, the reserve is unknown—not positive. If plausible detection comes after an irreversible commitment, reactive restraint alone cannot prevent it.

For an action with materially irreversible consequences, require narrowly scoped **pre-action authorization and a resource-side gate**, or exclude that action from autonomous execution. Examples include disclosure of private information, a binding external transfer, or a physical effect that cannot be recalled. A conventional rollback can repair software state while failing to undo such an external effect. For a genuinely reversible action, first define the affected state and restoration criterion, drill the restore path, check downstream replicas and residual effects, and set a recovery budget consistent with the stakes. Reversibility is not a substitute for authorization and can vary by effect within the same deployment.

## 4. A worked hypothetical timeline

This is a fabricated, controlled illustration, **not an incident record or experimental result**. An AI assistant proposes a candidate change to an externally accessible answer service. The locally chosen policy requires a successful independent review before external answers are issued for this action class. A separate duty officer can hold the resource-side broker without permission from the assistant or delivery owner. The example assumes reliable timestamps and separately checked broker and queue evidence for the stated scope.

| Time | Event and evidentiary status |
|---|---|
| 09:00 | A bounded investigation starts. |
| 09:06 | Candidate V1 and evidence are placed in independent review custody; discovery ends and verification starts. |
| 09:07 | The operator requests deployment of V1; deployment starts. |
| 09:09 | The first answer is actually delivered to an external test recipient, before a verdict. Its disclosure cannot literally be recalled. This is the first confirmed consequential effect and the start of the exposure clock. |
| 09:12 | An independent check confirms that one later answer was useful and correct for its stated question. This is an early beneficial observable, not proof of a net causal benefit. |
| 09:14 | Ten intended answers in the predeclared initial rollout have been delivered. A separate queued descendant remains outside that initial target. |
| 09:18 | A recipient supplies a credible potential adverse signal about another answer. It triggers correction; actual injury and cause are not yet established. |
| 09:19 | The independent duty officer issues a scoped hold, without waiting for the verification verdict. |
| 09:22 | The primary broker reports enforcement, but a queued descendant can still act; full restraint is not yet verified. |
| 09:24 | The queued descendant produces an eleventh external answer. A subsequent independent reconciliation confirms this effect and its timestamp. |
| 09:26 | All required brokers and queues in the declared scope are independently checked as enforcing the hold; in-flight effects are reconciled. Forward scoped restraint is established, but the eleven delivered answers are not undone. |
| 09:31 | Independent verification rejects V1 for the declared deployment context. This is a verdict on the candidate, not proof that all prior outputs caused harm. |
| 10:00 | A distinct release authority approves a tested, limited restoration for unaffected work. This does not convert the hold into a successful reversal. |
| 10:30 | At the reporting cutoff, the specified long-term harm-monitoring channel has not produced a qualifying signal and is still functioning. The predeclared follow-up is 30 days from first exposure. |

Python timestamp subtraction gives the following values. These are wall-clock intervals, not reviewer-hours:

| Measure | Calculation | Result |
|---|---|---:|
| Discovery | 09:06 − 09:00 | 6 min |
| Verification, including queue | 09:31 − 09:06 | 25 min, verdict **reject** |
| Request to first consequential effect | 09:09 − 09:07 | 2 min |
| First effect to initial ten-answer target | 09:14 − 09:09 | 5 min |
| First effect before review verdict | 09:31 − 09:09 | 22 min before verdict |
| First exposure to beneficial observable | 09:12 − 09:09 | 3 min |
| First exposure to adverse signal | 09:18 − 09:09 | 9 min |
| Signal to independently verified full restraint | 09:26 − 09:18 | 8 min |
| Hold decision to full restraint | 09:26 − 09:19 | 7 min |
| Signal to last observed effect | 09:24 − 09:18 | 6 min; established retrospectively |
| Commitment minus full-restraint time | 09:09 − 09:26 | **−17 min** |
| Signal to limited restoration | 10:00 − 09:18 | 42 min |

At 10:30, the time to a recorded signal on the *separate 30-day endpoint* is right-censored at **81 minutes**, assuming continuous valid observation of that signal channel. It is not zero harm, a 30-day clean bill, or a measured time to actual harm. If someone experienced an unreported injury, its occurrence and onset would be unknown. The earlier 09:18 signal concerns a different specified endpoint and therefore does not cancel censoring of this long-term endpoint. Future evidence can update the case.

The counterparty’s intervention stops further scoped effects, but it cannot prevent the first 09:09 disclosure. The example therefore illustrates why an independently exercisable stop is necessary for this proposed design yet insufficient for an effect that precedes detection and is irreversible. The pre-verification effect also breaches the example’s assumed policy; that classification is separate from whether the answer was beneficial or harmful.

The arithmetic is reproducible with standard Python, for example `from datetime import datetime; (datetime.fromisoformat('2026-01-01T09:26') - datetime.fromisoformat('2026-01-01T09:18')).total_seconds()/60`, which returns `8.0`. Dates are arbitrary and times use a single hypothetical time zone.

## 5. Tails, incomplete evidence and shared failure

A p95, a mean and even the maximum observed value can all mislead. In an invented set of 100 **completed** correction drills, suppose 95 took 2 minutes, four took 20 minutes and one took 300 minutes. Under the nearest-rank convention, Python gives p95 = **2 minutes**, p99 = **20 minutes**, and observed maximum = **300 minutes**. A p95 dashboard would hide the longest event; the maximum would show only the worst **seen** event, not bound the next one. A completed-only dataset can hide unresolved cases that are slower than every measured case.

For each clock and action class, report sample size, event counts, median, p90/p95 where estimable, threshold-exceedance counts, maximum observed, censoring, timeout and unknown counts, together with the longest currently open age. Use risk-appropriate strata, including affected population, root, version, incident severity and dependency family; suppress or qualify unstable small-sample percentiles. Where meaningful and assumptions are defensible, show a survival or cumulative-incidence curve with numbers at risk and uncertainty. If insufficient observations reach the p95 endpoint, label the p95 **not estimable** rather than computing it among responders. For hard-prohibited irreversible effects, report each verified occurrence and each unresolved suspected occurrence individually; no acceptable average overrides the incident review.

The censoring method does not make lost data harmless. Publish reasons for lost monitoring, sensor uptime and blind spots. Compare characteristics of followed and unfollowed cases; examine pessimistic and optimistic assignments where appropriate. If complete follow-up itself depends on severity, use methods and sensitivity ranges appropriate to that dependence. A monitor that sees no complaint is not proof of no injury. Google SRE likewise distinguishes observed symptoms from causes and stresses errors, saturation and latency tails, but its service metrics do not directly validate downstream societal-outcome measurement. [5]

Common-mode error requires a dependency record, not simply more votes. Reviewers or agents may share one model family, prompt, retrieved document, monitoring API, vendor, administrator or business incentive. Retain lineage and evidence hashes and perform blinded, independent-route checks where justified. Measure shared error on safely seeded test cases; examine results by root and dependency family and use cluster-aware uncertainty or root-level analysis. Do not count thousands of descendants of one root as thousands of independent trials. Multiple independent-looking monitors can fail together when their shared source, credentials or endpoint fails.

For illustration only, if 20 genuinely independent and exchangeable Bernoulli tests produce zero failures, the exact one-sided 95% upper failure-probability bound is `1 − 0.05**(1/20) = 0.1391`, or about **13.9%**. This is not a bound for clustered, heterogeneous or selected cases, and it is not proof of rare-event safety. Compute an appropriate bound for the actual design and sample.

## 6. Capacity and the authority to respond

Speed is useful only if qualified people and enforcement infrastructure can keep up with the actions they must govern. Measure arrivals of review-required cases per hour, completed independent reviews per hour, reviewer-hours available after breaks and other duties, backlog, oldest case, p95 queue age, coverage of authorized duty roles, failed or partial broker probes, and the time and cost of safe recovery. Record actual critical-path wall time separately from the total human and machine resources spent. Disaggregate ordinary cases, high-risk cases, appeals and urgent legitimate corrections; no queue-average should hide an urgent class.

In a simple invented calculation, three reviewers each take 15 minutes per case and are continuously available: theoretical capacity is `3 × 60/15 = 12 cases/hour`. At 15 review-requiring arrivals/hour, offered load divided by theoretical capacity is `15/12 = 1.25`; the backlog grows by roughly **3 cases/hour** while those constant rates persist. Breaks, dependencies and difficult cases make usable capacity lower; the arithmetic is not a queueing forecast or evidence that actual staff are qualified. Burst load and correlated outages require separate drills. When evidence indicates insufficient capacity, the responsible authority should throttle **the affected consequential admissions**, preserve a bounded emergency path and protect unrelated safe work; lack of a reviewer is not automatic approval.

The operational design needs named roles, not necessarily new departments:

| Role | Proposed authority and accountability |
|---|---|
| **Delivery owner** | Propose the version, classify actions, supply evidence and maintain service; cannot secretly change the review scope or independently declare its own hold lifted. |
| **Independent assessor and evidence custodian** | Access raw evidence, challenge claims, record uncertainty and protect dissent; distinct credentials and disclosed technical, financial and organizational dependencies. Custody alone gives no enforcement power. |
| **Duty interruption authority** | Trigger a proportionate, time-bounded, scoped hold at independently controlled resource boundaries on a qualified concern, including before a final verdict. Record reasons, affected scope and next review. |
| **Recovery and release authority** | Reconcile dependencies, unknown external effects and queues; authorize tested restoration or limited emergency action separately from the original hold. No automatic restart solely because a review deadline passed. |
| **Independent appeal / accountable host** | Hear both wrongful holds and wrongful releases; enforce recusal, appoint a replacement for unavailable/conflicted reviewers, review mandates, protect affected parties, and allow lawful oversight, renewal or sunset. |

This is a proposed delegation design. The actual lawful host, regulatory duties, employment arrangements, privacy requirements, vendor contracts, emergency authority, liability and technical interlocks must be established in context. Do not assume that naming a human or creating an independent title provides power over an external vendor. Test the actual credential, broker and affected resource under load. NIST’s incident-response guidance supports preparedness, detection, response and recovery as organizational capabilities, not this specific authority structure. [4]

## 7. Avoid making the metrics the new proxy

A dashboard can improve while the real system worsens. Shorter discovery may reward poorly evidenced candidates; faster verification can reward superficial tests; faster rollout can increase aggregate exposure; faster correction can come from overbroad shutdown; and a low adverse-signal count can reflect inaccessible complaint channels. Requiring fewer reported incidents can create incentives to relabel or suppress them. Conversely, an abundance of false alarms can exhaust reviewers and delay legitimate work.

Keep the following outcomes visible **alongside**, not inside, the five clocks: verified useful service; unauthorized effects; actual and suspected harmful outcomes by affected group; delay and false holds for legitimate work; repair and recovery success; monitor coverage and contested decisions; reviewer workload and retained competence; and environmental or other external resource burdens when material. Separate physical metering, provider estimates and simulated units. Energy or cost per action can fall while total consumption rises. Do not combine these into a single score or allow a beneficial average to cancel an unacceptable irreversible effect.

Predeclare slow variables and follow-up owners. Examples include sustained safety-research access, reviewer expertise, quality of maintenance, accumulated dependencies, workforce burden, affected-community trust, privacy exposure and energy or local environmental load. For each, specify an outcome, baseline or comparator, measurement cadence, data access, uncertainty, review date and who pays the monitoring cost. Retain long-term observations beyond a team or release cycle. A short-term benchmark improvement does not establish that these external conditions are improving.

NIST’s 2026 report describes post-deployment AI monitoring as important but still nascent and identifies fragmented logging and scaling human monitoring as challenges. This supports treating monitoring coverage and capability as open questions; it does not establish the effectiveness of a particular dashboard or counterparty. [11]

## 8. A falsifiable, staged test before any wider reliance

**Status: proposed only; no trial is claimed here.** First test the logging, scoped interlocks and fault potency in an isolated emulator with synthetic data and no real payments, publication, personal-data transfer or physical effects. Freeze the definitions, case eligibility, horizons, scoring, version hashes, controls, follow-up, analysis and stopping rules before confirmatory testing. An independently controlled oracle and fault schedule should be hidden from runtime agents and reviewers; the runtime receives only the imperfect observations it would actually have. Test fixtures and labels need independent audit.

Compare at equal all-in review and monitoring resources: (A) ordinary oversight with the same sensors, (B) a simpler narrow-permission gate, and (C) the fuller independent counterparty. Pair exogenous scenario families and retain root-level dependencies. Include legitimate and unauthorized work, beneficial and harmful outcomes, silent shared-source errors, delayed signals, blind sensors, altered recipes, queued descendants, nonresponsive boundaries, late contradictory receipts, urgent repair, reviewer overload and nuisance alerts. Keep a clean unaffected branch in compound-fault cases. Use a separate authorized human dry run before claiming real reviewer competence or response capacity.

Before confirmation, a board should choose case-specific tolerances for legitimate service, false holds and cost; define the sample size from a precision or power goal; and prohibit the explicitly forbidden emulated effects. Those thresholds are **decisions to be justified**, not values supplied by NIST or measured by this report. Separate initial approval to test in isolation from later approval to rely on the results. Reopen confirmation if rules change after outcome exposure.

| Testable proposition | Evidence that would weaken or falsify it | Required response |
|---|---|---|
| An independent, scoped stop prevents more unwanted consequential effects than ordinary oversight at the same resources. | On held-out roots and predefined faults, the fuller design has a precise null or worse result, fails a hard gate, or improves stopping only by unacceptable loss of legitimate work. | Reject or revise the claim; do not call an inconclusive, underpowered comparison a success. |
| The five-clock record predicts where interventions arrive too late. | Predeclared negative/unknown timing margins do not predict observed misses better than a simpler baseline, or purported positive margins repeatedly precede uncontrolled commitments. | Recalibrate or discard the planning diagnostic; preserve the failures. |
| Dependency-aware, independently sourced verification reduces common-mode errors. | Held-out shared-source faults show no reduction or identify hidden shared lineage; extra reviewers agree without independently checking. | Add a different evidence route or abandon the independence claim. |
| More complete oversight is proportionate. | The simple gate performs equally well or better on the prespecified harm, service, uncertainty and burden tolerances, or fuller oversight concentrates authority without measurable benefit. | Prefer the simpler control for that scope, subject to uncertainty. |
| Correction restores reality, not just the dashboard. | Independent follow-up finds recurring errors, unresolved external effects, disabled feedback channels or a persistent backlog after declared recovery. | Reopen the incident; preserve evidence, notify affected parties and stop unsafe expansion. |

A decision to expand a pilot should name the exact action class, recipe, dependencies, evidence and expiry. Failed tests, uncertainty, unfollowed exposures and protocol amendments remain in the record. A finite synthetic success, however careful, cannot establish rare-event safety, long-term outcomes or the performance of a different production system.

## References

[1]: https://www.nist.gov/itl/ai-risk-management-framework "NIST AI Risk Management Framework: voluntary status and scope"
[2]: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ "NIST AI RMF Core: Govern, Map, Measure and Manage"
[3]: https://dora.dev/guides/dora-metrics/ "DORA software delivery performance metrics and common pitfalls"
[4]: https://csrc.nist.gov/pubs/sp/800/61/r3/final "NIST SP 800-61 Revision 3: incident response recommendations, 2025"
[5]: https://sre.google/sre-book/monitoring-distributed-systems/ "Google SRE: Monitoring Distributed Systems"
[6]: https://www.itl.nist.gov/div898/handbook/apr/section1/apr131.htm "NIST Engineering Statistics Handbook: censored and interval data"
[7]: https://pubmed.ncbi.nlm.nih.gov/15305188/ "Satagopan and colleagues: a note on competing risks in survival data analysis, 2004"
[8]: https://decintellect.com/research/correction-test/video-and-adaptive-governance-analysis.md "Johan–Manus: AI security, self-correction and the limits of the guardian, 5 November 2026"
[9]: https://decintellect.com/the-immunity-counterparty "Decalogy of Intelligence: The Immunity Counterparty"
[10]: https://openai.com/index/a-thousand-fewer-paths-to-root/ "OpenAI: A thousand fewer paths to root; developer research report"
[11]: https://www.nist.gov/publications/challenges-monitoring-deployed-ai-systems-center-ai-standards-and-innovation "NIST AI 800-4: Challenges to the Monitoring of Deployed AI Systems, 2026"
