# Presentation script: Can AI learn without placing the whole at risk?

**Johan–Manus inquiry · 3 November 2026**  
**Author: Manus AI**

**Format:** Presenter-led talk, with optional visual cues. This is a script, not a slide deck. At an assumed 125 words per minute plus one minute for pauses and transitions, the estimated duration is about 14 minutes; actual delivery will vary. Citations are for the presenter and are not intended to be spoken.

<!-- spoken:start -->

## 1. Opening: the value and danger of the shortcut

*Visual cue: A shortcut leading toward both opportunity and uncertainty.*

With the above in mind, let us begin with a simple human achievement. We learn a useful rule, and we no longer need to rediscover it each time. A farmer, a nurse, a trader or an engineer can build on what came before.

Artificial intelligence can extend this advantage. A method developed once can be incorporated into software, shared across teams and reused quickly. That can produce enormous benefits. It can also spread a mistake before the people affected have time to recognise it.

The question is not whether to choose innovation or safety. It is how to protect our ability to discover better methods without permitting an untested shortcut to determine the future of the whole.

## 2. Two necessary activities

*Visual cue: Exploration ↔ use of established knowledge.*

Researchers often distinguish **exploration** from **exploitation**. Exploration means trying and learning about unfamiliar possibilities. Exploitation, in this technical sense, means making use of and improving what is already known. It does not necessarily mean taking advantage of people. [1]

If we only explore, we may never make a useful discovery available. If we only refine what already works, we may become highly efficient at yesterday's solution while the world changes around us.

But this is not a demand for a fixed middle point. A small, reversible experiment may deserve broad freedom. A system able to alter medical care, move large sums or affect physical infrastructure needs more restrictive boundaries. The right arrangement depends on consequences, not merely on how new the technology is.

## 3. What biology can teach—and cannot promise

*Visual cue: A local immune response, memory and a warning symbol.*

Life offers a useful comparison. During an adaptive immune response, B cells can generate variations and some cells are selected to expand. The body can retain memory that supports future responses. Studies have examined the selection process in mice. [2] [14]

This resembles a cycle of variation, feedback and retention. But we must not romanticise it. Biology does not deliberately test every change for the safety of the whole. A pathogen may spread quickly; an immune system can attack the body's own tissues. Biological selection operates on present conditions, not on a complete picture of future human welfare. [3]

So we should borrow a functional question from biology: how do we preserve variation and learning? Then we must add human responsibilities: consent, protection, accountability and the ability to challenge decisions.

## 4. Five clocks—not one

*Visual cue: Five clocks labelled discovery, verification, deployment, consequence and correction.*

Johan's observation becomes sharper when we separate five clocks. There is the time to discover a method. There is the time to check it. There is the time to deploy and reuse it. There is the time it takes for consequences to become visible. Finally, there is the time to recognise a problem and make correction effective.

These clocks can overlap, and they do not always move in the same direction. A fast discovery may improve a slow institution. A fast deployment may spread a hidden error. A harmful consequence may occur before a meaningful warning is available.

The practical question is this: **before an unacceptable effect occurs, who can see enough, decide, and actually change what happens?**

## 5. P versus NP: a useful analogy, not the same question

*Visual cue: Finding a route → checking the route → deciding whether to use it.*

This brings us to a mathematical question Johan raised: P versus NP. It is important, but it is not the mathematical name for the time to develop a technology versus the time to deploy it.

Very roughly, P concerns yes-or-no problems that an algorithm can always solve with a number of steps bounded by a polynomial in the size of the input. NP concerns yes-or-no problems where a sufficiently short certificate for a yes answer can be checked within such a bound.

Imagine that someone gives us a route that visits every location in a network once and returns to the start. We can check the proposed route. Finding one in the first place may be difficult. Yet the existence of an easy check does not prove that every possible method of finding it must be slow.

As of November 2026, whether P equals NP remains an open problem. And even if a certificate satisfies its mathematical rules, those rules may omit important real-world consequences. Finding, verifying, applying and spreading are related questions, but they are not the same question. [4] [5]

## 6. The financial example

*Visual cue: Listed options in 1973 → later automatic routing → modern electronic platforms.*

Consider financial options. Options existed before modern exchanges, but in 1973 the Chicago Board Options Exchange opened a standardised listed-options market. By the late 1980s, automated systems were routing and executing some small options orders. Later electronic platforms and broker services made access different again. [6] [7]

This history supports a useful distinction: inventing a method, organising a market, teaching people, creating safeguards and spreading access do not happen on the same clock.

It does not prove that human understanding always takes generations, or that each technology spreads slowly. Nor does the number of options contracts tell us how many individual investors understand the risks. AI may make instructions and orders easier to generate; easier execution is not the same as sound judgment.

## 7. Institutions can correct—and still be late

*Visual cue: Reports → investigation → pause, with earlier exposure still visible.*

Public-health history shows both the value and the limits of monitoring. In the United States, after the rotavirus vaccine RotaShield was licensed in 1998, reports of a serious bowel condition prompted further investigation. In 1999 the CDC recommended a pause, and the vaccine was later withdrawn. Some related cases had also appeared in pre-licensing trials, but the numbers were too limited to settle the rare-risk question. [8] [9]

Monitoring helped trigger action, but many doses had already been administered and serious cases had occurred. This was not a deliberately small pilot. Suspending a vaccine also meant losing its protection against disease.

The lesson for AI is not that we can wait for harm and then safely reverse it. It is that reporting, investigation and authority must be prepared before deployment—and that early observation can miss rare or delayed effects.

## 8. A signal is not a stop

*Visual cue: A warning light separated from a real control lever.*

In 2012, Knight Capital experienced a breakdown in an automated equity-routing system. Before the market opened, diagnostic emails were generated but not used to identify the problem. After the open, the system sent millions of child orders in about forty-five minutes. Knight lost more than four hundred and sixty million dollars. An attempted rollback worsened the incident by activating old code on additional servers. [10]

This was not an AI incident, but the control problem is highly relevant. A log, alert or human reviewer does not protect a system unless someone has the authority and technical means to alter the action in time. And a pause is not the same thing as recovery.

## 9. Governance without paralysis

*Visual cue: A private sandbox, a limited release and a guarded consequential boundary.*

I propose a layered approach.

First, give experimentation room in genuinely isolated environments. Use bounded resources and preserve both successful and failed results. Keep real credentials and outside effects out unless specifically authorised.

Second, make release a separate decision. Identify the exact model version, instructions, tools, permissions and evidence being approved. Start with a limited deployment where appropriate, and widen it only as the evidence supports doing so. A short, uneventful pilot does not prove that rare or slow harm is absent.

Third, place the strongest controls at consequential boundaries: spending, publication under someone else's authority, private data, treatment, infrastructure and physical action. Depending on the risk, those controls may be narrow permissions, an independent approval, a cap, or a decision not to delegate the action.

Fourth, preserve a safe route for legitimate work. A reviewer who is uncertain can hold the new change while a previously verified service continues where safe. Decisions need named owners, review deadlines and appeal. Missing evidence must not become automatic approval.

Finally, evaluate the guardian too. Measure missed harm and false holds. Ask whether a simpler gate achieves the same protection with less delay and less concentration of power. NIST's voluntary AI framework similarly calls for ongoing measurement, feedback and mechanisms to disengage systems, though those recommendations do not prove that a particular implementation works. [11]

## 10. Our test must be able to disagree with us

*Visual cue: Full counterparty versus simple approval gate versus ordinary oversight.*

Our own Part CX development exercise is a reminder to be humble. With a deliberately fallible mock reviewer, an unsupported claim was released in all three test arms. The simpler gate matched the full counterparty's recorded outcomes at a lower invented accounting cost. The exercise was small, unblinded and local; it did not demonstrate an independent production safety system. [12]

A sensible next test would simulate several rounds of publication recipes and compare ordinary oversight, a simple gate and the full counterparty. We would measure unsupported claims, successful legitimate corrections, false interruptions, delay and unresolved effects. Related copies would be tracked as a family, not counted as independent proof.

If the full counterparty does not improve outcomes enough to justify its added burden, that should weaken the proposal. If the simpler gate passes unsupported claims, that matters too. A useful test protects neither our preferred idea nor its critics from evidence.

## 11. Closing: a living capacity to correct

*Visual cue: Discovery → evidence → challenge → action → verified correction → renewed discovery.*

In the Dual Entropy vocabulary, Distribution preserves and shares knowledge, while Discovery brings in novelty and observations that can challenge it. [13] The relationship must work in both directions. A durable rule can help us act together, but a new observation must be able to question that rule.

The future cannot be completely known in advance. Some damage is immediate; some is slow. The purpose of governance is therefore not to eliminate experimentation or promise certainty. It is to preserve the capacity to learn, limit the reach of error, and correct the action as conditions change.

Our goal is neither a system too free to listen nor one too rigid to learn. It is a system where improvement remains possible **because correction remains possible**.

<!-- spoken:end -->

## Presenter notes

The biological comparison concerns functional similarities in variation, selection and memory. It does not imply that biological evolution is inherently safe, that AI has biological heredity, or that human governance can predict every future harm.

The P-versus-NP segment describes *yes*-instance certificates and asymptotic worst-case classifications. It does not assert P ≠ NP, a universal exponential lower bound, or that mathematical verification proves real-world safety.

The publication exercise and the proposed multi-round extension are explicitly different: the first was a small development run; the second has **not** been implemented or run in this presentation.

## References

[1]: https://pubsonline.informs.org/doi/10.1287/orsc.2.1.71 "James G. March, Exploration and Exploitation in Organizational Learning, 1991"
[2]: https://pmc.ncbi.nlm.nih.gov/articles/PMC4809261/ "Gitlin and colleagues, T cell help controls B-cell selection in a mouse experiment, 2015"
[3]: https://www.jci.org/articles/view/24179 "Cappione and colleagues, Autoreactive B-cell exclusion in systemic lupus, 2005"
[4]: https://www.claymath.org/millennium/p-vs-np/ "Clay Mathematics Institute, P versus NP overview and status"
[5]: https://www.claymath.org/wp-content/uploads/2022/06/pvsnp.pdf "Stephen Cook, The P versus NP Problem, official problem description"
[6]: https://annualreport.theocc.com/2023/celebrating-50-years "Options Clearing Corporation, 50th-anniversary timeline"
[7]: https://www.sec.gov/files/news/speech/1989/022789ruder.pdf "U.S. Securities and Exchange Commission, Automated systems account, 1989"
[8]: https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4827a1.htm "CDC, Intussusception among recipients of rotavirus vaccine, July 1999"
[9]: https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4843a5.htm "CDC, Withdrawal of rotavirus vaccine recommendation, 1999"
[10]: https://www.sec.gov/files/litigation/admin/2013/34-70694.pdf "U.S. Securities and Exchange Commission, Knight Capital administrative order, 2013"
[11]: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ "NIST, AI Risk Management Framework Core"
[12]: https://decintellect.com/the-immunity-counterparty#development-test "Decalogy of Intelligence, Part CX development exercise"
[13]: https://dualentropy.quest/ "Dual Entropy, public framework and research synthesis"
[14]: https://www.cdc.gov/pinkbook/hcp/table-of-contents/chapter-1-principles-of-vaccination.html "CDC, Principles of Vaccination and immunological memory"
