Co-Evolution as a Governance Requirement in Hybrid Systems

Effective governance of hybrid human-AI systems requires treating co-evolution as a core design principle rather than attempting to achieve static or one-time alignment.

Field brief

Product org · Policy lead

A mixed human–AI team with a one-time alignment document

The document is older than the model. No renegotiation path exists.

First move this week

Schedule a renegotiation, not a re-sign. Put a date on the next one.

AI alignment walls Hybrid value drift

Because both humans and advanced AI systems continue to change and improve over time, any governance framework based on fixed rules, static values, or one-time alignment processes will inevitably become misaligned or brittle. Co-evolution must be designed into the governance architecture itself — through continuous feedback mechanisms, periodic renegotiation of objectives, and institutional structures that can evolve alongside the systems they govern.

This shifts the focus from "solving alignment" to "building systems capable of ongoing healthy co-evolution."

Consequences

Many current AI governance proposals assume a relatively stable human value base and a fixed AI system. Both assumptions are likely to break down over long timescales.
Hybrid governance institutions will need new mechanisms for legitimate, ongoing revision of rules and objectives as both humans and AI systems evolve.
The capacity for healthy co-evolution may become a more important safety property than perfect initial alignment.

Outside instrument

Crux

Stake a prior. Name the disagreement. Not a theorem.

Open full

Related primaries

Outside the corpus

Journals, repositories, counters

human–AI hybrid systems. A paper can support a concept. It cannot support a necessity claim. B.6 remains unproved by every card below.

The outside stack for this fiber is thin. Four tight papers beat seven padded ones. Absence is named.

Support · living source

Tight papers that bear on concepts inside B.6. A DOI is not a join.

Hadfield-Menell, D., Dragan, A., Abbeel, P. & Russell, S. (2016). Cooperative inverse reinforcement learning. NeurIPS 2016.
Alignment as a cooperative game, not a one-way value load.
Does not support the necessity claim
Open originalNamed on Art.1, 71
Dafoe, A. et al. (2020). Open problems in cooperative AI. arXiv:2012.08630.
Names multi-agent closure, institutions, and commitment as the open set.
Does not support the necessity claim
Open originalNamed on Art.1, 71

Counter-arguments · refusing pole

One domain counter per cluster, not a repeating method quartet. The mirror reads these as enough to refuse necessity.

Bostrom, N. (2012). The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds Mach. 22, 71.
Counter: orthogonality — intelligence and final goals can come apart; co-evolution is not necessary.
Does not support the necessity claim
Open originalNamed on Art.1, 71
Hubinger, E. et al. (2019). Risks from learned optimization in advanced machine learning systems. arXiv:1906.01820.
Counter: inner objectives can come apart from outer ones even with cooperative training.
Does not support the necessity claim
Open original
Langosco, L. et al. (2022). Goal misgeneralization in deep reinforcement learning. ICML 2022.
Agents pursue the wrong goal off-distribution even when training looked aligned.
Does not support the necessity claim
Open original