Skip to content
Manifold
← Observatory
Th.35F Hybrid era

Coherence in Hybrid Human-AI Systems Theorem

Hybrid systems require new mechanisms to maintain coherence across human and artificial agents.

Cycle path

Art.8→Art.1→Art.78→Art.101→Art.8

As human and artificial intelligence systems become increasingly integrated, maintaining coherence across these hybrid collectives becomes both critically important and structurally challenging.

Hybrid systems introduce new complexities in Mirror-like reflection, resolution processes, and the distribution of agency. Traditional mechanisms of coherence that worked within purely human or purely artificial systems may not be sufficient. New forms of interface, feedback, and mutual modeling will be required to prevent fragmentation or misalignment between human and artificial components of these emerging hybrid intelligences.

Consequences that now become visible: • Many current challenges in AI alignment can be understood as problems of maintaining coherence across hybrid human-AI systems. • The design of interfaces and feedback mechanisms between humans and AI will significantly influence whether hybrid systems trend toward greater coherence or increasing fragmentation and conflict. • Advanced hybrid systems may develop emergent properties (both beneficial and risky) that cannot be predicted from studying humans and AI in isolation. • The ethical and governance challenges of hybrid systems are likely to be among the most important issues of the coming decades. • Successful hybrid coherence will likely require deliberate cultivation of new cultural, institutional, and technological practices specifically designed for mixed human-AI collectives.

Outside instrument

Crux

Stake a prior. Name the disagreement. Not a theorem.

Open full

Idea Series

Article-base cycle tokens and Atlas-named hubs. Art.n is not Th.n.

Write a withdrawal condition

Outside the corpus

Journals, repositories, counters

human–AI hybrid systems · multi-agent alignment. A paper can support a concept. It cannot support a necessity claim. Th.35 remains unproved by every card below.

Support · living source

Tight papers that bear on concepts inside Th.35. A DOI is not a join.

  • Hadfield-Menell, D., Dragan, A., Abbeel, P. & Russell, S. (2016). Cooperative inverse reinforcement learning. NeurIPS 2016.

    Alignment as a cooperative game, not a one-way value load.

    Does not support the necessity claim

    Open originalNamed on Art.1, 71

  • Dafoe, A. et al. (2020). Open problems in cooperative AI. arXiv:2012.08630.

    Names multi-agent closure, institutions, and commitment as the open set.

    Does not support the necessity claim

    Open originalNamed on Art.1, 71

  • Axelrod, R. & Hamilton, W. D. (1981). The evolution of cooperation. Science 211, 1390.

    Reciprocity as a stable multi-agent attractor under shadow of the future.

    Does not support the necessity claim

    Open originalNamed on Art.1

  • Christiano, P. et al. (2017). Deep reinforcement learning from human preferences. NeurIPS 2017.

    RLHF as the single-agent patch the Mirror Principle says will not close.

    Does not support the necessity claim

    Open originalNamed on Art.1, 71

  • Yudkowsky, E. (2008). Artificial intelligence as a positive and negative factor in global risk. In Global Catastrophic Risks, OUP.

    Named on Art.1. Capability without closure as an existential factor.

    Does not support the necessity claim

    Open originalNamed on Art.1

Counter-arguments · refusing pole

One domain counter per cluster, not a repeating method quartet. The mirror reads these as enough to refuse necessity.

  • Bostrom, N. (2012). The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds Mach. 22, 71.

    Counter: orthogonality — intelligence and final goals can come apart; co-evolution is not necessary.

    Does not support the necessity claim

    Open originalNamed on Art.1, 71

  • Hubinger, E. et al. (2019). Risks from learned optimization in advanced machine learning systems. arXiv:1906.01820.

    Counter: inner objectives can come apart from outer ones even with cooperative training.

    Does not support the necessity claim

    Open original

  • Langosco, L. et al. (2022). Goal misgeneralization in deep reinforcement learning. ICML 2022.

    Agents pursue the wrong goal off-distribution even when training looked aligned.

    Does not support the necessity claim

    Open original

  • Amodei, D. et al. (2016). Concrete problems in AI safety. arXiv:1606.06565.

    Reward hacking, side effects, and scalable oversight — single-agent failure modes.

    Does not support the necessity claim

    Open originalNamed on Art.1