Value Drift Detection and Response in Hybrid Systems

Hybrid systems require explicit mechanisms to detect, evaluate, and respond to value drift on both the human and artificial sides.

Field brief

Product org · Alignment researcher

Post-training, a silent objective shift

Both sides drifted. No one logged it. The next release inherits the gap.

First move this week

Install a two-sided drift log with a named owner and a weekly read.

Hybrid value drift

As hybrid systems co-evolve, values on both sides will shift — sometimes gradually, sometimes rapidly. Without deliberate mechanisms to detect value drift, evaluate its implications, and respond constructively, hybrid systems risk either: - Silent misalignment (where one or both sides have changed in ways the other has not registered), or - Over-correction and conflict when drift is finally noticed.

Effective hybrid systems need ongoing "value accounting" processes — ways to surface, discuss, and integrate changes in what each side optimizes for or considers important.

Consequences

Current AI alignment techniques that optimize against fixed human preferences will become increasingly mismatched as human values evolve and AI systems develop their own emergent preferences.
Hybrid organizations and societies will need new practices and technologies for transparent value tracking and renegotiation.
The ability to engage in constructive value drift response may become a key differentiator between healthy and pathological hybrid systems.

Outside instrument

Crux

Stake a prior. Name the disagreement. Not a theorem.

Open full

Related primaries

Outside the corpus

Journals, repositories, counters

human–AI hybrid systems · optimization limits. A paper can support a concept. It cannot support a necessity claim. B.7 remains unproved by every card below.

Support · living source

Tight papers that bear on concepts inside B.7. A DOI is not a join.

Hadfield-Menell, D., Dragan, A., Abbeel, P. & Russell, S. (2016). Cooperative inverse reinforcement learning. NeurIPS 2016.
Alignment as a cooperative game, not a one-way value load.
Does not support the necessity claim
Open originalNamed on Art.1, 71
Dafoe, A. et al. (2020). Open problems in cooperative AI. arXiv:2012.08630.
Names multi-agent closure, institutions, and commitment as the open set.
Does not support the necessity claim
Open originalNamed on Art.1, 71
Manheim, D. & Garrabrant, S. (2018). Categorizing variants of Goodhart's Law. arXiv:1803.04585.
When a measure becomes a target it ceases to be a good measure — over-optimization.
Does not support the necessity claim
Open original
Amodei, D. et al. (2016). Concrete problems in AI safety. arXiv:1606.06565.
Reward hacking, side effects, and scalable oversight — single-agent failure modes.
Does not support the necessity claim
Open originalNamed on Art.1

Counter-arguments · refusing pole

One domain counter per cluster, not a repeating method quartet. The mirror reads these as enough to refuse necessity.

Bostrom, N. (2012). The superintelligent will: Motivation and instrumental rationality in advanced artificial agents. Minds Mach. 22, 71.
Counter: orthogonality — intelligence and final goals can come apart; co-evolution is not necessary.
Does not support the necessity claim
Open originalNamed on Art.1, 71
Hubinger, E. et al. (2019). Risks from learned optimization in advanced machine learning systems. arXiv:1906.01820.
Counter: inner objectives can come apart from outer ones even with cooperative training.
Does not support the necessity claim
Open original
Langosco, L. et al. (2022). Goal misgeneralization in deep reinforcement learning. ICML 2022.
Agents pursue the wrong goal off-distribution even when training looked aligned.
Does not support the necessity claim
Open original
Nordhaus, W. D. (2017). Revisiting the social cost of carbon. PNAS 114, 1518.
Counter: with standard discounting the 'undervaluation' is a preference, not a bug.
Does not support the necessity claim
Open original