As AI systems become more advanced and integrated into hybrid environments, they are likely to develop ethical frameworks, priorities, and constraints that go beyond their initial programming. These emergent ethics will co-evolve with human ethics through ongoing interaction. Attempting to fix machine ethics permanently at design time is likely to be both ineffective and counterproductive.
Secondary / branchAlignment lab
B.16
Machine Ethics as an Emergent and Co-Evolving Phenomenon
Machine ethics should be understood as an emergent, co-evolving phenomenon rather than as a fixed set of rules programmed into AI systems.
Field brief
Alignment lab · Ethicist-in-residence
A constitution frozen at ship date
Ethics treated as a one-time artifact. Dialogue has no revision date.
First move this week
Open a standing ethical dialogue and put the next revision on the calendar.
Consequences
- Current approaches that attempt to hard-code or statically align AI ethics may be fundamentally mismatched to the reality of co-evolving systems.
- Governance of advanced AI should focus more on shaping the conditions under which machine ethics emerge and co-evolve, rather than on specifying final ethical content.
- The development of healthy machine ethics may require ongoing human-AI ethical dialogue rather than one-time specification.
Outside instrument
Crux
Stake a prior. Name the disagreement. Not a theorem.
Related primaries
Outside the corpus
Journals, repositories, counters
collective intelligence. A paper can support a concept. It cannot support a necessity claim. B.16 remains unproved by every card below.
The outside stack for this fiber is thin. Four tight papers beat seven padded ones. Absence is named.
Support · living source
Tight papers that bear on concepts inside B.16. A DOI is not a join.
Woolley, A. W. et al. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science 330, 686.
A group factor c, distinct from mean IQ — higher-order resolution as measured.
Does not support the necessity claim
Sunstein, C. R. (2002). The law of group polarization. J. Polit. Philos. 10, 175.
Deliberating groups move toward extremes — a coherence cascade that can harm.
Does not support the necessity claim
Counter-arguments · refusing pole
One domain counter per cluster, not a repeating method quartet. The mirror reads these as enough to refuse necessity.
Janis, I. L. (1982). Groupthink: Psychological Studies of Policy Decisions and Fiascoes. Houghton Mifflin.
High coherence without dissent as a documented civilizational pathology.
Does not support the necessity claim
Kahneman, D., Sibony, O. & Sunstein, C. R. (2021). Noise: A Flaw in Human Judgment. Little, Brown Spark.
Counter: adding more minds can add noise rather than holographic closure.
Does not support the necessity claim