Moral cognition & norms

How people and agents learn, represent, and enforce norms, and what it takes for an agent to take part in a normative community.

Curated by
TBD
Relevant for
4 grid cellsTacit norms for agent interactions and 3 others
Write to
norms@agi-institutions.orgAn operator routes it to the right experts.

Selected papers

2

Must-reads for understanding the field and why it matters for AGI institutions.

  1. A Theory of Appropriateness That Accounts for Norms of Rationality
    Joel Z. Leibo, Alexander Sasha Vezhnevets, Manfred Diaz et al.2026 · Preprint

    Develops a formal account of appropriate action that can represent rational norms alongside goals and outcomes.

  2. Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Games
    Ninell Oldenburg, Tan Zhi-Xuan2024 · AAMAS · Peer-reviewed

    Builds agents that infer obligative and prohibitive norms from observed compliance, letting groups converge on shared institutions such as resource management and compensation rules.

Work in the field

8

Recent work in the field worth knowing.

  1. CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives
    Ayoung Lee, Ryan Sungmo Kwon, Peter Railton et al.2025 · ICLR · Peer-reviewed
  2. Machine Theory of Mind and the Structure of Human Values
    Paul de Font-Reaulx2025 · NeurIPS · Workshop paper
  3. MoReBench: Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes
    Yu Ying Chiu, Michael S. Lee, Rachel Calcott et al.2025 · ICLR · Peer-reviewed
  4. Relational Norms for Human-AI Cooperation
    Brian D. Earp, Sebastian Porsdam Mann, Mateo Aboy et al.2025 · Preprint
  5. Resource Rational Contractualism Should Guide AI Alignment
    Sydney Levine, Matija Franklin, Tan Zhi-Xuan et al.2025 · IASEAI · Peer-reviewed
  6. Ethical learning, natural and artificial
    Peter Railton2020 · Oxford University Press · Book chapter
  7. A Common Framework for Theories of Norm Compliance
    Adam Morris, Fiery Cushman2018 · Social Philosophy and Policy · Peer-reviewed
  8. The Naïve Utility Calculus: Computational Principles Underlying Commonsense Psychology
    Julian Jara-Ettinger, Hyowon Gweon, Laura E. Schulz et al.2016 · Trends in Cognitive Sciences · Peer-reviewed

Foundations

5

Older work that gives background on the field.

  1. A Natural History of Human Morality
    Michael Tomasello2016 · Book
  2. The Secret of Our Success
    Joseph Henrich2015 · Book
  3. The Grammar of Society: The Nature and Dynamics of Social Norms
    Cristina Bicchieri2006 · Book
  4. Governing the Commons: The Evolution of Institutions for Collective Action
    Elinor Ostrom1990 · Book
  5. An Evolutionary Approach to Norms
    Robert Axelrod1986 · American Political Science Review

This list isn't exhaustive. About the lists.