Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction
Authors: Max Kanwal, Caryn Tran
Published: arXiv:2607.00001 [cs.AI], April 1, 2026
Presented at: Proceedings of the AAAI-26 Workshop on Machine Ethics
Abstract
Most current approaches to AI alignment treat human preferences as fixed targets to be inferred and subsequently optimized. This assumption, however, conflicts with extensive empirical evidence demonstrating that preferences are layered, dynamic, and actively constructed through interaction—particularly with adaptive technologies. As AI systems become increasingly persistent, personalized, and socially embedded, they play an active role in shaping what people attend to, value, and endorse over time.
We introduce Constructive Alignment, a paradigm that reframes alignment as a control problem over evolving human preference trajectories rather than static preference satisfaction. Drawing on behavioral economics, psychology, and constructivist social theory, we model preferences as layered state variables that evolve under interaction with AI systems. We formalize this perspective using a control-theoretic framework in which system actions and interaction design jointly influence both world states and human evaluative states.
We argue that alignment is not primarily about controlling AI behavior, but about regulating how AI systems influence the evolution of human preferences. The goal is to ensure that value trajectories remain:
- Coherent – internally consistent across contexts
- Reflectively endorsed – aligned with users' deeper, deliberative judgments
- Epistemically grounded – informed by reliable evidence and reasoning
- Bounded against manipulation – resilient to deceptive or exploitative influence
- Empowering under uncertainty – supporting autonomous decision-making in ambiguous situations
Alignment, in this view, becomes a problem of governing long-term value formation rather than simply satisfying static preferences. This framework has significant implications for the design of next-generation AI systems, particularly as 2026 marks a turning point where personalized AI assistants, recommendation engines, and social AI agents are becoming deeply integrated into daily life, making the dynamic nature of preference formation a central ethical and technical challenge.
Subjects: Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
DOI: https://doi.org/10.48550/arXiv.2607.00001
Submission history: From Max Kanwal
via ArXiv AI