Beyond Task Completion: Why AI Agents Need to Prioritize Human Growth Over Automation
A new research framework argues that AI agents should be designed to enhance human judgment and capability over time, not simply replace human work with automation. Rather than optimizing for how much work disappears from human hands, the emerging "Combodied Agents" paradigm measures success by what people retain, understand, and can do independently after interacting with AI systems.
The distinction matters in real-world scenarios. When an older adult misses a medication dose, a software agent might issue a reminder while a physical robot could deliver the medication. But neither action reveals whether the person forgot, became confused, experienced side effects, or deliberately refused the dose. More importantly, neither approach tells us what kind of support would actually help that person maintain their health and independence long-term.
What's Wrong With Today's AI Agent Design?
Current AI agents fall into two dominant categories. Digital agents navigate software interfaces, modify code, and coordinate workflows through application programming interfaces (APIs). Embodied agents combine language understanding with physical capabilities like navigation and manipulation. In both cases, progress has been measured by task completion, longer task horizons, and reduced human supervision.
This creates a hidden problem. When AI improves a document, the author may understand it less. When AI accelerates a decision, the user's ability to judge its basis may weaken. Research on AI-assisted knowledge work shows that people often reduce their cognitive effort and develop overreliance when they cannot determine when to trust, verify, or reject AI outputs. Task-success metrics capture immediate output but miss the capability and judgment that repeated AI use leaves with the person.
The concern extends beyond individual productivity. Human knowledge is often local, tacit, and renewed through participation. When standardized AI outputs replace those practices, expertise, variety, and resilience can weaken. General-purpose AI models are developed and aligned in relatively few institutions, while users differ in culture, commitments, relationships, and values. Concentrating intelligence while weakening people's productive and epistemic agency can concentrate the authority to define both goals and acceptable values.
How Should AI Agents Be Redesigned for Human Benefit?
The Combodied Agents framework proposes organizing AI systems around beneficial trajectories of human state and agency rather than external task completion. This means digital tools, wearables, robots, and human services all serve as action channels, but success is determined by what the person can understand, decide, do, and sustain over time.
The framework includes several interconnected components:
- Event-based multimodal perception: Reconstructs evidence about meaningful personal events from multiple data sources to build context.
- Longitudinal and correctable memory: Provides temporal context while allowing users to correct or update information about themselves.
- Personal World Models: Transform event evidence and current context into calibrated predictions about future personal states, observable events, and outcomes under different decisions and interventions.
- Admissible intervention policy: Selects proportionate support under constraints of consent, uncertainty, safety, reversibility, and user control.
- Feedback loops: Updates perception, memory, prediction, and intervention based on responses from the person and environment.
Rather than requiring an exhaustive digital twin of a person, this framework uses purpose-bounded, uncertainty-aware, and user-correctable representations focused on specific goals and contexts.
Steps to Implement Human-Centered AI Agent Design
- Distinguish routine from high-stakes tasks: Routine, low-risk, reversible tasks may justify near-complete delegation, while learning, health, emotional support, and high-stakes decisions require explanation, scaffolding, consent, or human escalation.
- Design for selective autonomy: Agents should learn when to act independently, request oversight, or return knowledge and capability to the person based on the specific context and the person's goals.
- Preserve user control and correction: Systems must allow people to choose, refuse, correct, and recover from AI recommendations, maintaining their ability to override or adjust agent behavior.
- Measure human outcomes alongside task completion: Evaluate success by what the person retains and develops, including understanding, competence, agency, identity, and calibrated reliance on AI.
- Apply proportionality constraints: Support should respect constraints of consent, uncertainty, safety, reversibility, and user control rather than maximizing automation for its own sake.
This approach does not require people in every low-level decision loop or resist automation entirely. Instead, human participation becomes a technical design problem that must be solved thoughtfully based on the nature of the task and the person's needs.
Why Personal Agents Make This Problem Especially Urgent?
Memory-enabled assistants, AI companions, health agents, and behavioral coaches increasingly participate in identity, emotion, habits, and care. They can provide continuity and meaningful support, but sustained interaction may also foster dependence, manipulation, social displacement, or inappropriate influence, especially among minors and vulnerable users. The problem is not automation itself but making substitution the default regardless of its long-term effects on understanding, autonomy, relationships, values, and future capability.
The research framework proposes organizing the design space through human-state targets, relational contexts, and agent roles. It also suggests moving toward edge-native personal models that run on users' own devices rather than centralized servers, scenario-centered evaluation methods, agency-preservation metrics, and governance directions that protect user autonomy.
By shifting AI agents from external task completion toward sustained human benefit, the Combodied Agents paradigm aims to improve health, learning, judgment, capability, relationships, and goal pursuit without treating engagement, dependence, or maximum automation as measures of success. The framework represents a fundamental rethinking of how AI should be developed and evaluated in domains where human growth and autonomy matter as much as task completion.