Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving
Abstract
Chain-of-thought (CoT) reasoning enhances generative models by prompting them to produce intermediate steps before arriving at a final answer. In autonomous driving, however, the output is a continuous action, meaning the reasoning process must align with the spatiotemporal structure of the physical world. This survey explores the critical shift from purely textual CoT to action-grounded reasoning. Reviewing 171 publications—including 130 method papers and 41 benchmarks, datasets, surveys, and analyses—we introduce a representation-centered taxonomy that uses the nature of intermediate states as the organizing principle. We classify the 130 methods into four primary categories: language-based, visual-spatial, latent-dynamic, and externalized reasoning, further subdivided into 13 subtypes tied to specific regions of interest. Our synthesis indicates that the frontier of reasoning in driving agents lies in developing intermediate representations that are grounded in real-world contexts, compatible with real-time action, and verifiable within safety-critical systems. For additional resources, visit our project page: https://github.com/tangzhengxu/awesome-av-cot.
1. Introduction
As autonomous driving advances, the integration of large language models (LLMs) has brought chain-of-thought (CoT) reasoning from natural language processing to the driver's seat. Traditional CoT, which articulates step-by-step textual reasoning, proves effective for tasks with discrete outputs such as question answering. Yet driving demands continuous, real-time actions in dynamic environments, creating a mismatch between textual reasoning and the physical realities on the road.
In 2026, the push toward Level 4 and Level 5 autonomy has intensified the need for robust, explainable decision-making. The key challenge is that reasoning must not only be interpretable but also seamlessly connect to action execution. This survey addresses this gap by examining how intermediate reasoning states are being reimagined to support action-grounded decision-making.
Throughout this paper, we provide a comprehensive overview of recent developments, propose a taxonomy for categorizing reasoning approaches, and identify challenges and opportunities for future research.
2. From Textual CoT to Action-Grounded Reasoning
2.1 Limitations of Textual Reasoning in Driving
Textual CoT fails to capture essential information such as spatial relationships, temporal dynamics, and uncertainty. In driving, hazards like a pedestrian stepping into traffic require swift, embodied responses that cannot be fully expressed through language alone. Moreover, textual outputs often lack the temporal resolution needed for real-time control.
2.2 The Emergence of Action-Grounded Reasoning
Action-grounded reasoning is a paradigm shift that aligns intermediate reasoning steps with the physical world, ensuring that generated actions are grounded in real-world observations. This approach supports both safety and interpretability, enabling systems to justify their decisions while acting swiftly.
By 2026, several production-grade systems have begun adopting hybrid architectures that combine latent reasoning with language explanation, demonstrating the practical viability of action-grounded approaches.
3. A Taxonomy of Methods
We propose a representation-centered taxonomy that organizes methods from 130 surveyed papers into four categories, each further divided into subtypes.
3.1 Language-Based Reasoning
Though textual CoT has limitations, language remains a flexible medium for expressing high-level plans. Methods in this category often use language models to generate driving plans that are then translated into actions. Subtypes include:
- Rule-based verbalization: Converting sensor data into textual descriptions for decision-making.
- LLM-driven planning: Using LLMs to generate step-by-step driving instructions.
- Multimodal integration: Fusing language with visual or sensor inputs to enhance context.
3.2 Visual-Spatial Reasoning
These methods rely on visual representations, such as semantic maps, object bounding boxes, or bird's-eye-view (BEV) grids, to construct intermediate states. Subtypes include:
- BEV-based reasoning: Reasoning over top-down views for holistic scene understanding.
- Attention-based spatial reasoning: Focusing on relevant regions using attention mechanisms.
- Graph-based reasoning: Representing relationships among objects as graphs.
3.3 Latent-Dynamic Reasoning
Here, intermediate states are represented in latent space, capturing subtle dynamics without explicit human-readable formats. Subtypes include:
- Neural predictive coding: Learning latent world models for planning and control.
- Variational latent reasoning: Using uncertainty-aware latent variables.
- Reinforcement learning (RL) with latent goals: Guiding action selection via learned latent objectives.
3.4 Externalized Reasoning
These methods offload reasoning to external components or modules, such as knowledge bases or simulation environments. Subtypes include:
- Symbolic reasoning and knowledge graphs: Leveraging structured world knowledge.
- Simulation-based inference: Testing hypotheses in virtual environments before execution.
- Hybrid human-machine reasoning: Collaborating with human oversight.
4. Synthesis and Future Directions
The reviewed methods reveal several converging trends. First, intermediate representations that are grounded in the real world—like BEV maps or latent dynamics—offer a stronger basis for safety. Second, integrating real-time action constraints into reasoning remains an open challenge, especially under uncertainty. Third, verification of reasoning under safety-critical systems is a growing priority, leading to research on formal verification and runtime monitoring.
Future work should focus on:
- Improving the interpretability of latent-dynamic methods.
- Developing benchmarks that evaluate both reasoning quality and action safety.
- Integrating reasoning modules with control-theoretic guarantees.
5. Conclusion
This survey charts the transition from textual CoT to action-grounded reasoning in autonomous driving, offering a structured view of 130 methods. Our taxonomy underscores the importance of intermediate representations that are physically meaningful, temporally precise, and verifiable. As autonomous systems evolve, action-grounded reasoning will be pivotal in achieving both safety and performance. The resources compiled here aim to support ongoing research in this critical area.
Acknowledgments
We thank the authors of the surveyed papers for their contributions to the field.
via ArXiv CV