Review Protocol
We adopted a structured narrative review protocol to cover both established E2E-AD literature and rapidly emerging work across peer-reviewed venues, arXiv preprints, benchmark repositories, challenge leaderboards, code releases, and official project pages. The primary sources included IEEE Xplore, ACM Digital Library, SpringerLink, ScienceDirect/Elsevier, arXiv, OpenReview, CVF Open Access, NeurIPS proceedings, public benchmark repositories, and official project websites.
The main search period covered work from 2015 to June 2026, with earlier studies included when they shaped neural or end-to-end driving. The core search terms included “end-to-end autonomous driving,” “learning-based planning,” “closed-loop driving,” “autonomous driving planning benchmark,” “nuPlan,” “NAVSIM,” “Bench2Drive,” “WOD-E2E,” “world model autonomous driving,” “vision-language driving,” “VLA driving,” “long-tail autonomous driving,” and “autonomous driving generalization benchmark.”
We also conducted backward and forward citation tracing from highly connected papers and benchmarks, including PilotNet, Conditional Imitation Learning, Learning by Cheating, TransFuser, TCP, UniAD, VAD, NAVSIM, Bench2Drive, DriveLM, SimLingo, WOD-E2E, and LEAD.
Inclusion and Exclusion Criteria
A work was included if it satisfied at least one criterion:
- Introducing an influential E2E driving formulation.
- Predicting planning-relevant outputs such as controls, waypoints, trajectories, trajectory distributions, or action tokens.
- Proposing a benchmark, dataset, simulator, or metric for planning evaluation.
- Contributing a world-model, VLM, or VLA mechanism relevant to ego planning.
- Releasing a public dataset, benchmark, codebase, evaluation server, or reproducibility resource.
- Providing a safety, interpretability, robustness, or evaluation critique of E2E driving.
We excluded studies focused only on perception, detection, segmentation, tracking, or motion prediction unless they directly supported ego planning or were widely used in an E2E driving stack. Purely proprietary systems were also excluded when technical details were insufficient for comparison.
For each retained work, we recorded its input representation, planning output, supervision signal, evaluation protocol, benchmark evidence, and public-resource status when available.
Scope and Definitions
An autonomous driving stack can be called end-to-end when the mapping from driving inputs to driving outputs is optimized as a learned policy or differentiable system whose final objective is a driving action, route-conditioned trajectory, or motion plan.
Inputs may include camera images, LiDAR, radar, ego state, HD maps, route commands, goal points, or natural language. Outputs may include steering and throttle, waypoints, trajectories, trajectory distributions, or discrete action tokens.
Evolution Toward Planning-Oriented E2E-AD
The evolution of E2E-AD is not a single architectural lineage, but a shift from imitating low-level control toward learning representations, supervision signals, and evaluation protocols that make planning behavior more inspectable and benchmarkable.
From Neural Control to Conditional Imitation
The roots of E2E driving predate modern deep learning. ALVINN showed that a neural network could map road images to steering commands, and later off-road driving systems demonstrated direct perception-action learning under constrained conditions. DeepDriving introduced affordance prediction as an intermediate representation between hand-engineered perception and direct control. PilotNet then became the modern reference point for camera-to-steering learning by showing that a convolutional network trained on human demonstrations could control a vehicle in real-world road scenes.
Control, Waypoints, and Trajectories
The choice of output also shapes evaluation. A trajectory can be scored by displacement error, collision proxies, route progress, time-to-collision, comfort, or preference labels. A direct control command is harder to compare under open-loop logs because the same control may have different consequences depending on the vehicle state and controller dynamics. As benchmarks have become more planning-centric, trajectory outputs have become the dominant interface for E2E research.
There is also an underappreciated mismatch between the supervised target and the deployed behavior. A logged trajectory is not a universal optimum; it is one human driver’s realization under one interaction history. If the model deviates slightly, the logged future may no longer be feasible or even desirable.
This is why pure pointwise L2 trajectory loss can be misleading. It rewards imitation of a single future but does not teach the policy how to choose among safe alternatives after perturbation. Modern planners increasingly add multi-modal heads, trajectory refinement, rule-aware costs, or learned scoring functions to bridge this gap.
Overall, the shift from direct control to trajectory output is not merely a change in output format; it makes the policy easier to inspect through controllers, safety constraints, and planning metrics, while exposing new questions about multi-modal futures and plan selection.
BEV, Occupancy, and Vectorized Scene Representations
BEV representations became popular because they provide a geometry-aware coordinate frame for planning. They can integrate multi-view cameras, maps, lanes, objects, and occupancy into a common spatial grid. ST-P3, UniAD, and related works use BEV-like structures to support detection, prediction, occupancy, and planning losses.
Vectorized representations go further by representing lanes, agents, and trajectories as structured tokens rather than dense grids. VAD demonstrated that vectorized scene representation can support efficient autonomous driving; VADv2 extended this direction with probabilistic planning.
The planning advantage of BEV and vector spaces is interpretability and constraint compatibility. A planner can reason about drivable area, lane topology, agent interaction, and route progress more naturally in a spatial representation than in raw image features.
The risk is that dense BEV processing can be expensive and may inherit the limitations of the perception labels used to supervise it. Recent sparse and register-based designs, including DriveTransformer and DrivoR, can be understood as attempts to retain planning-relevant structure while reducing computational cost.
Overall, the value of BEV, occupancy, and vector representations is not only stronger perception; their main planning value is to convert raw sensing into geometry that can be checked against maps, agents, free space, and route constraints.
Unified Stacks and Task Coupling
Task coupling also creates new risks. If a unified model performs well, it may be unclear which internal component is responsible. If it fails, debugging can be harder than in a modular stack. Moreover, joint training can allow shortcut learning: a model may rely excessively on ego state or route signals while underusing visual evidence.
Planning-oriented architecture must therefore be paired with causal diagnostics, ablations, and benchmark protocols that expose shortcut behavior.
A recurring architectural question is therefore not simply whether to be modular or end-to-end. It is where to place the interfaces that must remain stable for safety, debugging, and evaluation.
A system can be end-to-end trained while retaining explicit interfaces such as occupancy, lane topology, object tokens, route commands, uncertainty estimates, or candidate trajectories. Conversely, a system can appear modular but still propagate downstream losses into upstream representations.
Across recent systems, a common pattern is to expose variables needed for planning and safety checks while reducing hand-designed objectives that dominate the learned policy.
Uncertainty and Generative Planning
The open question is how to evaluate generative plans. A distribution is useful only if it ranks safe and goal-consistent trajectories above unsafe or irrelevant ones. World-model-based evaluation, preference labels, and closed-loop simulation are therefore natural complements to generative planning.
Architecture Lessons for Closed-Loop Behavior
These observations suggest a practical checklist for architecture papers. Authors should report:
- Whether the planner uses explicit route tokens or target points.
- Whether it predicts speed in addition to lateral path.
- Whether the controller is learned or fixed.
- Whether trajectory samples are ranked by a learned cost.
- Whether the policy can recover when the ego state deviates from the logged trajectory.
Without these details, architecture comparisons often become comparisons of hidden evaluation machinery.
Overall, architectural novelty is difficult to interpret without reporting the planning interface, controller, safety wrapper, and recovery mechanism. These details often explain closed-loop behavior as much as the encoder backbone.
Training Signals and Data Engines
The learner-expert mismatch becomes especially important in simulation. Privileged experts can generate large datasets cheaply, but an expert that uses unobservable ground-truth state may produce demonstrations that a camera-based student cannot reliably imitate.
LEAD reframes this issue by designing the expert and navigation conditioning to reduce visibility, uncertainty, and intent asymmetry between teacher and student. This is a useful conceptual advance because it treats expert design as part of the learning problem rather than as a fixed data source.
Metric Families
Open-loop metrics include average displacement error, final displacement error, trajectory L2 error, and collision proxies computed against logged scenes. They are useful for fast iteration but weak as final evidence.
A model may match the logged trajectory without being able to recover from deviations, and a different trajectory may be safer or more natural than the logged human path. The paper “Is Ego Status All You Need?” showed that open-loop protocols can be vulnerable to shortcuts in which models rely heavily on ego-state signals rather than perception. Hidden Biases similarly demonstrated that benchmark protocols can distort conclusions about E2E driving models.
Why Open-Loop and Closed-Loop Disagree
Open-loop and closed-loop metrics disagree for structural reasons. First, open-loop evaluation scores a policy under the expert’s state distribution, while closed-loop evaluation scores it under its own induced state distribution.
Second, open-loop metrics often penalize valid alternatives, especially in multimodal interactions. Third, open-loop metrics typically lack feedback from background agents and rule violations. Fourth, closed-loop systems depend on controller details, simulator dynamics, and stochastic scenario outcomes that may not appear in open-loop logs.
This disagreement has practical consequences for survey writing. A method should not be called state of the art without specifying the protocol. A strong nuScenes L2 score, a strong NAVSIM PDMS score, and a strong Bench2Drive Driving Score are different claims.
Comparative tables are more reliable when they group methods by benchmark and metric family rather than mixing scores across protocols.
Recent correlation work makes this point sharper. Cross-referencing NAVSIM and Bench2Drive results suggests that safety-aware open-loop scores can correlate with closed-loop driving score better than ADE or FDE, but ranking inversions remain and submetrics can saturate.
NAVSIM-style scores are valuable because they are scalable and more planning-aware than L2, but they should be described as proxy evidence, not as a substitute for closed-loop interaction. Conversely, Bench2Drive-style scores are more behaviorally meaningful, but they remain tied to CARLA’s simulator distribution.
A robust empirical claim should therefore triangulate across at least one real-log proxy and one reactive closed-loop protocol.
Recommended Reporting Protocol
For future E2E-AD papers, we recommend reporting five groups of evidence:
- Open-loop trajectory quality.
- Non-reactive real-log planning score.
- Closed-loop route and infraction metrics.
- Long-tail or preference-aware performance.
- Qualitative failure analysis.
Authors should also report sensor configuration, route inputs, output type, controller, safety wrapper, training data source, data filtering, compute budget, random seeds, benchmark version, and metric implementation.
Without these details, small leaderboard differences are difficult to interpret.
We recommend avoiding a single “best method” column. A more honest format is to use separate columns for open-loop imitation, proxy planning, reactive closed-loop, long-tail shift, code availability, and compute.
A method that is weak in open-loop L2 but strong in closed-loop route completion may be a better planner. A method that is strong on NAVSIM but slow in Bench2Drive may be over-optimizing safety at the expense of progress. A VLA method that produces strong rationales but mediocre trajectories should not be promoted as a driving breakthrough.
The unit of comparison should be the claim, not the model name.
Runtime Assurance and Safety Envelopes
Empirical benchmark safety is not the same as operational safety assurance. A practical E2E driving stack will likely require runtime mechanisms that constrain or monitor the learned planner.
These mechanisms can be added without abandoning end-to-end learning: the neural policy proposes trajectories, while an assurance layer checks whether the proposal remains inside a verified or rule-constrained envelope.
This view is compatible with planning-oriented E2E-AD because it treats learning as a source of candidate behavior, not as the only safety mechanism in the vehicle.
Several engineering routes are relevant:
- A safety monitor can reject plans that violate drivable area, speed limit, route topology, or time-to-collision thresholds.
- A fallback planner can switch to conservative braking or rule-based lane keeping when confidence is low.
- Rule-based shields can enforce traffic-light, stop-sign, or lane-boundary constraints.
- Control barrier functions and reachability analysis can define safe sets for low-level control, though scaling them to perception-conditioned, multi-agent, high-dimensional E2E policies remains difficult.
- Emergency braking modules can reduce collision risk but may create comfo
Frequently Asked Questions
What makes an autonomous driving stack end-to-end? It maps driving inputs to actions, route-conditioned trajectories, or motion plans through a learned policy or differentiable system optimized for a driving objective.
Why are trajectory outputs increasingly used in E2E driving research? Trajectories are easier to inspect through controllers, safety constraints, and planning metrics than direct control commands.
Why can open-loop and closed-loop metrics disagree? Open-loop evaluation uses the expert’s state distribution, while closed-loop evaluation uses the policy’s induced states and includes controller, simulator, and interaction effects.
What should future E2E-AD papers report? They should report open-loop, proxy planning, closed-loop, long-tail or preference-aware, and qualitative failure evidence, along with sensors, controllers, data, compute, and metric details.