Motivating Analysis
On NAVSIM v1, we first use identical fixed-exit, single-trajectory readouts to test sensitivity to video-noise level and DiT depth, and whether different depths solve redundant scene sets. Reported PDMS aggregates validation-best checkpoints over ten seeds.
Method
Motivated by the motivating analysis, Adaptive-WAM exposes multiple trajectory exits and routes by predicted quality.
Problem and Backbone
The conditional backbone prefix produces intermediate video features used for trajectory planning.
This operation is a single video-feature forward; it is separate from the five DDIM steps used by each trajectory head.
Multi-Exit Trajectory Decoding
All heads have the same architecture, optimization budget, batch size, and number of epochs. They use five action-denoising steps, matching ReCogDrive. The exit heads have independent parameters and do not exchange features or predictions. This makes differences across exits attributable to backbone depth rather than to head capacity.
Each single-trajectory head uses the same logged-trajectory diffusion objective and five-step training protocol as ReCogDrive.
Training
During imitation learning, the Wan backbone is adapted using LoRA, while the trajectory projections and heads are optimized in full. The actor objective is:
The actor objective combines the native Wan video-diffusion objective with trajectory-learning objectives.
The native Wan video-diffusion objective is used during training. Both video supervision and trajectory decoding use the same fixed video-noise index and the same conditional Wan forward during joint adaptation; no second backbone forward is required. Video prediction is retained as representation supervision rather than a required deployment output.
Scorer Reliability and Adaptive Trade-Off
The scorer is evaluated on an offline diagnostic pool covering 12,146 scenes. Exact top-score selection succeeds in 91.2% of scenes, and tie-aware soft accuracy reaches 94.4% when a selected trajectory within 5% of the true top score is accepted. Strict rank correlation is not informative here: more than 95% of scenes contain trajectory groups that are all perfect, all zero, or tied at the top. We instead stress-test consequential failures. Only 51 scenes (0.42%) select a trajectory at least 50% worse than an available near-perfect trajectory, and 69 scenes (0.57%) exceed a 20% gap.
The current threshold sweep shows that, at the reported operating threshold, the adaptive policy improves on fixed B15 by 0.17 PDMS while reducing average end-to-end planning latency by approximately 10%, from 190 to 170 ms. More than 94% of scenes terminate within the first three exits, and the fixed full-depth planner averages 320 ms. The permissive threshold 70 instead loses 2.13 points relative to fixed B15.
Ablations and Efficiency
Freezing Wan is insufficient. Fine-tuning Wan separately and then caching its features recovers only a small part of the loss, whereas joint LoRA training improves the single-trajectory model by 5.67 points over cached features. Full fine-tuning yields no meaningful gain over LoRA. Replacing Wan features with ViT-S, ViT-B, and ViT-L lowers single-trajectory PDMS by 6.71, 5.00, and 1.74 points, respectively.
NAVSIM-to-nuScenes Zero-Shot Protocol
The NAVSIM-trained model is evaluated on nuScenes without nuScenes fine-tuning. The model continues to use a single front view and predicts ego trajectories in the local coordinate frame. We follow prior world-model planners by reporting displacement error and collision rate at 1, 2, and 3 seconds and their horizon average. A separate comparison distinguishes methods trained on nuScenes from NAVSIM-to-nuScenes zero-shot methods.
Dataset Construction and Temporal Filtering
For an anchor at time t, the dataset retrieves the current CAM_F0 frame and the next eight front-camera frames at the native NAVSIM/OpenScene rate of 2 Hz.
Timestamps and each frame’s relative time are stored in the metadata and checked when constructing the clip. An anchor is removed if any of the eight required future frames is absent, which commonly occurs near the end of a log, or if a referenced camera image cannot be loaded. The aligned trajectory target contains the eight future poses over the same four seconds.
Starting from 103,288 candidate navtrain tokens, temporal filtering retains 82,555 complete nine-frame clips and removes 20,733 candidates that cannot provide the required current-plus-eight-future sequence. NAVSIM does not define an official train/validation partition for this converted corpus, so we report corpus-level counts rather than presenting an internal model-selection partition as an official benchmark split.
The metadata record stores the scenario token, log identifier, anchor timestamp, relative camera paths, camera intrinsics, ego velocity and acceleration, aligned future trajectory, and programmatic text description. The PyTorch dataset resolves image paths relative to the OpenScene root at load time.
Programmatic Text Descriptions
The Wan text condition is generated from structured, auditable attributes rather than from a separate vision-language captioner. The final template is:
> Vehicle {motion}{turn_info} at {speed:.1f} m/s in urban environment.
The motion descriptor uses the current speed: below 0.5 m/s is stationary, 0.5–3 m/s is slowly, 3–8 m/s is cruising, and above 8 m/s is fast. A turn descriptor is added only when the observed historical lateral displacement exceeds 0.3 m. Current map-area and traffic-flow metadata are also incorporated by the description generator. Crucially, all fields are computed from map metadata and observations at or before the anchor time; neither the ground-truth future trajectory nor future images are queried. The prompt is stored with the scenario token, and the finite set of resulting T5 text embeddings is cached in both training and inference.
Fixed Video-Noise Index and Multi-Exit Readout
Wan2.2-TI2V-5B contains 30 video-DiT blocks. The native video sampler has 40 noise steps, but the deployed planner executes one conditional Wan forward at a fixed sampling index. The future part of the latent is initialized from scheduler noise at that index; the current image provides the conditioned latent slice. No observed future latent, unconditional classifier-free-guidance branch, or video VAE decoding is used by the planning path.
Hidden states are read after selected DiT blocks. Each readout has its own projection and trajectory head. The six heads have independent parameters and do not exchange features or predictions, while their trajectory losses all backpropagate through the shared Wan LoRA during imitation learning. When routing continues from one exit to the next, the previously evaluated DiT prefix and its hidden state are reused.
Actor, Scorer, and Gradient Isolation
During imitation learning, the Wan backbone is adapted with LoRA; the trajectory projections and heads are fully trainable. The actor loss is:
The actor loss combines native video-diffusion supervision with the trajectory objectives.
The native video-diffusion objective and all trajectory heads use the same fixed video-noise index and conditional Wan forward, avoiding a second backbone pass.
The quality scorer receives detached trajectories. Thus, scorer fitting cannot reshape the trajectory generator through proposal coordinates. Actor and scorer updates alternate on the same training stream.
After imitation learning, Wan and the scorer are frozen. Only the six trajectory heads are refined using the DiffGRPO formulation adopted by ReCogDrive: one complete five-step trajectory-denoising chain is treated as a composite action and receives the NAVSIM planning score as its reward. Each exit uses the same training budget and validation-best checkpoint rule. Layer statistics reported in the main paper aggregate ten runs.
Fixed-Exit Planning Scores
Block 15 is the strongest fixed exit in both stages, but later blocks retain scene-specific advantages. This distinction motivates routing by candidate quality rather than always choosing either block 15 or the final block.
Qualitative Layer-Wise Trajectories
The examples are diagnostic overlays of the six fixed-exit trajectories, not additional model inputs. They show that layer-wise differences can correspond to distinct decisions, while metric saturation can also assign identical high scores to visibly different but acceptable plans.
- Early exits differ from the deeper consensus.
- Four exits overlap while two retain distinct modes.
- All six trajectories receive full score despite variation.
Scorer Architecture and Objective
The deployed scorer fine-tunes a DINOv2-Small image encoder. It uses only the current front image and the candidate trajectory; ego state and navigation command are not scorer inputs. The trajectory is flattened and embedded by an MLP. The trajectory and image features are concatenated, and six independent two-layer MLP heads output logits for NC, DAC, DDC, TTC, EP, and Comf.
For evaluator component targets, the scorer uses equal-weight soft-label binary cross-entropy:
The scorer minimizes equal-weight binary cross-entropy against soft evaluator-component targets.
Targets are not binarized. No global rank loss is used because the official composition contains many ties and near-ties. At inference, sigmoid component predictions are combined into the final quality score.
Tie-Aware Reliability
The scorer diagnostic uses a fixed offline candidate pool covering 12,146 scenes. This pool is used only to compare scorers and is distinct from adaptive inference, which accumulates at most six trajectories. More than 95% of diagnostic scenes contain candidates that are jointly perfect, jointly zero, or tied at the top. We therefore report selection quality and consequential errors rather than a strict total-order correlation.
Video-Index and Backbone Diagnostics
Wan scorer features are also insensitive to the five tested video indices. At index 17, the best Wan exit obtains 92.62, only 0.03 above DINO-Small. Because a Wan-based scorer would add a large world-model forward at every attempted exit, this negligible diagnostic difference does not justify its online cost. DINO-Small is therefore used as the quality verifier.
Conclusion
Intermediate video-DiT features are robust planning representations well before full image synthesis completes. By coupling multi-depth trajectory heads with a tie-aware trajectory-quality controller, Adaptive-WAM allocates world-model computation according to the quality of the current plan. It achieves strong NAVSIM and zero-shot nuScenes results while exposing a practical path from large generative world models to efficient driving policies.
Frequently Asked Questions
What is Adaptive-WAM? Adaptive-WAM is a planning method that exposes multiple trajectory exits from intermediate video-DiT features and routes by predicted trajectory quality.
How does Adaptive-WAM reduce planning latency? More than 94% of scenes terminate within the first three exits, reducing average latency from 190 to 170 milliseconds compared with fixed B15.
What data does the quality scorer use? The scorer uses the current front image and candidate trajectory, without ego state or navigation command inputs.
Why is DINOv2-Small used as the quality verifier? Wan-based scorer features provide negligible diagnostic improvement while requiring a large world-model forward at every attempted exit.