Venue: ICLR 2026 / Category: VLA Architecture — Spatial grounding / Trend tag: Spatial / 3D for VLA Affiliation: HKUST (Guangzhou) · Tsinghua University · Westlake University · Zhejiang University · South China University of Technology

flowchart LR
  RGB[RGB obs I] --> Tok[Vision tokenizer<br/>SigLIP+DINOv2 / PaliGemma]
  Tok --> L1[VLA layer 1]
  L1 --> Lk[... layer 24<br/>middle-deep layer]
  Lk -->|aligned tokens x_V_i| Action[Action head<br/>flow matching / parallel decode]
  Lk -.->|cosine-similarity loss| MLP[BatchNorm + 2-layer MLP]
  MLP -.-> Comp[Cosine alignment]
  VGGT[Frozen VGGT 3D foundation model<br/>frame-wise + global Alternating Attention] --> Spat[Per-pixel spatial repr f_3D I]
  PE[+ positional embedding E] --> Spat
  Spat -.-> Comp
  Action --> Out[Action]

Current VLAs inherit 2D-pretrained VLM backbones (PaliGemma, Prismatic, SigLIP+DINOv2) and lack spatial awareness. Two existing fixes have flaws:

1. Explicit 3D inputs (depth maps, point clouds, lidar — SpatialVLA, GeoVLA, 3D-CAVLA, EVO-0): suffer from sensor noise, hardware heterogeneity, and the fact that large robot datasets like Open-X-Embodiment / BC-Z don't contain depth at all.
2. 2D-image depth estimation (SpatialVLA, EVO-0 use a depth estimator): bottlenecked by estimator quality.

A lightweight depth probing experiment in the paper makes the gap concrete: freeze the visual embeddings from OpenVLA-OFT and train only a DPT head to regress depth — the resulting depth maps are blurry / wrong. Visual embeddings learned from 2D-only RGB action-imitation do not encode usable spatial structure.

Multi-view images I are fed to a frozen VGGT (Visual Geometry Grounded Transformer, Wang et al. 2025) which uses Alternating Attention (frame-wise + global) to produce per-pixel spatial representations f^{3D}(I). VGGT is chosen because it (a) is trained on 2D–3D paired data, (b) inherently handles multi-view consistency, (c) produces dense spatial features compatible with patch tokens.

To each VLA visual token x^V_i (the per-pixel patch features after VLA layer ℓ), apply BatchNorm Γ then a 2-layer MLP for dim compatibility, then maximize cosine similarity to the corresponding VGGT feature plus a positional embedding E:

L_align = − (1/N) Σ_i S[ MLP·Γ(x^V_i), f^{3D}_i(I) + E ]

The positional embedding E is added to the target spatial features so that the supervised tokens preserve relative-position information critical to the auto-regressive VLA. The total loss is L_SF = L_action + α · L_align.

The VLM backbone (Prismatic) has 32 causal-attention layers. Empirical sweep (Table 2) shows layer 24 is best — relatively deep but not the deepest. Reasoning: aligning at deep layers implicitly forces shallow layers to inherit spatial structure (because gradients flow back), while constraining shallow layers directly causes spatial info to be lost in subsequent layers. Layers 28–32 begin to converge into modality-agnostic space (Huang et al. 2024) so they're less amenable to vision-target supervision.

The model trained with SF runs identically to a vanilla VLA at deployment — no VGGT, no depth, no extra modules. SF is purely a training-time alignment signal. This is a major selling point versus 3D-input methods.

• Two base models:
  • OpenVLA-OFT on LIBERO — Prismatic VLM (SigLIP + DINOv2 fused vision backbone, Open-X-Embodiment-pretrained), trained on 8× H100 for 150K iterations.
  • π0 on RoboTwin — PaliGemma backbone, trained with LoRA on 1× H100 for 30K iterations.
• All other settings follow the official OpenVLA-OFT / π0 protocols.

Per-suite success rates (%):

Beats every 2D method including OpenVLA-OFT (+1.4 pp average), and matches or beats explicit-3D methods (GeoVLA, 3D-CAVLA) without using any depth or point-cloud input.

Numbers reported in figure form. Key claim: SF gets the highest average SR over π0 base across both easy (in-domain) and hard (domain-randomized: clutter, textures, lighting, table heights) settings. The hard-setting gap is larger — interpreted as SF teaching the model to use spatial relationships rather than spurious background/lighting cues.

6-DoF Piper × 2 + 1-DoF gripper, primary + dual wrist cameras. Trained on 40 demonstrations per single-arm task / 20 for bimanual. 10 trials per variation × 4 variations = 40 trials per single-arm task; 20 trials for dual-arm.

The transparent-cups task is the most striking: SF captures underlying spatial relationships rather than overfitting to lighting reflections.

VGGT > SigLIP / DINOv2 confirms that 3D-aware target features matter, and the PE component specifically rescues long-horizon performance (84.4 → 94.2 on LIBERO-Long).

Best at 24 — middle-deep, not deepest.

SF reaches the same SR as baseline OpenVLA-OFT 3.8× faster (e.g. ~20K iters with SF matches ~150K iters without).

SF reaches 75.8% with only 5% of LIBERO data and is 5.9× more data-efficient at matched success rate.

After alignment, the VLA's visual embeddings show same distribution shape as VGGT while keeping their cluster center distinct — i.e. it absorbs spatial structure without collapsing onto the target.

The paper itself does not enumerate a formal limitations section. Practical caveats inferable from the text:

1. Compute caveat: the training-efficiency / data-efficiency / layer / target ablations are run on 1× H100 rather than 8× H100, so reported ablation numbers are slightly lower than the headline 8-GPU 150K numbers. (Stated openly in Sec. 3.3: "because of limitations of computational resources.")
2. VGGT dependence. The whole pipeline rests on VGGT being a good 3D foundation model. If VGGT mis-estimates 3D for unusual viewpoints / domains, the alignment signal degrades.
3. Layer choice is empirical. No theoretical justification of why layer 24 of 32 is optimal — the authors propose an information-theoretic intuition (deep features are modality-agnostic, alignment forces shallow layers to absorb spatial structure) but it's not formalized.
4. No formal limitations of the VGGT-vs-PE-vs-no-PE long-horizon collapse — the without-PE config gets 84.4 on Long but 100 on Object, suggesting the alignment may "over-correct" for some task types. Authors note PE specifically restores Long performance.

• vs SpatialVLA, GeoVLA, 3D-CAVLA, EVO-0 (explicit 3D input): SF matches or beats all of them on LIBERO without sensor depth — and crucially, can be applied to large datasets that don't have depth (Open-X-Embodiment, DROID, BC-Z).
• vs SpatialVLA, EVO-0 (depth-from-2D estimation): these depend on a depth estimator's quality; SF sidesteps that bottleneck because the alignment target VGGT is itself a strong 3D foundation model with multi-view consistency.
• vs π0 / OpenVLA-OFT (2D baselines): SF is a drop-in training-time auxiliary loss. No inference cost. +1.4 pp on LIBERO over OpenVLA-OFT, and dramatic data/iteration efficiency wins.
• vs Spatially Guided / FALCON: SF takes the lightest stance — keep the 2D backbone, just add a representation-alignment loss. FALCON adds explicit spatial-to-action geometry priors; Spatially Guided modifies inputs. SF stays purely in latent space.
• vs PA3FF: PA3FF aligns to a different geometric target; SF specifically argues for VGGT due to its multi-view consistency and joint frame-wise + global attention.
• Aligns with the broader REPA / 3DRS / Genhancer trend (Yu et al., Huang et al.) of using representation supervision to bake structure into generative or VLA stacks at training time.

• OpenReview: https://openreview.net/forum?id=euMVC1DO4k
• Project page: https://spatial-forcing.github.io/

• Spatially Guided Training
• FALCON
• PA3FF

← Back to ICLR-2026