Vision · Language · Action

BeliefVLA: Implicit Belief-State Driven Vision-Language-Action
Models for Robust Manipulation

Yu Liu^1,†Tianlv Huang^1,†Wei Han^1,2,†Hetian Guo¹Chengyu Miao¹Fei Yan¹Zipei Fan^1,*Xuan Song¹

¹ School of Artificial Intelligence, Jilin University² IO-AI TECH

† Equal contribution* Corresponding author

Read the paper PDF ↗ Watch the film YouTube

Grounding the next action in a history of observations.

97.2%LIBERO average successFour simulation suites 78.3%Real-world general tasksSix manipulation tasks +15.4ppUnder observation perturbationsAverage gain over π_0.5

Belief in action.

Real robots. Dynamic environments. YouTube

Demonstrations of bimanual manipulation and observation perturbations.Watch on YouTube

01 / Overview

Beyond the current frame.

A single observation does not tell the whole story.

Objects move, views become occluded, and the scene changes. BeliefVLA uses a sequence of past observations to infer an implicit belief state, giving a vision-language-action policy access to temporal context and learned physical priors.

Current-frame semantics and history-derived dynamics meet in a shared representation that conditions continuous robotic actions.

Read the formulation

Paper Figure 1: a reactive policy loses the object behind a barrier; BeliefVLA uses observation history to infer its location. — Figure 1. Historical observations provide context when the current view is incomplete. Illustration from the paper.

Abstract

While Vision-Language-Action (VLA) models exhibit remarkable open-vocabulary generalization in robotic manipulation, they remain fundamentally constrained by a structurally reactive paradigm. Inferring actions from static visual representations or simple frame stacking forces policies to learn temporal dynamics from scratch, lacking the physical grounding necessary for robust real-world control. We introduce BeliefVLA, a framework that shifts robotic control from purely reactive mapping to reasoning within a continuous implicit belief state.

By leveraging a Video Joint-Embedding Predictive Architecture (V-JEPA2) to infer these latent states from historical context, BeliefVLA injects dynamics-aware priors directly into the policy. This approach seamlessly integrates physical reasoning and spatiotemporal dynamics without sacrificing the high-level semantic capabilities of foundational VLMs. Extensive experiments establish BeliefVLA’s state-of-the-art performance across simulated and real-world benchmarks. Furthermore, five observation-perturbation tasks explicitly validate its capacity for temporal memory, action-centric features, and physical priors.

02 / Method

History becomes a belief.
Belief informs an action.

A dual-perception architecture: instantaneous visual semantics alongside a dynamics-aware representation of the recent past.

BeliefVLA architecture: current images enter SigLIP, instructions enter the text tokenizer, and historical images enter V-JEPA2. Their tokens are fused in a pretrained VLM and condition a 300M-parameter action expert. — Figure 2. The pretrained vision-language front-end and domain-adapted V-JEPA2 encoder condition a continuous action expert. Original architecture figure from the supplied manuscript.

01 — Encode history

Implicit belief state

A V-JEPA2 ViT-Large encoder, adapted on robotic trajectories, extracts temporal features from historical observations. These features serve as a learned physical and temporal prior.

02 — Fuse perception

Shared multimodal context

Projected belief tokens are concatenated with language and current-frame SigLIP tokens. Bidirectional self-attention in PaliGemma integrates these complementary representations.

03 — Generate actions

Continuous action chunks

A 300M-parameter action expert uses conditional flow matching to refine Gaussian noise into multi-step motor commands, conditioned on the fused representation.

Training recipe

First, adapt V-JEPA2 with masked latent prediction on LIBERO, RoboTwin, and RealSource World data. Then freeze the adapted encoder and jointly fine-tune the VLM fusion layers and action expert. The VLM and action expert are initialized from π_0.5.

§ III-D ↗

03 / Experiments

From simulation to the real world.

Two simulation benchmarks. Three physical platforms. Eleven real-world tasks. Results below are those reported in the supplied paper.

LIBERO

Success rate (%) across four task suites; averaged over three random seeds.

Table I · p. 5 ↗

LIBERO success rates in percent. Bold values are the best in each column.
Model	Spatial	Object	Goal	Long	Average
Diffusion Policy	77.8	91.2	67.5	49.8	71.6
Octo	78.9	85.7	84.6	51.1	75.1
OpenVLA	84.7	88.4	79.2	53.7	76.5
SpatialVLA	88.2	89.9	78.6	55.5	78.1
CoT-VLA	87.5	91.6	87.6	69.0	83.9
π₀	96.8	98.8	95.8	85.2	94.2
π_0.5	98.8	98.2	98.0	92.4	96.9
OpenVLA-OFT	97.6	98.4	97.9	94.5	97.1
BeliefVLA Ours	98.8	98.6	97.3	94.2	97.2

Highest average among the compared methods. Bold marks the best value in each column, including ties.

RoboTwin

Average success rate (%) across eight representative bimanual tasks.

Table II · p. 6 ↗

RoboTwin average success rates in percent
Model	Easy	Hard
Diffusion Policy	24.4	1.0
ACT	20.5	1.3
RDT	14.3	4.4
π₀	20.3	8.1
π_0.5	33.1	14.1
BeliefVLA Ours	40.9	21.0

50 clean demonstrations per task; 100 evaluation rollouts per task in each setting. Easy matches the training environment; Hard introduces domain randomization. No in-domain large-scale pretraining. Gains over π_0.5: +7.8 pp (Easy), +6.9 pp (Hard).

Real-world manipulation

30 independent rollouts per setting with matched training data and fine-tuning protocols.

Figure 4 · p. 7 ↗

Real-world success rates in percent and absolute gains in percentage points
Task	π_0.5	BeliefVLA
General manipulation
Fold box	66.7	73.3
Organize pen	56.7	73.3
Stack cups	86.7	93.3
Stack cubes	66.7	80.0
Fold cloth	83.3	83.3
Uncap pen	50.0	66.7
General average +10.0 pp	68.3	78.3
Observation perturbations
Dynamic adaptation	43.3	66.7
Field-of-view occlusion	43.3	53.3
Illumination variation	63.3	80.0
Spatial perturbation	46.7	66.7
Background variation	63.3	70.0
Perturbation average +15.4 pp	52.0	67.3

Values are success rates (%). Aggregate BeliefVLA rates and gains follow § IV-B. The π_0.5 averages use the success counts implied by Figure 4 and 30 rollouts per task, rounded after averaging. pp denotes percentage points.

Paper Figure 3 shows six general manipulation tasks and five perturbation settings on Realman, ARX R5, and AgileX PIPER platforms. — Figure 3. Six general manipulation tasks and five observation-perturbation settings across Realman, ARX R5, and AgileX PIPER platforms.

04 / Robustness

When the observation changes.

Five controlled perturbations probe temporal memory, action-relevant representations, and physical priors.

A person hands a moving water bottle to the Realman robot.

Temporal memory

Dynamic adaptation

The target moves continuously during a human-to-robot bottle handover. Observation history supplies context about its motion.

66.7%BeliefVLA successπ_0.5: 43.3%

A checkerboard calibration board replaces the plain background behind the robot.

Action-relevant representations

Background variation

The background changes from the training curtain to a textured calibration board, testing resilience to irrelevant visual changes.

70.0%BeliefVLA successπ_0.5: 63.3%

Robot arms and target objects after a shift in their relative positions.

Physical priors

Spatial perturbation

The initial relative position of the robot base and target object is shifted, testing spatial generalization under changed geometry.

66.7%BeliefVLA successπ_0.5: 46.7%

The robot workspace under brighter natural illumination.

The same robot workspace under darker natural illumination.

Action-relevant representations

Illumination variation

Morning, noon, and evening trials test robustness to natural lighting fluctuations across the same workspace.

80.0%BeliefVLA successπ_0.5: 63.3%

A yellow garment obscures the view of the robot during cloth folding.

Temporal memory

Field-of-view occlusion

A large garment intermittently obstructs the base camera during complex cloth folding, making the current observation incomplete.

53.3%BeliefVLA successπ_0.5: 43.3%

Success rates from Figure 4(b), with 30 independent rollouts per setting. Experimental protocol ↗ (PDF, opens in a new tab)

05 / Analysis

What makes the difference.

Ablations isolate the belief representation, domain adaptation, and multimodal fusion strategy.

Ablation study from Table III. All values are success rates in percent.
Model variant	RoboTwin	LIBERO	Cube stack
Without V-JEPA2 (π_0.5)	33.1	96.9	66.7
Without domain adaptation	36.0	97.1	73.3
Causal, belief-isolated fusion	17.0	94.5	46.7
BeliefVLA Full model	40.9	97.2	80.0

Success rate (%). RoboTwin reports the average over eight tasks; Cube stack is a real-world task. Table III · p. 7 ↗ (opens in a new tab)

Inference efficiency

≈20 Hz / ≈50 ms

The paper reports approximately 20 Hz pure inference for BeliefVLA, compared with approximately 25 Hz for π_0.5. This is model inference frequency, rather than an end-to-end robot control guarantee.

Scope & limitations

A finite window of history.

Belief inference uses a fixed-length observation window, which can limit longer-horizon memory. The additional V-JEPA2 encoder also increases training and deployment cost. Persistent memory is a direction for future work.

Reference

Read. Reproduce.
Build on it.

Find the full method, protocols, and comparisons in the manuscript.

Read the full paper

BibTeX Manuscript

@misc{liu2026beliefvla,
  title = {BeliefVLA: Implicit Belief-State Driven
    Vision-Language-Action Models for
    Robust Manipulation},
  author = {Liu, Yu and Huang, Tianlv and Han, Wei
    and Guo, Hetian and Miao, Chengyu and Yan, Fei
    and Fan, Zipei and Song, Xuan},
  year = {2026},
  note = {Manuscript}
}

Manuscript citation; no publication venue or DOI is specified.

BeliefVLA: Implicit Belief-State Driven Vision-Language-Action Models for Robust Manipulation