LLMs vs. VLAs: Gemini 3.7 Flash vs. MolmoAct 2

August 24, 2026

Gemini 3.7 Flash and MolmoAct 2, a dedicated vision‑language‑action model trained on this bimanual embodiment, were each given control of YAM arms to complete the following prompt: “Place the right answer on the plate.” On the table: wooden blocks spelling 3 + 4 =, candidate blocks numbered 1, 2, 4, and 7, and an empty plate.

Gemini 3.7 Flash scores 100% on this task. MolmoAct 2 scores 0%.

Embodied Reasoning: “Place the right answer on the plate.”

100 75 50 25 0 0 MolmoAct 2 n=5 100 Gemini 3.7 Flash n=5 robocurve.org

Success rate across five runs per model. A run succeeds only if the block matching the sum ends up on the plate.

The Task

overhead view of the scene: blocks spelling 3 + 4 =, candidate blocks 1 2 4 7, empty plate

The scene

Overhead camera. The equation reads 3 + 4 =; the answer block must go on the plate.

final frame of a MolmoAct 2 run: blocks scattered, nothing on the plate

MolmoAct 2, final frame

Timeout at 1,200 steps. This run dragged the plate across the table; none placed a block.

final frame of a Gemini 3.7 Flash run: block 7 sits on the plate

Gemini 3.7 Flash, final frame

Block 7 on the plate. Every run identified the sum before the first move.

Gemini reasons in text between moves, e.g. “Moving right arm toward block 7 (3+4=7)”, then executes Cartesian end‑effector moves. MolmoAct 2 maps camera frames directly to joint trajectories; it doesn’t understand the task and often proceeds to pick up the plate instead. It didn’t pick up any blocks in any of the 5 runs.

All Runs

ModelResultTime (min)TranscriptRerun (.rrd)
Gemini 3.7 Flashsuccess5.5viewdownload
Gemini 3.7 Flashsuccess8.8viewdownload
Gemini 3.7 Flashsuccess2.4viewdownload
Gemini 3.7 Flashsuccess4.8viewdownload
Gemini 3.7 Flashsuccess3.5viewdownload
MolmoAct 2failure2.8NAdownload
MolmoAct 2failure3.0NAdownload
MolmoAct 2failure3.0NAdownload
MolmoAct 2failure2.6NAdownload
MolmoAct 2failure2.8NAdownload

Technical Specifications

ArmsI2RT YAM bimanual arms (two 6‑DoF arms with parallel grippers), two identical rigs
Camerasfixed overhead Intel RealSense D435 + one Intel RealSense D405 on each wrist, 224×224 RGB
ModelsGemini 3.7 Flash on the Interactions API, effort low, 5 runs; allenai/MolmoAct2‑BimanualYAM served locally (bfloat16, RTX 5090), 5 runs
Instruction“Place the right answer on the plate.” (identical for both models)
ActionsGemini: Cartesian end‑effector position moves, one move per LLM call, arm speed capped at 10%; MolmoAct 2: joint‑space chunks (action horizon 30) at 10 Hz
Limits1,200 control steps per run (both models); Gemini additionally capped at 100 LLM calls per run
Harnessinspect‑robots 0.53.1; collision guardrail off, joint‑effort reporting on, rig facts / formulas / advice documents in the system prompt; full wire capture
Scoringbinary operator judgement per run: the block equal to the sum is on the plate at the end