NVIDIA Research has launched SpatialClaw, a pioneering framework that enables spatial reasoning without the need for training. This innovation addresses a notable gap in vision-language models (VLMs), which often struggle with understanding object locations, relationships, and movements in three-dimensional space. Unlike traditional models, SpatialClaw modifies the action interface used by agents to access perception tools, suggesting that the interface itself is a limiting factor. By treating code as this interface, SpatialClaw achieves an impressive average accuracy of 59.9% across 20 benchmarks, surpassing the performance of the recent spatial agent SpaceTools by 11.2 points.
SpatialClaw operates through a stateful Python kernel, preloaded with input frames and various primitives. It utilizes perception tools that are standard Python functions, producing outputs such as masks and depth maps. The framework is designed to be training-free, maintaining consistent prompts and tool sets across all evaluations. The research team conducted extensive testing across multiple categories, demonstrating significant improvements over baseline models. The results highlight the effectiveness of SpatialClaw, particularly in dynamic tasks requiring complex geometric reasoning, making it a valuable tool for applications in robotics, video analysis, and indoor scene understanding.