4 ms·
The ability to accurately estimate distances from RGB image input is just at the frontier of current AI model capabilities. Nonetheless, distance estimation is
by backflippinbozo 1y ago
The ability to accurately estimate distances from RGB image input is just at the frontier of current AI model capabilities.
Nonetheless, distance estimation is a critical for perception and planning in embodied AI applications like robotics which must navigate around our 3D world.
We just released SpaceThinker, a 3B open-weight VLM designed specifically for spatial reasoning tasks like distance and size estimation from RGB images. It’s small and fast enough for on-device use, trained entirely on open-source data/code.
* Model: https://huggingface.co/remyxai/SpaceThinker-Qwen2.5VL-3B https://huggingface.co/remyxai/SpaceThinker-Qwen2.5VL-3B
* Data: https://huggingface.co/datasets/remyxai/SpaceThinker https://huggingface.co/datasets/remyxai/SpaceThinker
On the QSpatial++ benchmark, SpaceThinker sits between GPT-4o and Gemini 2.5 Pro in performance, see this comparison table: https://huggingface.co/remyxai/SpaceThinker-Qwen2.5VL-3B#qspatial-comparison-table-42525 https://huggingface.co/remyxai/SpaceThinker-Qwen2.5VL-3B#qsp...
Interesting finding: By switching model name in this colab, using the non-reasoning variant SpaceQwen (https://huggingface.co/remyxai/SpaceQwen2.5-VL-3B-Instruct https://huggingface.co/remyxai/SpaceQwen2.5-VL-3B-Instruct), you'll find using the step-by-step reasoning prompt actually hurts performance, challenging the convention that reasoning models don't benefit from complex instructions the way non-reasoning models do.
Additional resources:
* https://github.com/andrewliao11/Q-Spatial-Bench-code/blob/main/prompt_templates/spatial_prompt_steps.txt https://github.com/andrewliao11/Q-Spatial-Bench-code/blob/ma...
* https://huggingface.co/blog/NormalUhr/deepseek-r1-explained#74-prompt-engineering-sensitivities https://huggingface.co/blog/NormalUhr/deepseek-r1-explained#...
Feedback, suggestions, and collaborators are welcome!
If you're interested in contributing, we open-sourced VQASynth—our implementation of the SpatialVLM approach for generating VQA-style datasets from images. VQASynth was used to create the SpaceThinker dataset, which powered the fine-tuning of the SpaceThinker model showcased here.