Vision-language models (VLMs) like GPT-4V and Gemini perform remarkably well in controlled settings, but they collapse when faced with the messy realities of actual deployment: motion blur, low light, rain, and compression artifacts. A new benchmark called SpaceDG reveals that current state-of-the-art models suffer significant performance drops under these common visual degradations, raising urgent questions about whether today's most advanced AI systems are truly ready for embodied robots, autonomous vehicles, and real-world applications.

The problem is fundamental: existing benchmarks for vision-language models assume pristine, high-resolution, well-lit images. But in the physical world, cameras operate under constraints. Robots navigate dimly lit warehouses. Autonomous vehicles encounter rain and fog. Security cameras capture compressed video feeds. Yet researchers have largely ignored how these real-world conditions affect spatial reasoning, object detection, and visual understanding.

Why Do Vision Models Fail When Images Aren't Perfect?

Researchers at leading institutions constructed SpaceDG, the first large-scale benchmark designed specifically to test how vision-language models handle degraded visual inputs. The dataset includes approximately 1 million question-answer pairs across more than 160,000 images, with nine types of realistic degradations embedded using physically grounded simulation techniques.

The findings are sobering. When evaluated on SpaceDG-Bench, a curated set of 1,102 unique questions spanning 11 reasoning categories, all 25 tested models, including leading proprietary and open-source systems, showed consistent and substantial performance drops under visual degradation. The research demonstrates that fine-grained spatial tasks, such as object counting and boundary detection, are particularly vulnerable to degraded visual evidence, while certain geometric reasoning tasks remain more robust.

Interestingly, humans also suffer clear performance drops under degraded conditions. This suggests that the solution is not simply to make AI systems mimic human perception, but rather to develop models that learn degradation-aware spatial knowledge to handle diverse real-world visual inputs more effectively.

What Types of Visual Degradation Matter Most?

The SpaceDG benchmark simulates nine representative degradation types across four categories:

• Optical and Dynamic Degradations: Defocus blur, lens distortion, and motion blur from camera movement or fast-moving subjects.
• Meteorological Degradations: Haze and water droplets that obscure visual clarity in outdoor environments.
• Photometric Degradations: Low-light conditions and overexposure that reduce visual contrast and detail.
• Digital Degradations: JPEG compression artifacts and low-resolution imagery common in transmitted or archived video feeds.

Each degradation is generated from underlying physical formation processes, making the simulations realistic rather than arbitrary corruptions. This approach ensures that the benchmark reflects actual challenges faced by deployed systems.

How Can Vision Models Become More Robust?

The research offers a promising path forward: fine-tuning models on degradation-aware training data substantially improves robustness. When researchers trained models on SpaceDG using supervised fine-tuning, the models not only performed better on degraded images but also maintained or exceeded their original performance on clean, high-quality images. In some cases, models even surpassed human performance on degraded inputs after exposure to this type of training.

This finding suggests that degradation-aware training is not a trade-off but rather a genuine capability enhancement. Models that learn to reason about spatial relationships despite visual imperfections develop more robust and generalizable spatial understanding.

What About Self-Improving Vision Models?

Beyond robustness, another critical challenge facing vision-language models is the cost of human supervision. Training state-of-the-art VLMs requires massive amounts of carefully annotated data, where questions, answers, and reasoning traces must be manually crafted by experts. This bottleneck limits how quickly and affordably models can improve.

A framework called RISE addresses this by enabling vision-language models to improve themselves through self-evolution. Rather than relying entirely on human-annotated data, RISE allows models to autonomously generate questions from unlabeled images and learn to solve them, creating a closed-loop learning system. However, direct application of this approach reveals three major challenges: coarse-grained role alternation delays feedback between question generation and solver adaptation, generated questions degrade in quality over time, and question types collapse toward narrow distributions.

RISE solves these problems through three complementary mechanisms. Fine-grained role alternation shortens the feedback loop between the questioner and solver, improving training efficiency. A quality supervisor constrains question validity and verifies pseudo-label reliability, reducing interference from low-quality questions. Skill-aware dynamic balancing regulates question distribution to prevent collapse toward easy-to-generate categories like math and counting.

Experiments across two VLM backbones and seven benchmarks show that RISE consistently improves base models with broad and sustained gains, suggesting that self-evolution can be a scalable path to capability improvement without relying solely on expensive human annotation.

Why Can't Vision Models Handle Fine-Grained Visual Details Plus Knowledge Search?

Even as models improve at spatial reasoning and self-evolution, another critical gap has emerged: the ability to combine fine-grained visual grounding with external knowledge retrieval. Real-world scenarios often require both skills simultaneously. A tour guide identifying a distant mountain must both locate it precisely in a photograph and recall its historical significance. A shopper comparing prices must spot a small price tag and look up exchange rates.

A new benchmark called Pix2Fact tests this integrated capability using 1,000 high-resolution, 4K-plus images across eight real-world scenarios. Each question requires both detailed visual grounding and deliberate web search for external knowledge. The results are striking: even the most advanced model tested, Gemini-3.1-Pro, achieved only 51.7% accuracy, even when provided with visual ground truth and access to search tools.

Analysis of the failures reveals three core problems. First, models make frequent visual grounding errors even when given visual ground truth, suggesting that the problem is not merely seeing the detail but understanding its relevance to the question. Second, models perform shallow search execution, attempting only a few keyword searches without iterative refinement or re-searching. Third, models struggle to retrieve long-tail, unstructured local information, such as small business hours, local signage, or temporary events that require manual navigation of web pages.

This gap underscores a fundamental limitation in current vision-language models: they cannot reliably link fine-grained visual details with relevant external knowledge through search. Closing this gap will require developing more sophisticated, knowledge-aware vision-language architectures that treat visual perception and information retrieval as deeply integrated capabilities rather than separate tasks.

Steps to Advance Vision-Language Model Robustness

Based on emerging research, several approaches show promise for improving vision-language models:

• Degradation-Aware Training: Incorporate realistic visual degradations into training datasets so models learn to reason effectively despite motion blur, low light, compression, and weather conditions.
• Self-Evolving Frameworks: Implement fine-grained role alternation and quality supervision to enable models to improve from unlabeled data without relying entirely on expensive human annotation.
• Integrated Knowledge-Vision Architectures: Design systems that treat visual grounding and knowledge retrieval as coupled tasks, enabling models to search for information relevant to specific visual details rather than treating them as independent capabilities.

The convergence of these research directions suggests that the next generation of vision-language models will be defined not by performance on pristine benchmarks but by robustness in messy, real-world conditions where visual imperfections, limited supervision, and the need for external knowledge are the norm rather than the exception.