Vision-language models (VLMs), which are AI systems trained to understand both images and text, can now pinpoint the location of photos with remarkable accuracy, reaching 90% on challenging geolocation tasks. However, a new benchmark reveals a critical gap: while these models perform nearly as well as world-class human geolocators, they cannot reliably explain how they arrive at their answers, raising questions about whether they truly understand geography or are pattern-matching in ways humans cannot detect.

How Do AI Models Compare to the World's Best Geolocators?

Researchers at Georgia Tech partnered with Radu Casapu, the reigning GeoGuessr World Champion, to create the first rigorous benchmark for testing how well VLMs can identify locations from random street-view images. GeoGuessr is a popular online game where players guess the location of photos from Google Street View by analyzing clues like road signs, vegetation, architecture, and infrastructure.

Casapu and two other top geolocators were given 500 images each to analyze. They achieved a 96% accuracy rate while carefully documenting their reasoning for each guess. When the same images were fed to GPT-5, Gemini, Llama, and Qwen, the highest-performing model reached 90% accuracy, a gap of only six percentage points.

"VLMs are surprisingly good at geolocation right out of the box, even when they're not trained to be good at it," said James Hays, a professor at Georgia Tech's School of Interactive Computing.

James Hays, Professor, School of Interactive Computing, Georgia Tech

The research team, led by Hays and associate professors Alan Ritter and Wei Xu, created GeoRC, a dataset of 800 "ground truth" reasoning chains that documents how expert humans solve geolocation puzzles. This benchmark will help the research community evaluate not just how accurate VLMs are, but whether they can articulate their logic.

Why Does the Reasoning Gap Matter More Than Accuracy?

The most striking finding was not about accuracy but about transparency. While Casapu and the other human experts provided clear, step-by-step explanations for their guesses, the VLMs either offered vague reasoning or no reasoning at all. When they did explain themselves, experts found suspicious patterns.

"When experts have audited these reasoning chains, we've noted many suspicious or hallucinated attributes. When they hallucinate a geographic property, why is it so often consistent with the correct guess? I believe they're not revealing the true reasoning pathway that they used to determine the image was Italy. They're just implicitly recognizing that it was Italy for many reasons, then hunting for evidence to support that. Some of the things they say are true and supported by the image, and some are fabrications," explained Hays.

James Hays, Professor, School of Interactive Computing, Georgia Tech

This finding echoes concerns raised when OpenAI released GPT-4 Vision. The company initially claimed the model was not particularly good at geolocation, citing privacy concerns. However, Hays and Ritter's research contradicted that claim, showing that VLMs are actually state-of-the-art at image geolocation tasks.

How Expert Geolocators Approach the Problem Differently

Casapu describes his methodology as a "top-down approach." He begins by identifying infrastructure clues that are specific to a country or region, such as road designs or electricity poles, which tend to remain consistent within a country. Once he narrows down the location geographically, he then looks for more nuanced details like vegetation, specific landscapes, and architectural styles.

• Infrastructure Analysis: Road signs, electricity poles, and highway designs are examined first because they are standardized within countries and provide broad geographic clues.
• Regional Refinement: After identifying a country, experts analyze vegetation patterns, forest types, and landscape features that are specific to regions or provinces.
• Architectural Details: Building styles, roof shapes, wall colors, and construction materials are examined last because they are highly nuanced and region-specific.

Casapu noted that there may be only a handful of GeoGuessr players worldwide who can currently outperform top-tier VLMs, and that gap is likely to close soon.

"I think it could be more difficult playing against these models than playing against another human because a human has the possibility of making mistakes at the top level. If a well-trained model has that level of consistency, that is far beyond a normal person, and it would be much more difficult to beat," Casapu noted.

Radu Casapu, Master's Student in City and Regional Planning, Georgia Tech

What This Means for AI Transparency and Trust

The research highlights a growing concern in the AI community: models can achieve impressive accuracy without providing trustworthy explanations for their decisions. This is particularly important for applications where understanding the reasoning is as critical as the answer itself, such as medical diagnosis, legal decisions, or security assessments.

Hays has been studying geolocation through machine learning for nearly two decades, beginning with his Ph.D. work at Carnegie Mellon University in 2008, when he was the first researcher to apply machine learning to geographic location estimation from images. The new GeoRC benchmark represents a significant step forward in measuring not just model performance, but model interpretability.

The paper on GeoRC, co-authored by Hays, Ritter, Xu, Casapu, and several other researchers, will be presented at the 64th Annual Meeting of the Association for Computational Linguistics (ACL) in San Diego next week.

For Casapu personally, the collaboration with Hays proved valuable beyond the research itself. Working alongside VLMs and documenting his reasoning helped him refine his own geolocation strategy, ultimately contributing to his preparation for defending his GeoGuessr World Championship title in September 2026.