A new mathematical reasoning benchmark reveals stark performance gaps between leading AI models, with GPT-5 achieving 95.8% accuracy on undergraduate-level graph theory problems while Claude Sonnet 4.6 and Gemini 2.5 Flash-Lite struggle significantly as complexity increases. The study, called GTBench, tested five prominent large language models (LLMs), which are AI systems trained on vast amounts of text to understand and generate human language, across 63 graph theory problems ranging from basic definitions to graduate-level proofs.

What Did the GTBench Study Actually Test?

Researchers designed GTBench to evaluate how well these AI models function as mathematical research assistants. The benchmark organized 63 problems into three difficulty tiers sourced from respected academic materials, including Diestel's Graph Theory textbook.

• Group 1 (Undergraduate Level): Problems dealing with basic definitions and fundamental properties of graph theory, testing foundational knowledge and straightforward application of concepts.
• Group 2 (Intermediate Level): Algorithm tracing and structural reasoning tasks that require models to follow logical sequences and understand how graph structures behave under specific conditions.
• Group 3 (Graduate Level): Proof construction challenges requiring models to build rigorous mathematical arguments and demonstrate deep reasoning about complex graph properties.

How Did Each Model Perform on Mathematical Reasoning?

The results reveal a clear hierarchy in mathematical capability. GPT-5 nearly aced Group 1 problems with 95.8% zero-shot accuracy, meaning it solved them without any examples or hints. The model retained substantial competence when tackling graduate-level proofs, achieving 82% accuracy on Group 3.

The other models tested in the benchmark, however, faltered significantly as difficulty increased. Claude Sonnet 4.6 and Gemini 2.5 Flash-Lite showed meaningful performance drops moving from undergraduate to graduate-level problems. Most strikingly, Llama 3.3 70B scored zero percent on Group 3 when evaluated by human judges, suggesting fundamental limitations in reasoning completeness for complex mathematical arguments.

What's driving these discrepancies? Analysis indicates that most errors in Groups 1 and 2 stem from correct algorithm selection but flawed execution, meaning the models understood what to do but made mistakes in carrying it out. Group 3 problems uncovered deeper failures in reasoning completeness and revealed a systematic rift between human evaluators and automated judgments, particularly when proofs were verbose or nearly complete.

Why Should You Care About These Performance Gaps?

These results matter because they expose a critical limitation in AI reliability for specialized domains. While GPT-5 shows promise in mathematical reasoning, the overall lackluster performance of other models suggests that AI isn't yet ready to replace human expertise in higher-level mathematics. The disagreement between human and AI judges on complex proofs raises questions about the evaluation criteria themselves, making it difficult to trust these models in academic settings without human oversight.

For organizations considering deploying AI as a research assistant or educational tool, the benchmark reveals that current models have uneven capabilities. They excel in domains where they received extensive training but struggle when pushed into specialized territory. This pattern suggests that teams should not rely on a single AI model for all mathematical or reasoning tasks, and should maintain human verification for high-stakes applications.

How to Match AI Models to Your Specific Needs

• For Complex Mathematical Reasoning: GPT-5 demonstrated significantly higher accuracy than competitors in the GTBench study, particularly on graduate-level proofs where it achieved 82% accuracy. However, all models have limitations in advanced mathematics, so human expertise remains essential for verification and guidance.
• For Coding and Long Document Processing: Claude Opus 4.7, released in April 2026, can process up to 1 million tokens, roughly equivalent to 750,000 words at once, making it superior for handling massive codebases and long documents according to Source 1. This capability complements coding tasks, though the GTBench study tested Claude Sonnet 4.6, not Opus 4.7.
• For Ecosystem Integration: Gemini 3.1 offers deep integration with Google Workspace applications like Gmail, Google Docs, and Google Sheets according to Source 1. If your team already uses Google's ecosystem, native integration reduces setup friction, though the GTBench study tested Gemini 2.5 Flash-Lite, not Gemini 3.1.

What Do Experts Say About AI Reliability in Mathematics?

The GTBench study raises fundamental questions about whether these models can serve as reliable partners in mathematical research. The disagreement between human and AI judges on Group 3 problems highlights that evaluation methods themselves might introduce bias, making it difficult to assess true model capabilities. Before deploying LLMs as mathematical assistants in academic or professional settings, organizations must critically examine both the models' capabilities and the frameworks used to judge them.

The broader implication is clear: these AI models are not interchangeable. GPT-5 dominates mathematical reasoning based on the GTBench results, but other models excel in different areas. Claude Opus 4.7 leads in document processing and long-context tasks according to Source 1, while Gemini 3.1 leads in ecosystem integration. Teams should test each model on their specific workflows before committing to a single platform, particularly when mathematical accuracy or reasoning completeness is critical.