OpenAI's latest reasoning model, GPT-5, has achieved a remarkable milestone in competitive programming, reaching the 81st percentile when compared directly to elite human contestants. However, a comprehensive new benchmark study reveals that even the most advanced AI models still fall short of the world's top programmers, and that raw reasoning power alone does not guarantee success on the hardest algorithmic challenges.

Researchers at multiple institutions have introduced LiveOIBench, a new competitive programming benchmark featuring 403 expert-curated problems drawn from 14 International Informatics Olympiads held between 2023 and 2025. Each problem includes an average of 60 official test cases, creating one of the most rigorous evaluation frameworks for coding AI ever assembled. The benchmark allows direct comparison between AI models and human contestants who competed in these prestigious competitions.

How Do Modern Reasoning Models Compare to Human Programmers?

• GPT-5 Performance: Achieves an 81.76th percentile ranking against elite human contestants, demonstrating strong overall capability while remaining below the very top performers.
• Open-Source Model Progress: GPT-OSS-120B reaches the 60th percentile, showing significant gains from additional reasoning tokens and narrowing the gap with proprietary models.
• Efficiency Advantage: GPT-5 reaches its high performance with fewer than 20,000 reasoning tokens, positioning it on the efficiency frontier compared to models requiring substantially more computational resources.

The study evaluated 34 leading open-source and proprietary models, revealing a substantial performance advantage for proprietary systems. Among open-weight alternatives, Seed-OSS achieved the 54th percentile and Qwen3-32B reached the 42nd percentile, both showing meaningful improvements when given additional reasoning tokens to work with.

Why Do Reasoning Models Struggle With Certain Algorithm Types?

One of the most striking findings from LiveOIBench concerns the limitations of current reasoning models on specific algorithmic challenges. Despite their overall strength, these models show particular weakness in dynamic programming problems, which require creative pattern recognition and recursive decomposition. This suggests that the reasoning approaches these models use may not be optimally suited for all types of algorithmic thinking.

Detailed analysis of reasoning traces from high-performing models revealed an important insight: the best-performing systems strategically allocate more computational tokens to focused problem analysis rather than excessive exploration. This finding challenges the assumption that more reasoning is always better. Instead, carefully managed reasoning behaviors appear crucial for robust performance on genuinely difficult tasks. Models that waste tokens on unfocused exploration tend to underperform compared to those that concentrate their reasoning power on precise problem analysis.

What Makes LiveOIBench Different From Previous Coding Benchmarks?

Earlier coding benchmarks like HumanEval and MBPP have become saturated, with modern models solving them too easily to provide meaningful evaluation. Subsequent benchmarks based on competitive programming platforms like Codeforces introduced greater difficulty but suffered from significant limitations. These included incomplete test suites that led to false positives, insufficient difficulty granularity, reliance on external APIs that limited reproducibility, and heavy reliance on coarse pass-or-fail metrics that obscured nuanced model capabilities.

LiveOIBench addresses these gaps through several key features. The benchmark sources problems and test cases directly from official Informatics Olympiad contests, eliminating the high false-positive rates common in previous benchmarks. It provides fine-grained subtask scoring rubrics that enable nuanced performance analysis beyond simple pass rates. The benchmark includes official results from human competitors, enabling direct human-model comparisons. Finally, it features a fully offline, self-contained evaluation system that removes reliance on external APIs and enhances research accessibility.

The researchers conducted extensive analysis to verify that their benchmark remains free from data contamination, a concern that has plagued previous AI evaluation studies. They found minimal correlation between model performance and problem release dates, task familiarity, or solution familiarity, suggesting that the benchmark genuinely measures model capability rather than memorization of training data.

As AI models continue advancing through inference-time scaling techniques, which allow models to spend more computational resources thinking through problems, benchmarks like LiveOIBench become increasingly important for understanding where these systems truly stand relative to human expertise. The results suggest that while AI has made extraordinary progress in coding tasks, the gap between state-of-the-art models and elite human programmers remains meaningful, particularly on problems requiring creative algorithmic insight.