A team of researchers has cracked a fundamental problem in AI self-improvement: how to train language models to get better at tasks where there's no single correct answer. Until now, the most effective method for scaling AI reasoning, called Reinforcement Learning with Verifiable Rewards (RLVR), only worked for domains like mathematics and coding, where a computer can definitively check if an answer is right or wrong. For open-ended tasks such as creative writing, summarization, or brainstorming, AI systems have relied on slower, more expensive human judges or imperfect reward models. A new approach called Reinforcement Learning with Self-Verifiable Rewards (RLSVR) changes that equation.

What's the Problem With Training AI on Open-Ended Tasks?

The challenge has been straightforward but stubborn. When you ask an AI to write a poem or summarize an article, there's no objective "correct" answer. Unlike a math problem with a definitive solution, quality is subjective. This means researchers have had to rely on human preference judgments, external reward models trained by humans, or LLM-based judges to score outputs. Each approach introduces problems: human evaluators are expensive and slow, reward models can be biased, and LLM judges add significant computational costs and may themselves be unreliable.

Meanwhile, RLVR has proven remarkably effective at scaling AI reasoning in verifiable domains. The method allows models to optimize themselves at massive scale by receiving clear, automatic feedback. But that same power couldn't be applied to the majority of real-world tasks, which are inherently open-ended and subjective.

How Does the New Self-Verifiable Rewards Method Work?

The researchers' solution draws inspiration from self-supervised learning, a technique where AI systems learn by creating their own training signals from raw data. The key insight is elegant: transform an open-ended task into a verifiable game or environment where rewards emerge naturally from the rules of the game itself.

The team instantiated this idea with a method called SpyRL, inspired by the social deduction game "Who Is the Spy?" Here's how it works in practice: multiple AI agents receive asymmetric information, complete the same target task, and then vote to identify a designated spy. Because the spy's identity is predetermined before the game starts, the voting outcome is fully verifiable. The clever part is that successful identification of the spy remains closely correlated with output quality, so the game's reward signal actually incentivizes better performance on the original task.

This task-transformation approach sidesteps the need for human judges or external reward models entirely. The game itself generates the verification signal automatically.

What Tasks Did Researchers Test This On?

The researchers evaluated SpyRL across three different types of open-ended tasks to demonstrate its versatility. The experiments covered:

• Text Summarization: Training models to condense longer documents into concise summaries while preserving key information.
• Creative Writing: Improving the quality and coherence of AI-generated stories, essays, and other creative content.
• Mathematical Reasoning: Testing whether the method also maintains or improves performance on verifiable reasoning tasks where traditional RLVR already works well.

Across all three domains, SpyRL outperformed existing self-improvement methods on the non-verifiable tasks (summarization and creative writing) and delivered consistent gains even on verifiable reasoning tasks. This dual performance is significant because it shows the method doesn't sacrifice performance on domains where verification is already possible.

Why Does This Matter for AI Development?

The implications are substantial. RLVR has been one of the most important techniques for scaling reasoning in recent AI breakthroughs, enabling models to optimize themselves at scale without constant human feedback. By extending this capability to open-ended tasks, researchers have removed a major bottleneck in AI self-improvement. Instead of being limited to math, coding, and other verifiable domains, the technique can now apply to the vast majority of real-world language tasks.

This also addresses practical concerns about cost and bias. Human evaluation is expensive; reward models can encode human biases; and LLM judges add latency and computational overhead. A self-verifiable reward system that emerges from task transformation eliminates these intermediaries, making large-scale optimization feasible for tasks that were previously difficult to scale.

How Can Researchers Apply This Approach to New Tasks?

The researchers have released both their models and code, enabling other teams to build on the work. The core principle is task transformation: taking an open-ended objective and reformulating it as a game or environment with built-in verification. To apply this framework to a new task, researchers would need to:

• Identify the Core Objective: Clearly define what quality or success looks like for the task, even if it's subjective.
• Design a Verifiable Game: Create an auxiliary game or environment where agents interact in a way that naturally produces verifiable outcomes tied to the original objective.
• Ensure Reward Correlation: Verify that winning the game is strongly correlated with performing well on the original task, so the game's feedback signal actually improves the desired behavior.
• Scale the Training: Use the verifiable rewards from the game to train models at scale, similar to how RLVR works in math and coding domains.

The SpyRL example demonstrates one instantiation of this principle, but the underlying idea is general and could be adapted to many other open-ended tasks.

The research was presented at COLM 2026, a major conference in machine learning, and represents a meaningful step forward in making AI self-improvement more scalable and practical across a broader range of real-world applications.