A new framework called SkillGym converts human-written workflows into executable training environments, allowing AI models to internalize procedural knowledge rather than just following external instructions. Researchers constructed 2,756 training environments across 12 major categories and collected 8,364 verified successful task trajectories, enabling supervised fine-tuning that produced a 35-billion-parameter model exceeding performance benchmarks set by much larger competitors.

Why Are AI Models Struggling With Real-World Tasks?

Large language models (LLMs) have become remarkably good at reasoning, following instructions, and using tools. Yet when deployed in real-world scenarios, they often stumble. The problem isn't a lack of intelligence; it's a lack of procedural competence. Real-world problem solving demands more than isolated reasoning steps. It requires following domain-specific workflows, coordinating multiple tools over extended sequences, recovering from failures, and verifying that outcomes actually meet task requirements.

Traditionally, AI systems access human expertise as external knowledge at inference time, meaning they retrieve and follow instructions when needed. But this approach has limitations. The model's effectiveness depends on how well it retrieves the right instructions, manages context, and faithfully follows them. More importantly, repeated access to external instructions doesn't necessarily translate into reusable capabilities the model can apply independently.

What Makes SkillGym Different From Previous Approaches?

SkillGym takes a fundamentally different approach. Instead of treating human-written workflows as external reference material, the framework transforms them into concrete, executable training problems. The system uses category-specific templates to convert abstract skill descriptions into actual tasks with real inputs, clear objectives, and interactive protocols. Crucially, it includes code-based outcome verifiers that check whether the model actually completed the task correctly, not just whether it followed the instructions.

The framework also employs contrastive execution, running tasks with and without the target skill to identify which tasks genuinely depend on that skill. This ensures that training focuses on meaningful procedural knowledge rather than spurious correlations. The result is a dataset of verified workflows that support both supervised fine-tuning and reinforcement learning with outcome-based rewards.

How to Build AI Agents With Internalized Skills

• Convert Workflows to Executable Tasks: Transform human-written skill documents into concrete problems with specific inputs, interaction interfaces, and measurable success criteria using category-specific templates.
• Verify Outcomes With Code-Based Checkers: Implement automated verification systems that confirm whether the model actually achieved the task objective, not just whether it followed instructions correctly.
• Assess Empirical Skill Dependence: Run contrastive executions with and without the target skill to identify which tasks genuinely require that procedural knowledge for successful completion.
• Collect Verified Trajectories: Gather successful execution sequences from multiple models and harnesses, logging detailed interaction data to create high-quality training material.
• Apply Supervised Fine-Tuning and Reinforcement Learning: Train models on verified workflows using both supervised learning from successful trajectories and reinforcement learning with outcome-based rewards.

What Results Did the SkillGym Training Produce?

The researchers constructed and released 2,756 environments spanning software development, business operations, data processing, and other real-world workflows across 12 major categories and 63 subcategories. They collected 8,364 successful trajectories from multiple teacher models, with each trajectory averaging 49 tool calls and over 60,000 logged text tokens.

When they fine-tuned Qwen 3.5-35B, a 35-billion-parameter open-source model, on these verified trajectories, the results were striking. Under the Claude Code harness, the model improved by 199 Elo rating points on GDPval-AA v2, a professional task benchmark. It gained 19.10 percentage points on Terminal-Bench 2.1, a command-line execution benchmark, and improved by 28.13 points on SkillsBench v1.1 when skills were available, and 14.68 points without them.

Most remarkably, the resulting SkillGym-Agent reached 51.47% on skill-assisted SkillsBench, exceeding reported scores for Claude Sonnet 4.6, GPT-5.4 Mini, and DeepSeek V4 Pro. Even without access to external skills at inference time, the model surpassed its base configuration under both harnesses, suggesting that the procedural knowledge had been genuinely internalized.

What Does This Mean for AI Development?

The SkillGym results suggest a fundamental shift in how AI models acquire real-world competence. Rather than training models to reproduce skill descriptions or follow external instructions, the framework makes the execution of underlying workflows the source of learning experience. This distinction matters because it means models develop reusable procedural knowledge that persists even when external guidance is unavailable.

The framework also demonstrates that smaller models, when trained on verified procedural knowledge, can outperform larger models that lack such training. This has implications for compute efficiency and deployment costs. Organizations may be able to achieve better real-world performance with smaller, more specialized models rather than relying on increasingly large general-purpose systems.

The release of 2,756 training environments and 8,364 verified trajectories provides a shared resource for the research community, potentially accelerating progress in agent-based AI systems. By connecting procedural knowledge, executable tasks, and interaction experience in a single training resource, SkillGym offers a template for how human expertise can be systematically converted into model capabilities.