Researchers have identified the key ingredients for training AI agents that can handle multiple complex tasks, not just excel at a single benchmark. A new study from the OpenThoughts-Agent project reveals how to systematically curate training data for agentic language models, the AI systems that can use tools, navigate computers, and reason over long sequences of actions. The team conducted over 100 controlled experiments to understand what makes training data effective, then assembled a dataset of 100,000 examples that outperforms existing open-source approaches.

The research addresses a critical gap in AI development. While companies like OpenAI and DeepSeek have released powerful agentic models, they rarely explain how they prepared the training data that makes these systems work. Most open-source efforts focus on a single benchmark, leaving researchers uncertain about how to build agents that generalize across different types of tasks. The OpenThoughts-Agent project changes that by releasing not just the trained model, but the entire data pipeline, experimental results, and training sets publicly.

What Makes Training Data for AI Agents Different?

Training an AI agent is fundamentally different from training a model that simply answers questions. Agents need to learn how to break down complex problems, use tools like code editors or terminals, and recover from mistakes. This requires a different kind of training data. The researchers tested multiple variables across their pipeline to understand which factors matter most.

The team's experiments revealed several surprising findings about what works:

• Instruction Quality Matters Most: The choice of instructions given to the model during training ranks among the most important factors, similar to what researchers found with reasoning models.
• Teacher Models Aren't Interchangeable: The model that performs best on benchmarks doesn't necessarily produce the best training data for teaching other models.
• Execution Traces Drive Learning: Filtering training data to keep examples where the model takes more steps improves the resulting datasets.
• Diversity Beats Repetition: Repeating the same data sources leads to diminishing returns, so expanding the variety of data sources produces better results.

These insights come from rigorous experimentation. The researchers fine-tuned the Qwen3-32B model, a 32-billion-parameter language model, on their curated dataset of 100,000 examples. The results were significant: the model achieved 44.8% average accuracy across seven different agentic benchmarks, a 3.9 percentage point improvement over the previous best open-source approach, Nemotron-Terminal-32B, which scored 40.9%.

How Does This Compare to Existing Approaches?

The performance gains are most dramatic on specific benchmarks. On SWE-Bench Verified, a test that measures how well models can resolve GitHub issues, the new model achieved 54.0% accuracy compared to 41.9% for the previous best open model. On Terminal-Bench 2.0, which tests command-line reasoning, it scored 26.2% versus 25.1%.

What's particularly noteworthy is that the training data shows strong scaling properties. In compute-controlled comparisons, where researchers train models on different amounts of data while keeping computational resources equal, the OpenThoughts-Agent dataset outperformed alternative open datasets at every training set size. This means the data is not just good because there's more of it, but because it's fundamentally higher quality.

The researchers also explored reinforcement learning, a technique where models learn by receiving rewards for good behavior. They created a new curated reinforcement learning dataset and tested it by training an 8-billion-parameter model in two stages: first with supervised fine-tuning, then with reinforcement learning. This two-stage approach outperformed their best single-stage 8-billion-parameter model and beat other existing models at that scale.

How to Use These Findings for Future AI Research

• Access the Open Pipeline: Researchers can visit openthoughts.ai to download the complete data curation pipeline, allowing them to understand and reproduce the methodology without starting from scratch.
• Leverage the Training Sets: The 100,000 curated examples are publicly available, enabling other teams to fine-tune their own models using the same high-quality data that produced state-of-the-art results.
• Study the Ablation Experiments: All experimental data from the 100+ controlled tests are released, showing exactly which design choices matter and which don't, accelerating future research iterations.
• Build on Multiple Benchmarks: Rather than optimizing for a single test, researchers can use the methodology to train agents that perform well across diverse agentic tasks like software engineering, terminal commands, and financial analysis.

The broader significance of this work lies in democratizing agentic AI development. Until now, building state-of-the-art agents required either access to proprietary data or the resources to generate massive amounts of training examples. By releasing the pipeline and datasets, the OpenThoughts-Agent project enables smaller research teams and academic groups to contribute to this frontier. The project builds on prior work called OpenThoughts, which focused on reasoning models, but extends those principles specifically to agents that interact with tools and environments.

The research also highlights an important trend in AI: the shift from simply scaling up models to carefully engineering the data that trains them. As models have grown larger, the quality and composition of training data has become increasingly important. This work provides a systematic framework for that data engineering process in the specific domain of agentic AI, an area that's rapidly becoming central to practical AI applications.

" }