Open-source coding agents trained on one platform collapse dramatically when deployed on another, according to new research from Huawei Canada and Queen's University. A model that achieved 52.6% accuracy under OpenHands, a popular training environment, plummeted to just 8.4% accuracy on a different platform called OpenCode. The culprit isn't the model itself, but rather the planning conventions baked into it during training.

What's Causing This Massive Performance Drop?

The ecosystem of open-source coding agents has quietly converged on a single training harness, OpenHands, which has become embedded in how these models think and plan. When researchers tested SWE-Lego-Qwen3-32B, a fine-tuned model, they discovered it performed well only under the exact conditions it was trained on. The untrained base model, by contrast, showed consistent performance across different platforms, suggesting the fine-tuning process itself was the problem.

The research identifies two types of planning that get locked into models during training. Explicit planning is when a model produces a step-by-step plan before acting, similar to how Claude Code has a dedicated "Plan Mode." Implicit planning refers to the structural patterns the model learns from how the training platform organizes work, sequences tool calls, and triggers replanning. When a model trained under one platform's planning style encounters a different platform's approach, it becomes disoriented.

How Can Developers Fix Scaffold-Locked Models?

• Train Planning as a Learned Skill: Instead of relying on the platform to handle planning, embed planning capability directly into the model through fine-tuning on diverse trajectories. Models trained this way retain performance across different platforms.
• Use Cross-Platform Trajectory Collection: Gather training data from multiple platforms simultaneously to expose the model to different planning conventions during the learning process, preventing platform-specific overfitting.
• Prioritize Plan Quality Over Model Size: For open-source models running on budget-constrained setups, investing in a stronger external planner can yield better results than upgrading to a larger model, potentially saving significant costs.
• Enable Self-Planning at Inference: Allow models to generate their own plans at runtime rather than relying solely on external planning, which helps them adapt to unfamiliar platform conventions.

What Do the Experiments Actually Show?

The researchers built a tool called DCAS that intercepts API traffic between any coding platform and any model, allowing controlled testing without modifying the platforms themselves. Using this setup, they ran experiments with Qwen3-Coder-30B, a 30-billion-parameter model, on the SWE-bench Verified benchmark, which measures how well models solve real software engineering tasks.

The first finding was striking: plan quality matters more than the executor model. With no planning step, the model scored 42.8% accuracy. Allowing the model to plan for itself improved this to 48.2%. When they supplied a frontier-grade planner, Claude Sonnet 4.5, the same executor model jumped to 57.8% accuracy, a 15-point improvement attributable entirely to plan quality. This swing exceeds the cross-platform performance drops the researchers measured, suggesting that for many teams, upgrading the planner is more cost-effective than upgrading the model.

The second experiment tested whether planning could be internalized. The team fine-tuned the same 30-billion-parameter model on 576 training trajectories collected from Claude Code, deliberately using a non-frontier model as the source to ensure gains came from learning platform conventions, not from distilling a more powerful model. After full-parameter fine-tuning with 65,000-token context windows, the model achieved 52.8% accuracy without external planning and 55.8% with self-planning, nearly matching the 57.8% of an external frontier planner.

The third experiment checked whether the learned capability generalized. The fine-tuned model improved to 57.2% accuracy under a newer version of the training platform and gained consistently on platforms it had never encountered during training: 3.4% improvement on OpenCode and 7.0% improvement on mini-swe-agent when using self-planning. This demonstrated the learned behavior was structural, not memorized platform-specific patterns.

Why Does This Matter for the Open-Source AI Community?

The implications are significant for anyone deploying open-source coding agents. Most practitioners choose platforms based on cost, licensing, latency, and data privacy constraints, not on which platform their preferred open model was trained under. Yet if that model was fine-tuned exclusively under OpenHands, deploying it on a different platform can result in an 80% performance loss.

The research suggests a path forward: train planning as a structural skill on trajectories collected under multiple platforms, which is exactly what DCAS enables. The researchers released both the weights and trajectory data publicly, allowing the community to build platform-portable models rather than platform-locked ones.

The broader insight challenges how the open-source model ecosystem thinks about "openness." If a model's performance depends entirely on the platform it was trained under, it isn't truly portable, even if the weights are freely available. By moving planning from a platform artifact to a learned capability, the next generation of fine-tuned models could run effectively anywhere, fundamentally changing what "open" means for coding agents.