Why AI Models Are Splitting Over Vision: The Multimodality Divide Reshaping Chinese AI
China's leading AI developers are making fundamentally different bets about whether artificial intelligence models need to see the world the way humans do. Moonshot AI released Kimi K3 in July with native multimodality, meaning the model was trained on images and text together from the ground up. Other major players like DeepSeek are taking the opposite approach, building text-only models and adding vision as a separate tool when needed. This technical fork is exposing a deeper question: Is visual understanding essential for AI, or just a convenient add-on?
What Is Native Multimodality, and Why Does It Matter?
Native multimodality refers to training images, text, and other data together during a model's initial learning phase, so visual and linguistic understanding develop in tandem. This differs from bolting vision onto a text-only model after the fact using separate tools like optical character recognition (OCR) or a smaller vision-language model. The distinction sounds technical, but it has real consequences for how well AI agents can work independently.
Kimi K3 is a mixture-of-experts model with 2.8 trillion total parameters and a context window of one million tokens, meaning it can process roughly one million words at once. After its release, it topped Arena's Frontend leaderboard with a score of 1,679, which ranks models by how well they generate interactive websites. The leaderboard evaluates not just whether the code works, but how the finished page looks visually.
Browser development platform Puter tested Kimi K3 by deliberately introducing five visual discrepancies into a website. By comparing screenshots of the target page with the rendered version, Kimi K3 identified all five discrepancies without producing a false positive. Moonshot AI calls this iterative process "vision in the loop," where the model writes code, views the resulting page, and continues making adjustments based on what it sees.
Which AI Companies Are Betting on Native Multimodality?
The split between approaches is becoming clearer across China's AI landscape. Several major developers are pursuing native multimodality as a foundational capability:
- Moonshot AI: Released Kimi K3 with native multimodality as a defining feature, enabling the model to inspect its own work and iterate based on visual feedback.
- Alibaba Group: Treating vision as a foundational capability in Qwen3.8-Max, presenting visual understanding and multimodal input as major features.
- ByteDance: Pursuing native multimodality in Doubao-Seed-2.1, also presenting visual understanding as a core capability.
By contrast, DeepSeek, Z.ai, and Tencent's Hunyuan remain primarily text-based models. DeepSeek founder Liang Wenfeng has stated that training AI effectively does not require a world model or even multimodality. Yet he has also acknowledged that multimodality is something that ultimately must be developed. The two positions are not necessarily contradictory; model developers broadly agree on the long-term value of multimodality. Their disagreement concerns timing and cost.
How Do Natively Multimodal Models Outperform Text-Only Systems?
The practical advantage becomes apparent during long-horizon tasks that require a model to inspect a screen repeatedly. When a text-only model uses an external vision tool, the tool must first convert the screen into text and then pass it to the main model for judgment. A natively multimodal model can interpret changes on the page directly and decide what to do next without that translation step.
One researcher from a leading foundation model team explained the convenience factor: "A lot of the time, I simply prefer to upload a screenshot. Text might be able to achieve the same result, but you would have to spend a lot of context describing the visual relationships." The researcher noted that while most people may not notice the difference, multimodality is very convenient for developers. Professionals may need more capabilities and more modalities, but ordinary users will also use them in specific scenarios.
During vision-based reinforcement learning, a natively multimodal model can examine a page or image it has generated and incorporate the visual result into the same training trajectory. When an external tool is used, the perceptual result must still be passed between two models, creating a bottleneck. In a natively multimodal model, the communication channel between visual input and the language backbone has greater bandwidth. The system does not need to compress a scene into text before passing it to another model for judgment.
Why Are Developers Divided on the Timing of Multimodality?
The disagreement between companies is not about whether multimodality matters eventually, but about whether it should be a priority now. As coding capabilities improve rapidly, decisions about whether to train vision alongside language, and how much model capacity, data, and computing power to allocate to it, are shaping the technical strategies of China's leading model developers.
A text-only model cannot directly interpret visual input. In practice, a system can use OCR or a separate vision-language model to convert an image into text, labels, coordinates, or structured fields, then pass that information to a text model for reasoning. These tools can be connected through an agent framework or Model Context Protocol (MCP), allowing visual interpretation to function as an external component. From this perspective, giving an agent access to visual input is not only an R&D challenge; it is also a product design decision.
A well-tuned system can use a large language model (LLM) as its central controller and delegate visual interpretation to a smaller downstream model. This modular architecture is sufficient for many routine tasks and may be more cost-effective than training a massive natively multimodal model. However, the modular approach has limits. When you call a tool, you are still dealing with two models: one LLM that is "blind," while a smaller sidekick downstream can see clearly.
What Do Experts Say About the Future of AI Vision?
There is still no consensus on whether vision is simply a tool that helps models complete tasks or an indispensable part of understanding the world. OpenAI co-founder and former chief scientist Ilya Sutskever once said that text is "a projection of the world." Humans have compressed a vast number of real-world rules into language, suggesting that a model continuing to learn from text may still develop an understanding of the world.
"Without native multimodality, the model has no eyes, and its feedback is limited to text. But a great deal of user-facing output is visual, including websites, images, and videos. The model needs to understand those results and then use them to provide feedback to itself," explained a multimodal researcher.
Multimodal Researcher, Leading Foundation Model Team
Turing Award winner and Meta chief AI scientist Yann LeCun has reached nearly the opposite conclusion. "The vast majority of human knowledge is not expressed in text," he said. Language contains information that humans have already filtered and summarized. For models to understand objects, space, and action, they may ultimately need to learn from images, videos, and interactions with the physical world. Text is highly condensed, while images and videos are closer to raw signals. Those signals contain aspects of the world that may never have been recorded in language.
An enterprise usage report OpenAI released in January found that image uploads ranked third among the ChatGPT tools most commonly used in R&D roles, behind search and data analysis. As AI agents begin generating websites, operating software, and checking their own work, vision is taking on a broader role. Visual input is no longer limited to images users provide; it is becoming a feedback mechanism that models can use to inspect their output, identify errors, and adjust their actions.
Steps to Understanding the Multimodality Debate
- Recognize the architectural difference: Native multimodality means vision and language are trained together from the start, while modular approaches add vision as a separate tool after training the main text model.
- Understand the trade-off: Native multimodality offers better integration and faster feedback loops for agents, but requires more computing power and data; modular approaches are cheaper but introduce latency when converting visual information to text.
- Track the competitive split: Chinese AI companies like Moonshot, Alibaba, and ByteDance are betting on native multimodality now, while DeepSeek and others are waiting, betting that text-only models can achieve similar results more efficiently.
- Watch for practical outcomes: The leaderboard rankings and real-world agent performance will reveal whether native multimodality delivers enough advantage to justify the extra cost and complexity.
The multimodality divide is not just a technical disagreement; it reflects different philosophies about how AI should evolve. As agents take on increasingly long sequences of tasks, the question of whether vision is simply a tool or an indispensable part of understanding the world is becoming more concrete. The next few years will show whether Moonshot's early bet on native multimodality pays off, or whether DeepSeek's text-first approach proves sufficient for most real-world applications.