Researchers at MIT and the MIT-IBM Computing Research Lab have created a training dataset that solves a critical gap in enterprise AI: the inability of even the most advanced commercial models to reliably read charts and graphs. The dataset, called ChartNet, contains 1.7 million synthetic chart samples and enables small, open-source models to outperform GPT-4o across all standard chart comprehension tasks.

Why Can't Advanced AI Models Read Charts?

Charts are everywhere in business. Financial reports, scientific publications, and dashboards all rely on bar graphs, scatter plots, and line charts to communicate trends that would be buried in raw numbers. Yet reading a chart requires three simultaneous capabilities: parsing the visual geometry of axes and markers, recovering the numerical data encoded in those shapes, and interpreting the natural-language labels and legends that give the numbers meaning.

The problem is that vision-language models (AI systems trained to understand both images and text) need thousands of high-quality examples to reliably recognize something as simple as a line chart. Most existing chart datasets contain only a limited number of images scraped from the internet, often without the detailed annotations that help a model understand what the chart actually means.

"A vision-language model, unlike our brains, may need to see thousands of examples during training to reliably recognize something as a line chart," explained Jovana Kondic, an MIT electrical engineering and computer science graduate student and lead author of the ChartNet paper.

Jovana Kondic, MIT Electrical Engineering and Computer Science Graduate Student

How Does ChartNet Generate 1.7 Million Chart Samples?

Rather than manually creating or annotating millions of charts, the MIT-IBM team designed a two-stage synthetic data generation pipeline. In the first stage, a vision-language model examines a seed chart image and generates approximate executable plotting code that reconstructs it. In the second stage, a code-focused large language model (LLM) iteratively augments that code, varying the chart type, color scheme, data values, topic, and visual style to produce hundreds of distinct chart variants from a single seed.

The pipeline spans 24 chart types across six plotting libraries, including matplotlib, seaborn, and plotly. Every generated sample passes through an automated quality filter that verifies the code is executable and the rendered image is visually accurate before it enters the dataset.

"We can start from a single chart that we use as a seed and come up with hundreds of augmentations of it. This is how we were able to build a dataset with more than a million diverse images," said Kondic.

Jovana Kondic, MIT Electrical Engineering and Computer Science Graduate Student

What distinguishes ChartNet from prior synthetic chart datasets is not just its scale but its five-component cross-modal alignment per sample. Every entry contains:

• Executable Code: The plotting code used to generate the chart
• Chart Image: The rendered chart image itself
• Structured Data: A data table with the underlying numbers
• Natural Language Summary: A description of what the chart shows
• Question-Answer Pairs: Q&A pairs with step-by-step reasoning

This alignment trains a model to ground visual structure in the numerical and linguistic semantics beneath it, rather than simply learning to pattern-match chart shapes.

What Are the Real-World Performance Results?

The results are striking. On the human-verified ChartNet benchmark, IBM's Granite 4.0 3B Vision model (a compact model with just 3 billion parameters) scored 86.4% on chart-to-summary tasks and 62.1% on chart-to-table tasks, outperforming significantly larger models including commercial alternatives. The key finding: compact open-source models fine-tuned on ChartNet consistently outperformed models orders of magnitude larger, including GPT-4o, across all four standard chart comprehension tasks.

The practical implications are enormous. A 3-billion-parameter open-source model running on local hardware can now match or exceed the chart comprehension performance of a frontier model costing orders of magnitude more per inference. This advantage holds specifically for four tasks: chart reconstruction, data extraction, chart summarization, and question answering with chain-of-thought reasoning.

"Granite Vision can serve as an alternative to frontier models to perform these tasks at scale and at a fraction of the cost," said Eli Schwartz, a research manager with the IBM Research multimodal AI group.

Eli Schwartz, Research Manager, IBM Research Multimodal AI Group

How Does This Change Enterprise AI Economics?

The cost savings are straightforward. Granite 4.0 3B Vision was trained on IBM's Blue Vela supercomputing cluster using 32 NVIDIA H100 GPUs over approximately 200 hours, a training run within reach for well-resourced research teams. Once trained, the model runs on commodity hardware at near-zero inference cost.

Organizations currently using frontier commercial APIs for document understanding workflows face a significant decision. They can continue paying per-inference costs for cloud-based models, or they can deploy a smaller, faster, cheaper model on their own hardware that performs better on chart tasks. The dataset is publicly available on Hugging Face at no cost under an Apache 2.0 license, meaning any team can use it today to fine-tune their own model.

"The finance industry thrives on charts. If vision-language models can extract information out of charts, like descriptions of trends, that facilitates a lot of workflows that happen downstream," added Dhiraj Joshi, Senior Scientist at IBM Research and co-author of the paper.

Dhiraj Joshi, Senior Scientist, IBM Research

How to Deploy ChartNet-Based Models in Your Organization

• Access the Dataset: Download ChartNet from Hugging Face at no cost under an Apache 2.0 license, which allows commercial and research use
• Fine-Tune an Open-Source Model: Use the dataset to fine-tune compact models like Granite 4.0 3B Vision or other open-source vision-language models on your own infrastructure
• Deploy Locally: Run the fine-tuned model on commodity hardware or on-premises servers, eliminating per-inference API costs and reducing latency for document processing pipelines
• Benchmark Against Current Solutions: Test the fine-tuned model against your existing chart comprehension workflows to quantify cost savings and performance improvements

The ChartNet breakthrough was unveiled at CVPR 2026, the IEEE Computer Vision and Pattern Recognition Conference, currently underway in Denver, Colorado. The work represents a significant shift in how organizations can approach enterprise AI: rather than relying exclusively on frontier models accessed via expensive APIs, teams can now leverage publicly available datasets and open-source models to achieve superior performance at a fraction of the cost.