The speed of AI adoption in schools has far outpaced the research needed to prove these tools actually work. Nearly two-thirds of teachers now report using AI in their daily work, yet only 1 in 5 edtech products in classrooms have any evidence they can improve teaching and learning outcomes. This gap between deployment and validation is creating a critical challenge for educators and administrators trying to make informed decisions about which tools to invest in.

Why Traditional Research Methods Are Failing AI Tools?

The education field has long relied on randomized controlled trials (RCTs) to evaluate whether interventions actually work. These studies have proven valuable for assessing everything from class-size reductions to reading instruction methods. But AI tools present a fundamentally different problem: they're not stable, well-defined interventions that stay the same long enough to study.

Developers constantly update AI models and refine how they work, while teachers evolve their interactions as they learn what works best. By the time a traditional multi-year study concludes, the technology being evaluated may have changed dramatically or even become obsolete. "RCTs assume a stable intervention applied consistently across contexts. That assumption does not hold for most early-stage AI tools," according to research from the Overdeck Family Foundation.

This creates a paradox: school leaders need evidence now to make adoption decisions, but the traditional research process takes years to deliver answers. Many organizations are reaching significant scale with AI tools without even basic outcome data on how they affect teaching and learning.

How Can Schools Get Evidence Faster?

Rather than waiting for perfect evidence from large-scale trials, education researchers are proposing a different approach called implementation research and development, or implementation R&D. This method focuses on rapid cycles of testing, feedback, and refinement in real-world settings before moving to formal large-scale evaluations.

Implementation R&D is particularly well-suited to AI because these tools often capture real-time data on how educators use them. Researchers can analyze this usage data immediately, rather than waiting months for manual observations and coding. This enables faster, more responsive research that matches the pace at which AI tools are actually developed and updated.

One practical example is iterative A/B testing, which isolates and tests specific design choices rather than evaluating entire programs. An ongoing study of an AI coaching tool from Teaching Lab compares a standard version to one that emphasizes student-centered instruction, allowing researchers to see whether design differences shift coaching practices in real time.

Steps to Building Better Evidence on AI Tools

• Build Evidence in Stages: Early-stage tools should focus on feasibility and rapid-cycle testing to assess specific design choices. Larger randomized controlled trials should come later, once the intervention is stable and there is initial evidence that teacher and student outcomes are improving.
• Ask How It Works First: Before asking whether a tool improves student achievement, researchers should ask whether it changes the behaviors it is designed to change. For example, does an AI coaching tool actually shift how teachers plan, instruct, or respond to student thinking? If not, that issue should be addressed before assessing student impact.
• Let Questions Drive Methods: AI tools make it possible to run frequent, low-cost experiments and collect detailed usage data. The field should leverage these capabilities through A/B testing and continuous evaluation, rather than relying solely on methods designed for static programs.

The Research Partnership for Professional Learning's Shared Measures Toolkit demonstrates implementation R&D in practice. Rather than beginning with a fully developed measurement product, the partnership worked with professional learning organizations and researchers to identify what practitioners most needed to understand and improve. The resulting measures are now being tested, refined, and validated across multiple contexts.

What Role Will AI Play in College and Career Readiness?

Beyond classroom instruction, the Gates Foundation has made AI a central part of its updated education strategy, which includes ambitious goals to reach students before its 2045 sunset deadline. The foundation pledges to foster higher levels of math achievement, increase first-generation college-goers, improve college freshman pass rates, and achieve a 25 percent jump in college credits successfully transferred between institutions by 2030.

The foundation is particularly focused on using AI to support college advising, especially in under-resourced high schools where advisors may be responsible for 400 or more students each year. New AI agents could help track which students have completed applications or filled out FAFSA forms, allowing human advisors to focus on more personal services like encouraging high-performing students to pursue dual-enrollment courses.

"The burden of trying to make meaning out of all those inputs, which are critical to providing a quality education, rests on the teacher right now. We think it's possible to have AI do that behind the scenes and give teachers the information they need to be outstanding teachers for every single student," said Allan Golston, president of the Gates Foundation's U.S. Program.

Allan Golston, President of the Gates Foundation's U.S. Program

One example of this approach is the Atlas dashboard, developed by Kiddom and piloted among roughly 180 teachers in New York City. Using short quizzes administered at the end of the school day, Atlas provided regular status reports on each student's grasp of different math concepts. Middle schoolers using the program performed measurably better on tests over time compared with demographically similar peers learning the same curriculum.

Stanford researcher Carly Robinson noted that easing the paperwork burden might allow advisors to focus on what actually benefits from human relationships. "A huge share of their time goes to red tape rather than actually working with kids. If AI can absorb that logistical load, you're not replacing the counselor. You're directing them to be able to do more of the important work with students," Robinson explained.

Carly Robinson

The challenge ahead is ensuring that as AI tools proliferate in schools, the evidence infrastructure keeps pace. Without faster, more practical research methods, educators will continue to face a critical information gap: knowing which tools to trust and how to use them effectively. Implementation R&D offers a path forward, but it requires commitment from funders, researchers, and school leaders to prioritize evidence alongside innovation.