Universities Can't Wait for Perfect AI Evidence: How Higher Ed Is Learning to Lead Through Uncertainty
Universities face an unprecedented challenge: students are already using generative AI in their coursework, but institutions haven't yet gathered enough evidence to decide how to responsibly integrate it into teaching and learning. This mismatch between rapid AI adoption and slow evidence-building has forced higher education to rethink how it approaches technological change altogether.
Researchers at the University of Colorado Boulder have published a new framework arguing that existing models for educational technology adoption no longer fit the speed and scale of generative AI. The paper, titled "A Framework for institutional change in the age of AI," challenges universities to move beyond both blanket bans and uncritical adoption, instead embracing what the authors call "humble inquiries" into how AI is actually being used on their campuses.
Why Traditional Education Technology Models Are Breaking Down?
For decades, universities have adopted new teaching tools through a predictable sequence: develop the technology, test it rigorously, gather evidence of effectiveness, then scale it across campuses. This approach worked for interactive engagement methods like Peer Instruction, Tutorials in Introductory Physics, and Learning Assistant programs. Faculty could point to peer-reviewed studies before committing to major instructional changes.
Generative AI has shattered this timeline. Unlike previous education technologies, AI tools have arrived in classrooms before institutions fully understand their impact or have sufficient evidence to guide their use. The CU Boulder researchers call this an "arrival technology," meaning it enters learning environments on its own terms, not through institutional planning.
"AI educational practices are arriving to our classes before they are proven and often without our choice," said Noah Finkelstein, Distinguished Professor at the University of Colorado Boulder.
Noah Finkelstein, Distinguished Professor at the University of Colorado Boulder
This creates a practical bind for universities. A single generative AI tool can affect homework, assessment, feedback, academic integrity, student writing, coding, research, tutoring, and career preparation simultaneously. It is not a single classroom intervention that can be neatly piloted in one module and then scaled unchanged.
What Does the New Framework Actually Propose?
The CU Boulder framework identifies six critical areas where institutional change models need rethinking. Three dimensions focus on the tools themselves, while three focus on the people involved in educational change.
- Evidence Base: AI tools are different because the evidence base is still forming, making it impossible to wait for the same level of certainty universities expect from other interventions.
- Rate of Change: Generative AI tools evolve rapidly, meaning a policy or practice that works today may become obsolete within months as the underlying technology shifts.
- Scope of Use: AI extends beyond a single course, discipline, or platform, affecting institutional systems in ways previous technologies did not.
- Faculty Agency: Many instructors did not choose AI adoption; their courses have been affected because students, vendors, and colleagues are already using it, whether faculty recognize it or not.
- Role of Change Agents: Rather than helping faculty adopt established practices, change agents should facilitate shared inquiry, helping departments compare experiments and learn from local evidence.
- Student Involvement: Students should be treated as partners in institutional change, not just recipients of reform, because they are already shaping AI practice through their own use.
The framework explicitly rejects both extremes. Universities should not pretend they can ban AI from their campuses, nor should they uncritically scale AI-based teaching practices as if the benefits and risks are already proven.
How Can Universities Implement This Framework in Practice?
The CU Boulder paper includes a case study from the University of Colorado Boulder Physics Department, which designed a six-session workshop series to help instructors respond to AI in their courses. The workshops covered AI policy development, student dialogue around AI use, and specific ways AI might appear in coursework, such as checking homework or generating physics simulations.
This practical approach grounds the framework in real institutional work rather than leaving it as purely theoretical. The design implications are deliberately cautious, calling for "humble inquiries" into local AI use rather than premature claims about best practice. Institutions should organize reform around teaching approaches and learning goals, not specific AI tools that may change or disappear.
The authors acknowledge important limits to their framework. It has not yet been empirically tested against alternative approaches, and it is drawn mainly from U.S.-based change initiatives. The AI landscape is still moving quickly, which means the most important dimensions may shift as the technology and classroom practice evolve.
What Are Universities Actually Doing Right Now?
Beyond the CU Boulder framework, universities across the country are taking concrete steps to integrate AI into teaching and learning. Penn State University launched two new grant programs this week to support faculty experimenting with generative AI in their courses.
The AI in Instruction Microgrant Program offers awards of up to $1,000 for small-scale instructional innovation projects using generative AI. The second program, the Large Course Transformation Grant Program, provides awards of up to $50,000 to teams of three or more faculty seeking to reimagine teaching and learning at the course or program level. Penn State expects to fund between six and ten of these larger transformation projects.
These grant programs reflect a broader institutional shift toward supporting faculty experimentation rather than mandating top-down AI policies. Penn State previously launched an AI literacy course for its employees last April, aiming to provide "the knowledge, skills and ethical grounding needed to engage with AI responsibly and effectively across academic and professional contexts".
How Is AI Tutoring Being Evaluated for Effectiveness?
While universities grapple with institutional change, education technology companies are pursuing rigorous evidence for AI-powered tutoring systems. Eedi Labs, a London-based education impact lab, has been selected by Accelerate, a U.S. education nonprofit, to conduct the first randomized controlled trial (RCT) of its AI Tutor in American middle schools.
The study will bring Eedi's math-focused AI Tutor to classrooms in two U.S. school districts, with a focus on addressing the "Math Gap" highlighted by the 2024 National Assessment of Educational Progress (NAEP) report, which found that nearly 40 percent of U.S. eighth graders performed below "Basic" levels in mathematics.
"When children leave school without basic math skills, their economic options narrow dramatically, exacerbating inequality. It is impossible for a teacher in a classroom of 30 to offer the precise, individualized support every student needs to stay at grade level," said Ben Caulfield, CEO of Eedi.
Ben Caulfield, CEO of Eedi
Eedi's approach differs from general-purpose chatbots. The company has designed what it calls a "constrained" AI Tutor, which activates only when a student's response to a diagnostic question reveals a specific misconception. The AI then engages the student in a brief conversation to resolve that misunderstanding before returning them to the lesson. This design deliberately avoids the pitfall of unconstrained AI, which research suggests can encourage "cognitive offloading" and reduce active learning.
The company enters this U.S. trial with a strong foundation of prior research. In trials of 1,192 students in the United Kingdom, those with the lowest baseline scores showed the largest gains. Additionally, Eedi's 2025 AI tutor trial was recognized by Stanford University's SCALE Initiative as one of only 20 high-quality causal studies out of more than 800 reviewed.
The upcoming U.S. study will involve at least 500 students and is designed to meet the rigorous standards required for ESSA Tier 1 certification, a designation that signals evidence-based effectiveness to educators and policymakers.
What Does This Mean for the Future of AI in Higher Education?
The CU Boulder framework and the broader institutional responses suggest that universities are moving toward a more pragmatic approach to AI integration. Rather than waiting for perfect evidence or banning AI outright, institutions are creating spaces for controlled experimentation, faculty development, and student involvement.
This shift reflects a deeper recognition that AI in education is not a single problem to be solved but an ongoing process of adaptation. As the technology evolves and evidence accumulates, universities will need to continuously reassess their policies and practices. The institutions that succeed will be those that treat AI adoption as a collaborative inquiry involving faculty, students, and institutional leaders, rather than a top-down mandate or a hands-off approach.