Logo
FrontierNews.ai

A New $750K-$3M University AI Safety Program Just Opened: Here's What Researchers Need to Know

DARPA and the National Science Foundation have jointly launched AI Forge, a new university-led research forum that will distribute $750,000 to $3 million in grants annually to study how AI systems can be made more interpretable, controllable, and robust against attacks. The initiative represents a significant shift in how federal agencies are funding AI safety research, moving away from traditional grant structures toward a collaborative forum model that involves universities, defense agencies, and intelligence partners. The Request for Information (RFI) closes June 22, 2026, at 5:00 PM Eastern, giving U.S. universities roughly one week to signal their interest in participating.

What Makes AI Forge Different From Traditional Federal Funding?

AI Forge breaks from the standard DARPA and NSF funding playbook in several important ways. Rather than issuing a traditional Broad Agency Announcement (BAA) or program solicitation, the two agencies created a jointly governed forum administered by a nonprofit organization, with involvement from the Cybersecurity, Artificial Intelligence, and Infrastructure Security (CAISI) partnership at the National Institute of Standards and Technology (NIST). This structure signals that participating universities are not simply applying for one-time grants; they are joining an ongoing research community with shared intellectual property expectations and multi-year commitments.

One of the most significant departures from standard practice is the intellectual property requirement. Project Venture intellectual property is "expected to be shared among forum participants, preferably through open-source licensing approaches," a stronger pre-commitment than typical DARPA terms. This means universities responding to the RFI should understand that any research funded through AI Forge will likely need to be released as open-source software, which may conflict with some institutions' technology transfer strategies or commercialization goals.

What Are the Three Research Areas AI Forge Will Fund?

AI Forge organizes its research agenda around three interconnected thrust areas, each addressing a core challenge in making advanced AI systems safer and more trustworthy. The Critical AI Challenges for National Security report that accompanied the launch identifies 15 specific research priorities distributed across these three areas, with plans to revisit and update priorities every six months as the field evolves.

  • AI Interpretability: The forum frames this as making "the behavior, decisions, and impacts of AI systems understandable to humans," with an emphasis on operational interpretability that holds up in adversarial and high-stakes conditions. This goes beyond post-hoc explainability research that dominated through 2024. Eligible research includes mechanistic interpretability (studying how individual neural network components work), circuit-level analysis, activation engineering, steering vectors, formal verification of neural network behavior, and chain-of-thought faithfulness studies.
  • AI Control: This thrust focuses on developing tools that "provide strong, verifiable evidence of bounded, auditable, and reliable model behavior" and establish foundations for maintaining human oversight of advanced systems. Research areas include debate protocols, recursive reward modeling, AI-assisted oversight, constitutional approaches, formal verification of bounded behavior, hardware-rooted attestation of model identity, and runtime monitoring approaches.
  • Adversarial Robustness: The most established of the three research lines, this thrust emphasizes resilience "by design so that it maintains its integrity even when under deliberate attack." Eligible research includes certified robustness, formal robustness verification, robust training approaches, security-oriented red-teaming, physical-world adversarial attacks, multi-modal attacks, and attacks on agentic systems.

How Should Universities Respond to the RFI?

The RFI itself is not a proposal submission; it is a capabilities statement that will populate a repository of U.S. universities interested in accelerating next-generation AI research. Each institution can submit only one authorized response, and the military service academies are also eligible to participate. Universities should be prepared to describe their AI research expertise, computing infrastructure, national security partnerships, and technology transition experience. The six-month revisit cadence for research priorities is unusually fast for a multi-year program and signals that the forum body will be actively re-prioritizing as the field moves.

For research administrators and AI faculty considering whether to respond, the operative question is not whether the forum will fund interesting research. The real strategic decision is whether the institutional commitments AI Forge is asking for fit the institution's broader AI strategy. The open-source intellectual property expectation and the national security positioning are not standard for academic research, and institutions should carefully evaluate whether these align with their mission and existing partnerships before committing to the forum structure.

What Are the Award Details and Timeline?

Project Venture grants are expected to range from approximately $750,000 to $3 million per award, with project durations up to one year and multiple awards expected to be distributed annually. The forum itself is targeted to launch in summer 2026, meaning the first cohort of funded projects could begin shortly after the RFI closes. The announcement was accompanied by a Critical AI Challenges for National Security report that lays out the 15 specific research challenges across the three thrust areas, providing universities with a detailed roadmap of research priorities that align with federal interests.

The joint governance structure involving DARPA, NSF, CAISI, and the Department of War and Intelligence Community engagement represents a significant coordination effort across federal agencies. This level of coordination suggests that AI interpretability, control, and adversarial robustness are now viewed as critical national security priorities, not merely academic research interests. Universities that participate in AI Forge will be directly contributing to federal efforts to ensure that advanced AI systems remain understandable, controllable, and resilient to attack.