Automated AI research and development (RSI) is moving from theoretical concern to practical reality, forcing policymakers and companies to grapple with how to manage a technology that could accelerate AI progress in unpredictable ways. A new policy framework from think tank IFP outlines 23 specific recommendations for managing the risks, while a startup called Intology is already demonstrating that AI systems can improve other AI models at scale, and researchers at MIT and Columbia are studying whether competing firms can actually coordinate to slow down their development race.

What Exactly Is Automated AI Research, and Why Should You Care?

Automated AI research refers to using AI systems themselves to help design, train, and improve other AI models, rather than relying solely on human researchers. This creates a feedback loop: as AI systems become more capable, they can theoretically accelerate the pace at which new AI systems are developed. The concern isn't that AI will suddenly become conscious and rebel; it's that the speed of development could outpace our ability to understand and manage the risks.

Think of it like a car that only has an accelerator pedal and no brakes. Right now, the AI industry is moving fast, but we lack the tools and transparency to slow down if we need to. That's the core problem policymakers are trying to solve.

What Are Policymakers Actually Proposing?

The Institute for the Future (IFP) has released a set of policy recommendations designed to give governments more control over how automated AI research develops. Rather than trying to stop progress entirely, the recommendations focus on building what the researchers call "low-regret" options, meaning moves that make sense whether or not automated AI research becomes a major concern.

The recommendations span seven key areas:

• Accelerating Beneficial Uses: Directing compute resources and talent toward inference (running AI models) and developing new applications rather than just building bigger models.
• Improving Safety Research: Investing in R&D to make automated AI research safer, either by improving model safety directly or by building societal resilience to potential risks.
• Transparency Requirements: Creating mechanisms to provide visibility into how companies are automating their AI research processes.
• Building State Capacity: Helping governments develop the expertise and tools to understand and respond to automated AI R&D developments.
• Risk Management Strategy: Developing frameworks that accelerate defensive and commercial AI uses while managing risks.
• Verification Technology: Investing in tools to verify that companies are complying with any agreed-upon safeguards.
• International Cooperation: Creating options for countries to coordinate on managing automated AI R&D risks, similar to how nuclear arms control has historically worked.

Can Companies Actually Agree to Slow Down?

A new research paper from MIT and Columbia University titled "Racing to Ruin" tackles a fundamental question: if AI companies are competing to build the most powerful systems, can they actually coordinate to slow down development without one firm gaining an unfair advantage? The answer is nuanced and depends heavily on two factors: trust and transparency.

The researchers modeled competition between two AI firms racing to develop powerful systems while facing a shared risk, like an accident or misuse scenario that could harm both companies equally. Their analysis reveals that achieving a stable slowdown requires firms to trust each other and to have reliable ways to verify that the other company is actually slowing down, not just claiming to while secretly continuing development.

"With low trust, every equilibrium races to ruin: the disaster arrives with probability one. With intermediate trust, immediate stopping and racing to ruin are both equilibria. With high trust, in every equilibrium, the probability that two rational firms race forever vanishes quadratically in the prior odds ratio of rationality," the researchers concluded.

MIT and Columbia University researchers, Racing to Ruin

The paper also reveals something counterintuitive: transparency can actually make coordination harder in some situations. If one company can quickly detect that a rival has slowed down, it becomes more tempting to keep racing and only stop after confirming the rival has truly stopped. This is the classic "free-rider" problem in game theory.

How Close Are We to Fully Automated AI Research?

While we're not there yet, the progress is real. Intology, an AI startup focused on automating R&D, released a new version of its Locus software that achieved a score of 44.7% on PostTrainBench, a benchmark that measures how well AI systems can take an existing model and improve its performance. This outperformed every other frontier AI agent baseline tested, and when given additional computing resources, the system even surpassed the official human-tuned baseline.

For context, when PostTrainBench was first introduced in March 2026, the best-performing system scored only 23.2%. The rapid improvement in just a few months suggests that automating the process of improving AI models is becoming increasingly feasible.

What About Open Source and Local AI Models?

Meanwhile, Meta released Muse Glimmer, a 30-billion-parameter multimodal AI model designed for local, agentic use cases and released under the Apache 2.0 open source license. The model can process both text and images, and it's optimized to run on local hardware rather than requiring cloud computing resources. This represents a shift toward making capable AI systems more accessible and privacy-preserving, since data doesn't need to be sent to remote servers.

Muse Glimmer includes a speculative decoding feature that can speed up text generation, making it practical for real-world applications like coding assistance, document analysis, and personal assistants. The model is available immediately with support in major AI frameworks like transformers, vLLM, and llama.cpp.

How to Stay Informed About AI Policy Developments

• Follow Policy Releases: Monitor think tanks like the Institute for the Future and academic institutions for policy recommendations and research on AI governance and automated research systems.
• Track Benchmark Progress: Keep an eye on AI benchmarks like PostTrainBench that measure progress in automating AI research, as these indicate how close we are to fully autonomous AI development.
• Understand Transparency Initiatives: Learn about efforts to create transparency mechanisms between AI companies, similar to nuclear arms control verification, which may become increasingly important as competition intensifies.

The convergence of these developments, from policy recommendations to working AI research automation systems to open source models, suggests that the AI industry is entering a new phase. The question is no longer whether AI systems can help automate research, but how quickly this will happen and whether we'll have the governance frameworks in place to manage it responsibly.