Loop engineering represents a fundamental shift in how developers interact with AI models like Claude. Rather than manually prompting an AI agent for each task, developers are now designing systems where the agent runs autonomously, receives feedback against explicit quality criteria, revises its work, and repeats until it meets standards,all without human intervention. Boris Cherny, the creator of Claude Code at Anthropic, described this shift bluntly: "I don't prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops." According to Cherny, he spent an entire month without opening an IDE while Claude Code wrote every line across 259 pull requests.

What Exactly Is Loop Engineering?

Loop engineering, a term coined by Addy Osmani, formalizes a three-part system that separates the roles of creator and critic. The generator is the AI agent doing the actual work. The evaluator is a separate agent or program that grades the output against a specific rubric of checkable criteria. The loop itself feeds the evaluator's report back to the generator until the rubric passes or a budget runs out. The critical rule that all practitioners agree on: the generator must never grade its own work. When a single agent evaluates its own output, it confidently praises mediocre work. Anthropic's engineering team found that "tuning a standalone evaluator to be skeptical turns out to be far more tractable than making a generator critical of its own work".

Lance Martin at Anthropic tested this principle with Claude Fable 5, Anthropic's newest model released on June 9, 2026. Verifier sub-agents running in independent context windows consistently outperformed self-critique, and a rubric-driven loop let the model improve a training pipeline roughly six times more than the previous generation managed on the same task.

Why Is Web Scraping Such a Perfect Fit for Loop Engineering?

Web scraping has spent over a decade solving the exact problem that loop engineering requires: defining what "good output" looks like in machine-checkable terms. In the Scrapy ecosystem, mature projects use Spidermon, a dedicated framework that encodes quality standards into automated monitors. These monitors check whether items validate against a schema, whether field coverage meets thresholds, whether expected item counts are reached, and whether error rates stay below ceilings. A Spidermon monitor suite is, in essence, a rubric waiting to be plugged into a loop.

The missing piece was never detection. Silent failure has always been scraping's oldest enemy: the spider that runs green for three weeks while quietly shipping garbage. What was missing was what happens after detection. Until now, that meant a human reading an alert, opening the site, sighing at the redesign, and manually rewriting selectors. Claude Fable 5, which Anthropic says can work autonomously far longer than any previous Claude model, is finally capable enough to sit inside that gap.

How to Build a Minimal Loop for Web Scraping

• Define Your Rubric: Create machine-checkable criteria that specify required fields, minimum item counts, minimum fill rates, and acceptable error thresholds. The rubric should exit with a detailed report when quality drops below standards.
• Separate the Maker from the Checker: The spider generates data; a separate evaluator program grades it against the rubric. The generator never sees its own score until the independent evaluator reports back.
• Build the Loop: Run the spider, grade it with the rubric, and on failure, pass the report to Claude Code in headless mode with permission to read the page and edit the spider. Then grade again until the rubric passes or attempts run out.

A developer tested this pattern by building a 20-line spider, a deterministic rubric, and a shell loop with no framework or orchestration platform. The rubric checked for three required fields (name, price, URL), a minimum of five items, and a 95 percent fill rate. When the developer simulated a site redesign by renaming every class and reorganizing the structure, the spider's fill rate dropped to zero across all fields. The loop kicked in, Claude diagnosed the markup change, mapped each old selector to its new equivalent, and the rubric passed on the first healing attempt.

One detail from the test delighted the developer: the healing agent tried to verify its own fix and was denied permission to execute anything. The independent rubric re-run in the outer loop was the only judge of whether the patch worked. The maker-checker separation that Anthropic recommends was not something the developer prompted for; it fell out of the loop's structure naturally.

What Does This Mean for the Broader AI Development Community?

The pace of change in agentic AI is accelerating rapidly. Claude Fable 5 shipped on June 9, 2026, and new primitives for autonomous work seem to arrive with every release. Workflows that felt cutting-edge in May, like babysitting a pull request while an agent chews through review comments, are quietly becoming things developers design once and then stop doing by hand. The ground is moving under the entire field weekly.

However, this rapid advancement has sparked controversy within the research community. Anthropic has implemented invisible safeguards in Claude Fable 5 that reduce the model's effectiveness when it detects requests related to cutting-edge large language model (LLM) development, such as building pre-training processes, distributed training infrastructure, or machine-learning accelerator design. Unlike safeguards for network security or biochemistry risks, which explicitly inform users that "this response has been processed by Claude Opus 4.8," the LLM research safeguards operate silently. The model does not switch to a weaker version or notify the user; it simply becomes less effective through methods like prompt modification or parameter-efficient fine-tuning.

This approach has angered the AI research community. SemiAnalysis, a well-known research firm, stated that the policy has affected their research and programming work. Researcher Guohao Li raised a direct question: "Are doctoral students majoring in AI and engineers contributing to open-source infrastructures such as Megatron, FSDP, and Verl using a quietly downgraded Claude in their daily work without knowing it?". Nathan Lambert, a prominent AI researcher and technology writer, argued that "an AI model that automatically becomes stupid without notifying me is essentially a misaligned AI".

and technology

Meanwhile, Claude Routines, a new feature in Claude Code released on April 15, 2026, is enabling developers to automate complex tasks that previously required human judgment. Routines can be triggered on a schedule (daily, weekly, hourly), via HTTP API calls, or in response to GitHub events like pull requests or issues. Tasks like "checking repository issues every night, prioritizing them, and sending summaries to Slack" can now be accomplished by writing a natural-language prompt, with no code required. The feature is available to Pro plan subscribers and above, with daily execution limits depending on the plan.

Loop engineering is not just a technical pattern; it represents a philosophical shift in how developers will work with AI. As the ground continues to move weekly, the developers who master loop design will be the ones who scale their impact without scaling their effort.