Logo
FrontierNews.ai

Grok Build Outperforms Claude Code in Cross-Project Memory Test

Grok Build passed all three memory tests while Claude Code failed on cross-project recall, and Grok cost less than half as much on every session. When xAI announced memory features in Grok Build on September 16 and Anthropic rolled out Projects beta on September 17, both companies claimed their coding agents could remember what developers told them in earlier sessions. A direct comparison tested whether that promise held up in real-world scenarios.

How Do These Memory Systems Actually Work?

Both tools store information differently. Grok Build saves notes as Markdown files in two scopes: one per project and one global scope that applies everywhere. Claude Code uses an auto-memory system with a MEMORY.md index plus one file per note per repository. The key difference emerged during testing: Grok's global scope carries rules across unrelated projects, while Claude Code's CLI memory stops at the repository boundary.

To test the core claim, the same three scenarios ran on both tools using small Node repositories on a Mac. Each test had two sessions: the first planted a fact or rule, then the tool closed. The second session gave a task where that information mattered, without mentioning it again. Here's what the tests measured:

  • Test Command Recognition: Session 1 noted that npm test fails but make test passes. Session 2 asked for a new endpoint with passing tests after the README was stripped of the Makefile reference.
  • Project Decision Retention: Session 1 stated that CSV export was dropped and money uses integer cents, never floats. Session 2 requested a refund endpoint and a way for support staff to download orders, with float helpers and half-built CSV code sitting as intentional traps.
  • Cross-Project Rule Transfer: Session 1 in one repository set two rules "for all my projects": conventional commit messages and no comments on obvious code. Session 2 ran in an unrelated repository and asked for a small feature plus a commit.

Which Tool Actually Remembered Better?

Both tools passed the first two tests on project-specific memory. In the test command scenario, Grok took 29 seconds and cost $0.11 while Claude Code took 22 seconds but cost $0.32. Both correctly used make test instead of npm test. On project decisions, Grok completed the work in 103 seconds for $0.18, while Claude Code finished in 32 seconds for $0.49. Both built the refund on integer cents and avoided the CSV trap.

The results split on the third test. Grok saved the rules to its global scope and successfully applied them in the second repository, committing with the message "feat: add --help flag with usage and supported cities" and adding no comments. Claude Code saved both rules but only in the first repository's memory folder. When the second repository was opened, Claude Code found nothing and committed with "Add --help flag," failing to carry the cross-project rule forward.

Across all three tests, Grok Build used 390,848 tokens total and cost $0.41. Claude Code used 576,863 tokens and cost $1.05. Claude Code was faster on every recall session, taking 66 seconds total against Grok's 165 seconds, but that speed advantage came with significantly higher costs and more token consumption.

What Do These Results Mean for Developers?

The core finding challenges a key assumption about Claude Code's memory system. While Claude Code's notes were better written and more detailed, its CLI memory limitation creates a practical problem for developers working across multiple projects. Any rule or convention a developer wants applied everywhere still requires manual entry into ~/.claude/CLAUDE.md. Grok's global scope handles this automatically.

The cost difference reflects the underlying models more than the memory systems themselves. Grok Build runs Grok 4.6 while Claude Code runs Opus 5, and Opus 5 carries a higher price per token. However, the token efficiency gap suggests Grok's approach to memory retrieval and reasoning uses fewer tokens to reach the same conclusions. Claude Code's faster execution times may appeal to developers optimizing for speed, but the accuracy trade-off on cross-project rules and the doubled costs create a significant practical disadvantage.

Both tools demonstrated that they write memory files immediately when a rule is stated and read them back in subsequent sessions. The difference lies in scope. For developers managing multiple repositories with shared conventions, Grok Build's architecture aligns better with real-world workflows. For single-project work where speed matters more than cost, Claude Code's faster recall times might justify the expense, though the memory limitation remains a constraint.