A New York investment firm has just extended a $400 million loan backed by inference chips rather than Nvidia GPUs, marking a significant shift in how AI infrastructure gets financed and suggesting the market is moving beyond training-focused hardware toward cheaper, faster inference systems. The deal, between Upper90 and General Compute, represents the first major financing secured by inference-specific silicon, a move that echoes how the same firm pioneered GPU-backed lending five years ago.

What Are Inference Chips and Why Do They Matter?

Inference chips are specialized processors designed to run already-trained artificial intelligence models quickly and efficiently, rather than the expensive hardware used to build those models in the first place. General Compute, a startup founded by CEO Finn Puklowski and CTO Jason Goodison, is using SambaNova's SN50 processors, which are air-cooled and don't require the expensive water-cooling systems that modern GPU clusters increasingly demand. This means the chips can be deployed into existing data centers and even repurposed cryptocurrency-mining facilities without major infrastructure overhauls.

The practical advantage is significant. General Compute claims the SN50 chips will generate 600 to 700 tokens per second, compared with about 250 tokens per second for GPU-based systems, according to the company's own figures. A token is roughly a word or small piece of text that an AI model processes. While these numbers haven't been independently verified by neutral benchmarks, they illustrate why inference hardware is attracting serious capital attention.

Why Is Capital Suddenly Organizing Around Inference?

The shift reflects a broader market realization: not every company needs a supercomputer to train cutting-edge AI models, but nearly every organization needs inference to actually deploy and use those models. Billy Libby, co-founder and CEO of Upper90, explained the thesis plainly: "Not everyone needs a supercomputer, but everyone does need inference and AI". This insight has proven prescient, with companies providing access to open-source models, like OpenRouter and Fireworks, raising new funding rounds at substantial valuations.

The financing also reflects concerns about the cost of frontier AI tools and tokens. As enterprises seek cheaper alternatives to the newest large language models (LLMs) from companies like OpenAI and Anthropic, they're turning to open-source models that run efficiently on inference-optimized hardware. General Compute positions itself as an "inference neocloud," a purpose-built cloud service designed specifically for AI workloads, unlike the general-purpose infrastructure offered by traditional hyperscalers like Amazon Web Services or Microsoft Azure.

How Did Upper90 Become the First to Finance Inference Chips?

Upper90's playbook isn't new; it's proven. In 2021, Libby's firm financed GPU purchases for Crusoe Energy, an energy-focused data center startup, in what was then considered a risky bet. Traditional banks avoided chip-backed lending because nobody understood how quickly advanced processors would lose value on the resale market. But as CoreWeave, another AI infrastructure company, made chips-backed loans into a sustainable business model and then went public in March 2025 with an $8.5 billion delayed draw term loan facility, the financing structure became mainstream.

Now Upper90 is applying the same playbook to the next wave. "When we financed Nvidia GPUs as the first group to do that, the market was inefficient," Libby stated. "We could really put together something as an early participant, and kind of get compensated for the risk". With GPUs now comparatively well understood and potentially over-bought, the firm is betting that inference chips represent the next inefficiency to exploit.

What Makes This Deal Risky?

General Compute is barely a year old. It raised just $15 million in seed funding in May, led by FUSE VC with Carya Venture Partners and Village Global Ventures, at a $60 million post-money valuation. A $400 million loan against a company that small only makes sense if the lender believes the collateral, not the balance sheet, carries the risk. The challenge is that inference ASICs (application-specific integrated circuits) like the SN50 don't have the resale history that Nvidia GPUs now enjoy. If SambaNova's chips hold their performance edge and General Compute keeps landing colocation deals, Upper90's bet looks prescient. If the tokens-per-second numbers don't survive contact with real customer workloads, the firm is holding collateral nobody has ever had to resell.

How Are Other Chipmakers Competing for Inference Market Share?

The capital chasing inference silicon is real and growing. Several competitors are making aggressive moves:

• SambaNova: Completed the first close of a $1 billion Series F financing at an $11 billion post-money valuation on July 8, led by General Atlantic with Seligman Ventures, T. Rowe Price Associates, and Capital Group joining in.
• Groq: Signed a non-exclusive licensing agreement with Nvidia in December, then announced a $650 million raise in June to expand its inference cloud, with the Nvidia transaction valued at roughly $20 billion.
• Etched: Came out of stealth on June 30 saying it had raised $800 million and signed more than $1 billion in customer contracts.
• Cerebras: Signed a multi-year deal with OpenAI in January to deploy 750 megawatts of wafer-scale systems for high-speed inference.

None of these companies has yet proven it can dominate the inference market the way Nvidia has dominated training hardware. But the sheer volume of capital flowing into inference-specific silicon suggests the market believes the opportunity is real.

How to Evaluate Inference Chip Investments as a Business Decision

If you're considering whether to bet on inference chips as part of your AI infrastructure strategy, consider these factors:

• Performance Benchmarks: Demand independently verified token-per-second measurements from neutral third parties, not just vendor claims. General Compute's 600 to 700 tokens per second figure is the company's own estimate, not an audited benchmark.
• Cooling and Deployment Costs: Evaluate whether air-cooled inference chips can actually be deployed into your existing data centers without expensive retrofits, and calculate the total cost of ownership compared to GPU alternatives.
• Resale Value and Market Liquidity: Understand that inference ASICs lack the established secondary market that Nvidia GPUs now enjoy, meaning your hardware may be harder to sell or repurpose if your needs change.
• Vendor Lock-in Risk: Consider whether committing to a specific chipmaker like SambaNova creates dependencies that could limit your flexibility as the market evolves.

Finn Puklowski, General Compute's CEO, framed the Upper90 deal as more than just a funding round. "This is not just a startup that got some money for compute," he stated. "This is the first sign that capital is organizing itself and fragmenting Nvidia's monopolistic dominance". That's a bold claim, and it isn't ridiculous, but it also isn't proven yet.

What is clear is that the inference market is attracting serious institutional capital and that alternative chipmakers are gaining traction. Whether General Compute and SambaNova can execute on their promises, and whether Upper90's $400 million bet will generate returns comparable to its GPU-backed deals, remains an open question. The next 18 to 24 months will likely determine whether this financing structure becomes as standard for inference chips as it has for GPUs, or whether it remains a high-risk outlier in AI infrastructure finance.