Gimlet Labs Raises $300 Million at a $3 Billion Valuation. AI Chips Just Got Interchangeable.

News Breakdown · FiscEdge Academy

Gimlet Labs raised $300 million in a Series B led by Andreessen Horowitz, valuing the AI inference startup at $3 billion, roughly triple where investors priced its $80 million Series A just six months ago. Arm Holdings and Microsoft's venture arm M12 joined as new backers, alongside returning investors Menlo Ventures, Sapphire Ventures and Factory. The round brings Gimlet's total funding to $392 million since its 2023 founding.

The number is the least interesting part. Gimlet doesn't make chips, sell cloud capacity, or train models. It sells software that decides, in real time, which chip should run which sliver of an AI request, so that the GPUs and specialized accelerators everyone is already paying for actually get used.

The signal under the headline

Every AI inference request has two very different phases: prefill, where the model chews through your prompt in parallel, and decode, where it generates the response one token at a time. Prefill wants raw parallel compute. Decode is bottlenecked by memory bandwidth. Running both phases on the same GPU, which is how most inference stacks still work, wastes one or the other almost by design.

Gimlet's software splits inference into stages and routes each one to whichever silicon actually fits: Nvidia or AMD GPUs for prefill, memory-rich accelerators from Cerebras or d-Matrix for decode, with Arm and Intel hardware also in the mix. The company says pooling chips this way delivers up to 10x gains in throughput and interactivity inside the same power envelope, and pitches itself as the industry's first multi-silicon inference cloud built for agentic AI, where a single user request can trigger dozens of model calls chained together.

That framing matters because of who is writing the checks. Arm and M12 are not financial investors making a bet on a hot sector. They are a chip designer and a hyperscaler's venture arm, both of whom benefit directly if inference stops being locked to one vendor's silicon. When your suppliers co-invest in the thing that makes their hardware fungible, that is a signal about where the market is going, not just where the money is.

Why this is the other half of the Crusoe story

If the AI infrastructure boom has a supply side (deploy more GPUs, build more data centers), Gimlet is betting on the demand side: get more useful work out of the GPUs that already exist. Those are opposite bets on the same bottleneck. Compute-heavy startups have spent the last two years assuming the fix for a ballooning inference bill is more capacity. Gimlet's pitch, and the reason a16z, Arm and Microsoft's venture arm are willing to pay a 3x markup on a six-month-old valuation, is that a meaningful chunk of that spend is just poor utilization, fixable with better scheduling software rather than more hardware.

For a founder shipping an AI feature, that is the more actionable half of the infrastructure story. You cannot influence how fast Nvidia ships next-generation chips. You can influence how efficiently your own inference stack uses the chips you're already renting.

What this changes for founders running AI products

If your product does anything agentic, meaning a single user action fans out into several chained model calls, your inference cost curve is not linear with usage. It is worse, because most of those intermediate calls run on infrastructure optimized for neither prefill nor decode specifically. That is the exact inefficiency Gimlet is charging investors a $3 billion valuation to fix, and it's the same inefficiency sitting inside your own cloud bill whether or not you ever touch Gimlet's product.

Three things worth doing this quarter if inference cost is a real line item for you:

• Separate prefill and decode in your own cost model. Most billing dashboards from OpenAI, Anthropic or your inference vendor blend the two. If you can't see the split, you can't tell whether you're paying for wasted GPU-hours or genuinely necessary compute.
• Ask your inference vendor what they're doing about disaggregation, whether that's an internal project or a partnership with a company like Gimlet, Baseten or Together AI. A vendor with no answer is a vendor who hasn't started optimizing your bill yet.
• Treat "more GPUs" and "better scheduling" as two separate levers when you're forecasting AI spend for your next round. Investors increasingly know the difference, and conflating them in a deck reads as not understanding your own unit economics.

If you remember one thing

The AI infrastructure trade just split into two distinct bets: capacity (Crusoe, CoreWeave, the neoclouds) and efficiency (Gimlet and its peers). Both are real, but only one of them is a lever you can actually pull on your own cost structure without waiting for someone else to build more data centers.

---

We break down infrastructure economics like this in FiscEdge's AI for entrepreneurs course, and teach the cost modeling behind an AI product's unit economics in financial modeling. If you're still scoping what to build, the building SaaS with AI track covers inference architecture decisions like this one. Browse the full blog for more breakdowns. Follow @fiscedge for daily Business & AI analysis.