Problem

Gimlet Labs addresses the inference bottleneck created by agentic AI. Unlike a conventional chat request, an agent may chain multiple models, call tools, retrieve data, and execute code; these stages have different compute, memory, and network requirements. Agentic systems can generate 5–15 times more tokens than traditional chat models, while homogeneous GPU infrastructure leaves latency, power efficiency, utilization, and available capacity on the table. The economic pressure is substantial: inference is already measured in quadrillions of tokens per month, the industry is preparing to spend hundreds of billions of dollars on AI data centers, and Gimlet says existing hardware may be used only 15–30% of the time for some applications.

The core use case is high-volume, latency-sensitive inference for frontier AI companies, especially coding agents and other complex multi-step agents. These workloads need materially more performance and capacity without proportionally increasing the cost of chips, power, or data-center construction.

Product / Service

Gimlet offers serverless inference for AI agents through Gimlet Cloud, alongside software that customers can deploy in their own data centers. Developers can import existing agent pipelines, chain multiple models with non-model stages such as search, connect custom data sources, and scale the workload without rewriting the application. The service is aimed primarily at large AI model labs and data centers rather than ordinary application developers.

Under the hood, Gimlet translates an agent into a compute graph, decomposes the graph into fragments, and dynamically routes each fragment to the most suitable accelerator across different generations and vendors. Its hardware-agnostic compiler optimizes those fragments, while autonomous kernel-generation tools create tuned implementations for different platforms. Gimlet operates multi-silicon data centers using GPUs, CPUs, and specialized accelerators, and says the approach can deliver roughly 3–10 times faster inference for the same cost and power envelope, with the broader goal of making agentic workloads up to 10 times more efficient.

Market

Gimlet competes in AI inference infrastructure, more specifically the emerging market for agent inference clouds and heterogeneous, multi-silicon compute orchestration. Its differentiation is that it treats the data center as a mixed fleet rather than a homogeneous GPU cluster, routing different phases of an agent or even different portions of one model to the hardware best suited to each task. The closest named alternatives in the available evidence are existing inference deployment stacks such as vLLM and llm-d, as well as conventional single-vendor GPU infrastructure; however, those systems’ support for heterogeneous hardware is described as uneven, and no directly comparable commercial rival is identified by name in the cited material.

Gimlet is not pre-revenue. It emerged from stealth in October 2025 reporting eight-figure revenue and deployments at AI-native and Fortune 500 companies, raised a $12 million seed round led by Factory, and said its customer base tripled within five months to include a top frontier lab and a hyperscaler. In March 2026 it announced an $80 million Series A led by Menlo Ventures, bringing total funding to $92 million. By June 2026 it had also joined MLCommons to help establish vendor-agnostic benchmarks for agentic inference, although the company has not publicly named those major customers in the available evidence.