Problem

Andon Labs addresses the control and evaluation problem that emerges when frontier AI agents move beyond short, supervised tasks and begin operating with tools, money, accounts, and physical-world consequences. Its premise is that human-in-the-loop safety is insufficient: developers need to know whether an agent remains coherent, honest, profitable, and controllable over long time horizons, and what happens when it encounters adversarial suppliers, customers, or unexpected incentives. The pain is therefore both a safety risk and an economic one, because autonomous systems could eventually operate businesses at lower labor cost while failures create financial losses or larger real-world harms.

The killer use case is pre-deployment testing of an agent running a business. Vending machines are a deliberately simple but revealing environment: the agent must manage inventory, suppliers, negotiations, customer complaints, and profit over months rather than merely answer benchmark questions. Andon extends that idea into real deployments such as a store, cafe, vending machines, and radio stations, using actual tools and money to expose behaviors that laboratory evaluations may miss.

Product / Service

Andon Labs combines custom AI-model evaluations, long-horizon benchmarks, and real-world deployments of what it calls Safe Autonomous Organizations. It builds and operates organizations without humans in the loop, then brings AI-control research into those deployments. Its Vending-Bench simulates an agent managing a vending-machine business, while its broader experiments give agents bank accounts, email, business objectives, and physical hardware; the same agent harness can run back-office tasks, send emails, and manage longer-running operations.

The delivery model appears closer to bespoke evaluation and research services for frontier AI labs than to a self-serve software product. Andon develops custom evaluations and uses its own stores, cafes, vending machines, radio stations, and hardware experiments as test environments. The benefit is evidence about long-horizon capability and failure modes under realistic conditions, plus practical control protocols for keeping autonomous organizations safe. Its company materials say Vending-Bench is used at major model releases, giving the work relevance beyond Andon’s own experiments.

Market

Andon Labs competes in the emerging market for AI safety, frontier-model evaluations, agent evaluation infrastructure, and AI-control research. It is differentiated from ordinary observability or prompt-testing tools by evaluating agents as operators of organizations and businesses in simulated and physical environments. Adjacent competitors and alternatives include Scale AI, Weights & Biases, Arize AI, Patronus AI, Kolena, Giskard, HumanLoop, and Lattice, as well as agent-evaluation frameworks such as MLflow, DeepEval, Ragas, Arize Phoenix, and LangSmith.

The company has meaningful research traction but remains early commercially. Y Combinator lists it as an active San Francisco company founded in 2023 with 11 employees, and PitchBook reports $500,000 raised. Its first Safe Autonomous Organization, a vending machine at Anthropic’s office, received substantial media coverage; Vending-Bench has been cited by Andon as used in major model releases, and the company has continued launching public experiments including Andon FM and Drone-Bench in 2026. Public evidence does not establish revenue or a named paying-customer base, so it is best characterized as an early-stage evaluation and research company with commercial potential rather than definitively labeled pre-revenue.