So You Want to Sell Inference: Pricing vs Value in AI

The article exposes the uncomfortable tension at the core of most AI start-ups: selling inference at a markup is a zero-margin commodit. The fastest-growing AI companies are effectively reselling inference, but markups compress toward zero as raw API costs fall and customers compare prices directly. The real problem is that cost-plus pricing caps willingness to pay at the inference cost, turning the business into a payment rail with a dashboard instead of a defensible software company.

The concrete path to keeping 30+ points of gross margin is shifting to value-based pricing or investing in optimization. Value-based means charging for the outcome, not the token: Sierra prices per resolved ticket, Devin sells Agent Compute Units, and Databricks and Snowflake use credits to decouple pricing from raw compute. Optimization reduces inference cost through model routing, caching, and distillation to proprietary small models. Distillation is the most defensible tactic: run production traffic through frontier teacher models, distill to a sub-8b parameter student, and deploy on cheap hardware, ending with a proprietary model competitors can’t easily replicate.

The decisive insight for builders is the bring-your-own-key scenario. When a customer sees raw inference cost on their cloud bill, cost-plus becomes a visible tax they route around. Value-based pricing survives because you sell work, not tokens, and optimization survives because you sell a platform fee for making their budget go further. Every board should ask which pricing model their inference reselling business is running, because the answer determines whether they’re building a payment processor or actual software.