Inside AI Tokenomics: How to Profitably Turn Tokens Into Business Value | NVIDIA AI Podcast Ep. 299

Shruti Kopakar defines tokenomics as a framework built on four pillars: token utility, token supply, token demand, and token monetization. Token value depends on two factors she calls token intelligence, which is shaped by the model used and the context length it processes, and token interactivity, measured as tokens per second per user. A domain-specific fine-tuned small language model can sometimes match or exceed a large model's accuracy on a narrow task, making the added cost of a larger model unjustifiable in that context. She also notes that while longer context generally improves accuracy, quality can degrade if context length grows too large.

Base token demand is estimated by multiplying number of users by requests per user per period by tokens per request, but several multipliers complicate that baseline. Reasoning models generate thinking tokens invisible to end users, and deployments can cap how many such tokens are allowed per interaction. Agentic workflows involve significantly more LLM calls per use case than human-alternating interactions, and can accumulate latency such that even a few milliseconds per turn produces several seconds of total delay. KV cache, which stores previously computed input requests to avoid recomputation, also affects demand estimates. Time-of-day patterns, seasonal surges such as holiday spikes in retail and e-commerce, and user growth must all be factored into forecasts.

Kopakar argues that input metrics like cost per GPU hour or flops per dollar are insufficient for evaluating AI infrastructure ROI because they do not reflect actual delivered token output. Cost per token, which incorporates both input costs and output delivery, is the metric she recommends. On this basis, NVIDIA Blackwell NVL72 delivers a 35x lower token cost compared to Hopper, even though Blackwell's hourly GPU cost is approximately 2x higher. Blackwell also delivers 50x more tokens per watt compared to Hopper, a gain she says is invisible to flops-per-dollar analysis, illustrating why cost per token is a superior metric.

She attributes these gains to what she calls extreme co-design, meaning simultaneous design of compute, memory, storage, networking, and software from CUDA kernels through serving software, extended to silicon partners, OEMs, cloud providers, and open source frameworks. Stacking all software optimizations together produces the 50x throughput improvement and the 35x lower token cost figure. VLLM and SG Lang inference runtimes have seen 8x performance improvement in approximately six months, with gains credited to the broader ecosystem including open source frameworks, partners, customers, and the developer community. Blackwell NVL72 is specifically suited for mixture of experts models due to optimized inter-GPU communication, and techniques including disaggregated serving via Dynamo and wide expert parallelism using TensorRT, VLLM, and SG Lang further reduce cost per token for those models.

Kopakar applies Jevons paradox to GPU efficiency: lower cost per token unlocks new use cases and drives demand for more GPUs rather than fewer. She cites test-time scaling and reasoning as a use case that emerged after cost per token fell, consuming the efficiency gains and requiring additional GPU capacity. She describes the current moment as an agentic revolution enabled by efficient deployment of mixture of experts and reasoning models. The Vera Rubin platform, which contains seven chips, is built for agentic AI and requires the Rubin GPU, Grok 3LPX for ultra-low latency, and the Vera CPU for tool calling and code sandboxing.

Four primary business models for turning tokens into business value are selling tokens directly, building AI-native products, enhancing existing products with AI, and deploying AI internally for employee productivity. Companies selling tokens directly include Fireworks, Base10, TogetherAI, and Deep Infra. AI-native companies building products from the ground up include Perplexity and Cursor. Companies enhancing existing products include Shopify, Airbnb, and Adobe, with Adobe having built the Firefly family of models and used them to add capabilities to existing products such as Photoshop.

Kopakar recommends that business leaders start from the desired outcome and customer need, let the use case determine model type, context length, and interactivity requirements, and then derive infrastructure decisions and monetization strategy from those requirements using cost per token as the central metric.