For the last three years, flat-rate pricing made AI experimentation cheap. Enterprises spun up models, agents, and pipelines with minimal financial scrutiny. Token costs were largely absorbed by vendors chasing market share, and the result was an era of data abundance: more context, more documents, more inputs fed into models with little consideration for efficiency. That era is coming to an end.
In April 2026, Anthropic began charging Claude Enterprise subscribers for the full cost of compute resources, ending the subsidized pricing that had previously cushioned heavy usage. GitHub followed weeks later, announcing a shift to usage-based AI Credits billing for Copilot starting June 1, citing what it called “escalating inference cost” it could no longer absorb. GitHub’s internal costs reportedly nearly doubled week-over-week since January 2026. Across the industry, infrastructure spending is projected to exceed $5 trillion, vastly outpacing current revenues.
Google now processes 3.2 quadrillion tokens per month. Demand for GPU capacity continues to exceed supply. Major AI vendors are recalibrating their models accordingly, and pay-as-you-go billing is becoming the industry standard.
What Tokens Cost and Why It Compounds
Tokens are the fundamental unit of AI processing. On average, 100 words generate roughly 135 tokens. Every prompt, every document fed into a model, every generated response carries a token cost, and those costs vary significantly. Output tokens for high-end GPT models currently run as high as $30 per million. Input tokens cost less, but the volume at the enterprise scale makes even input costs material.
Agentic AI workflows amplify the problem. Token consumption in agent loops grows quadratically, not linearly. One wrong decision early in a pipeline wastes every token downstream. A single misconfigured agentic session can produce a bill of hundreds of dollars in a single weekend. Rework compounds the exposure: approximately 37% of AI time savings gets consumed correcting outputs that require multiple rounds of refinement before they are usable.
Behavioral incentives are pushing usage higher still. Some hyperscalers now require more than 80% of developers to use AI weekly and track consumption on internal leaderboards, fueling a “tokenmaxxing” trend where employees run tools on trivial tasks to inflate their numbers.
The previous subsidized pricing model covered a lot of this inefficiency, but metered billing will not.
The Enterprise Data Volume Problem
Enterprises have spent years accumulating unstructured data: email threads, documents, reports, case files, transcripts. The volume is enormous and the governance is often thin. When AI pipelines arrived, the default approach was to feed the model as much context as possible and let it sort out relevance.
That approach made sense when tokens were cheap. At pay-per-token rates, it becomes a liability. Consider what happens when full document files enter a pipeline:
- Formatting metadata, embedded objects, and structural noise inflate token counts well beyond the actual content
- ROT data (redundant, obsolete, and trivial content) generates inaccurate model outputs and forces costly rework cycles
- Duplicate records across repositories multiply token consumption for every query that touches them
- Unclassified sensitive content creates compliance exposure alongside rising costs
Enterprises that built AI pipelines during the subsidy era, and have not since revisited what they feed their models, are carrying real financial risk. Budget forecasts built on flat-rate assumptions will not hold under consumption-based billing.
Context Curation as a Competitive Discipline
Token efficiency is emerging as a genuine differentiator. Gartner analysts have noted that one developer might consume 10,000 tokens completing a task that another finishes in 1,000. The difference is rarely model quality, it is prompt quality and input quality. Some organizations are already building internal tools to reduce the steps required to answer a query, cutting follow-up questions from ten to four and measurably reducing token consumption in the process.
The strategic imperative is curation over volume. Clean, classified, deduplicated content delivered as extracted text and metadata — rather than full files — produces better model outputs at a fraction of the token cost. Enterprises that build this capability into their data architecture before the billing environment fully shifts will carry a cost advantage.
The Architecture That Supports It
Addressing token efficiency at the data architecture level requires a governed layer between enterprise data and the AI pipeline. This governance layer extracts what matters, removes what does not, and delivers clean context rather than raw files.
In-place data management provides this architecture. Rather than moving or copying files to a central location, it extracts and indexes the essence of every document — metadata and full content — while original files remain in their source systems. AI pipelines query the index rather than requesting bulk exports, which eliminates API throttling and transfer overhead. The token cost of delivering a document’s extracted essence is a fraction of the cost of passing the full file.
The governance layer arrives before content reaches the model:
- Content-based classification identifies what is relevant, sensitive, or subject to retention policy
- ROT remediation removes redundant and obsolete records before they enter any pipeline
- Sensitivity identification keeps regulated data from reaching models without proper authorization
- Continuous indexing ensures the pipeline receives current, low-latency data without waiting for export queues
The result is AI systems that receive curated, governed context rather than unfiltered data sprawl.
The New Margin
The compute crunch has reframed the AI ROI conversation. As token pricing reflects real infrastructure costs, the efficiency of what goes into a model matters just as much as the model itself.
Organizations that governed their data before the billing environment shifted are positioned to run leaner, more accurate AI pipelines. The data architecture underneath the AI pipeline and the governance decisions made now will determine cost structures for years to come.
See how in-place data management positions your enterprise for the token economy.
