Why AI Token Budgets Run Out: Resource Allocation in the Credit Era¶

Decisions to make before setting personal caps

Shared-pool requirement	Control to design first
Preserve minimum access	Guaranteed allocation
Let heavy users consume idle capacity	Shared burst pool
Smooth concentrated demand	Time-decayed priority
Keep incident response running	Workload priority classes

A user can remain below a personal limit even after the company's shared AI credit pool is empty. This contradiction occurs because a personal cap defines how much someone may consume, not how much they can count on receiving. An enterprise should design AI budgets as a scheduler with guarantees, shared burst capacity, and congestion-time priorities rather than as a list of caps.

A shared AI credit pool split into guaranteed and burst capacity

AI spend has moved from seat management to shared-resource management¶

The FinOps Foundation's 2026 survey received 1,192 responses overall. For the AI-spend management question, 98% of 693 respondents selected current management, up from 31% in 2024 and 63% in 2025.¹ AI cost management also became the most desired skillset to add.

GitHub AI Credits combine pooling efficiency with shared exhaustion¶

GitHub Copilot monthly plans moved to GitHub AI Credits on June 1, 2026, and Business and Enterprise included credits are pooled across the billing entity. Existing customers received higher promotional amounts for June through August before returning to standard included amounts in September.² Once included pooled credits are exhausted, an organization must either permit paid additional usage or block usage until the next billing cycle.³ Pooling reuses idle allowances, but it does not define fairness or business priority.

Personal caps turn allocation into first come, first served¶

Consider a company with 1,000 employees sharing one million credits per month. It expects many employees to use little and gives each person a 2,000-credit cap.

If 500 employees reach that cap early in the month, 500 × 2,000 = 1,000,000, and the pool is empty. The remaining 500 employees cannot use the service even though they are below their personal caps.

The arithmetic is correct; the meaning of the cap is not. A cap is a maximum consumption level, not a minimum available amount. The distinction stays hidden while capacity is plentiful and appears as first-come allocation only when demand converges.

GitHub supports budgets at enterprise, cost-center, and user levels, but those budgets do not inherently reserve a minimum entitlement for every user.² Preserving minimum access requires an allocation policy in the common gateway that admits requests, separate from each person's spending cap.

Kubernetes separates placement decisions from runtime limits¶

Cluster schedulers have long handled the problem of placing multiple workloads on finite compute resources. In Kubernetes, a request informs placement, while a limit constrains a running container. The scheduler keeps scheduled requests within node capacity, and the kubelet applies limits at runtime.⁴

It would be incorrect to say that specifying only a Kubernetes limit leaves the request empty. If no admission-time default request applies, Kubernetes copies the limit into the request.⁴ The useful idea for AI budgeting is therefore not the surface syntax but the separation between the amount used for admission and the ceiling enforced during execution.

Translated into enterprise AI credits, the functions look like this.

Scheduler function	AI-credit role	What it protects
Request equivalent	Time-bounded guaranteed allocation	Minimum availability
Limit equivalent	User or workload maximum	Runaway-spend control
Spare capacity	Shared burst pool	Utilization of idle credits
Priority	Congestion-time ordering	Business continuity

This mapping is not a recipe for copying Kubernetes into a billing system. Unlike CPU or memory, the final cost of an AI request is often uncertain before execution, and credits from different vendors may not be fungible. Kubernetes provides vocabulary and separation principles, not a finished enterprise-credit feature.

Guaranteed and burst pools balance availability with utilization¶

In the example, the company could test a split of 300,000 guaranteed credits and 700,000 shared burst credits. Each employee would receive 300 guaranteed credits, with only consumption above that amount drawing from the shared pool.

These numbers illustrate the mechanism; they are not a recommended ratio. An actual guarantee should follow the minimum workloads that must pass, demand concurrency, overage policy, and metering delay. Guarantees can also belong to departments or workloads instead of individuals when those units produce more predictable demand.

Unused guarantees should not remain fixed forever. A weekly reservation can return unused capacity to the burst pool after a deadline, preserving both early access and later utilization. If the system still calls that capacity guaranteed after reclaiming it, it must preserve enough headroom or paid overage capacity to restore it on demand.

Fairshare changes order only when demand is congested¶

Requests above the guarantee should not fall back to simple first-come ordering. During congestion, the scheduler can temporarily lower the priority of cost centers that consumed more burst capacity recently, then decay that effect over time.

Slurm's Fair Tree uses Level FS values in the account hierarchy to calculate user fairshare rankings, and its priority calculations apply historical-usage half-life decay.⁵ The same pattern can prevent an early burst from becoming a permanent penalty for the rest of the billing month.

An initial policy can keep history by cost center and order requests by ascending decayed burst usage ÷ assigned share. Use a 24-hour half-life and FIFO for ties. The 24-hour value is a starting point for weakening daytime concentration by the next day, not a universal recommendation; tune it from observed waiting time.

Fairshare is not a total-spend reduction mechanism. When shared capacity is available, all requests can proceed; ordering changes only when requests compete. Personal caps stop runaway use within one cost center; fairshare handles burst competition between cost centers.

Priority classes connect allocation to business impact¶

The loss from delaying production-incident diagnosis differs from the loss from delaying an exploratory experiment, even when both consume the same credits. A common gateway can start with three testable classes.

Class	Admission evidence	Capacity	Aging
Continuity-critical	Incident or customer-impact ID, approval, 24-hour expiry	Separate 5% emergency reserve	No promotion into this class
Standard	Registered routine workload	Guarantees plus burst capacity outside the emergency reserve	In-class fairshare
Spare-capacity	Experiments, learning, restartable batch	Only when higher classes have no queue	One-step promotion to Standard after two hours

Five percent and two hours are starting values to tune from incident and waiting-time evidence.

Priority should follow workload impact rather than seniority. A title-based rule could let a senior employee's low-impact request displace an operator's incident response.

The queue compares priority classes first, then applies fairshare within each class. Promotion from Spare-capacity to Standard cannot draw from the emergency reserve, so aging does not erode Continuity-critical capacity.

Users should not be allowed to self-select the continuity-critical class. The workload owner records an incident or customer-impact ID and expiry, the platform team approves it, and expired classifications return to Standard. Renewal requires another approval; a misclassified request returns to Standard and its owner is notified. A monthly review checks emergency-reserve use and misclassification.

Stopping a running low-priority AI job does not refund tokens already consumed. Preemption should therefore happen at admission, before the next agent iteration, or at a restartable batch boundary whenever possible.

Atomic holds prevent guaranteed capacity from being overspent¶

A displayed guarantee can still be overspent if concurrent requests read the same remaining balance. The control point needs a gateway that handles identity, estimates, atomic holds, priority, and settlement. The following are design invariants, not a complete vendor-specific implementation or failure-handling specification.

The gateway can evaluate each request in this order.

The workload owner registers the use case, priority, expiry, and maximum output; each agent iteration becomes a separate request.
The gateway holds the maximum possible cost of that call from known input, model price, and maximum output.
The caller supplies a stable idempotency key scoped to billing entity, workload, and logical operation; a uniqueness constraint replays the same result.
Insufficient capacity routes by class order and in-class fairshare to waiting, reduction, approval, or rejection.
Settlement refunds unused capacity and charges any overrun to a dedicated safety margin.

The safety margin is not advertised as available capacity; it exists to reconcile pricing differences and provider-metering delay. The gateway passes the same maximum output to the provider, and every new agent iteration needs a new hold. Normal usage therefore stays within the hold; a price change that creates an overrun blocks new burst requests, settles from the safety margin, and updates the price table.

The gateway maps one request ID to each idempotency key and returns the existing state or result on replay. The hold ledger records amounts drawn from guarantee, burst, emergency-reserve, and safety-margin buckets separately. The normal path is HELD → DISPATCHED → SETTLED. Before the external call, compare-and-swap claims HELD → DISPATCHED and the same transaction writes an outbox record. The dispatcher retries an outbox record with the same request ID only when the provider supports an idempotency token. Without that support, an unknown response is not retried automatically and remains isolated in DISPATCHED; in an expiry race, only the winner of the state transition acts. Only pre-dispatch expiry or rejection can take HELD → CANCELED, atomically releasing capacity with that transition. DISPATCHED → CANCELED is allowed only after the provider's maximum metering delay and confirmed non-use, and capacity is released with that transition. If usage is found, the record takes DISPATCHED → SETTLED; SETTLED and CANCELED are terminal. No transition returns DISPATCHED to HELD, and no capacity is released before reconciliation. The DISPATCHED → SETTLED mutation, settlement, refunds, and per-bucket balance updates share one transaction and are idempotent on request ID and provider usage ID. An unknown dispatch result remains isolated in DISPATCHED. Late usage after CANCELED charges the safety margin, freezes new burst work for every funding bucket recorded on the hold, and triggers human reconciliation.

Fairness requires outcome and enablement measures, not equal shares alone¶

Equal allocation is easy to explain, but concentrating credits only among current heavy users withholds learning opportunities from functions that have not adopted AI effectively. A normal-work guarantee preserves access, while learning can wait in the Spare-capacity class. Business-value measurement and model routing are covered in the existing AI FinOps design.

The design still has failure conditions¶

Guarantees and fairshare work when a company can observe the shared pool quickly and admit requests through one control point. If departments own separate SaaS contracts, usage data arrives days late, and credits cannot be reassigned, a common scheduler becomes an accounting overlay rather than an enforcement layer.

Flat-rate pricing could also return for lightweight workloads. A two-tier market that includes completion and short chat while metering frontier models and long-running agents would reduce the value of scheduling every small request.

Some workflows exhaust human review capacity before they exhaust tokens. For those workloads, improving verification queues, approvers, and automated tests will create more value than increasing the credit budget.

Four acceptance examples should be fixed in the implementation spec¶

Example	Initial state and event order	Pass condition
Concurrent replay	Guarantee 300; same key requests max 100 twice; actual-80 callback twice	One request ID and, with provider idempotency, one billable operation; final guarantee 220; replay changes no balance
Dispatch-expiry race	Guarantee 300; hold 100; CAS operations race	Dispatch win: balance 200 and one call pending reconciliation. Cancel win: balance 300 and zero calls
Exhaustion and priority	Burst 0; another cost center has guarantee 50; A/B shares 1; decayed use A=200, B=100	Guarantee 50 passes; B then A; FIFO on ties; critical work uses only the 5% reserve
Aging and delayed metering	Evaluate Spare-capacity at 1h59m and 2h; return use/no-use after dispatch	Spare before 2h, Standard at 2h, never emergency reserve; use settles, only confirmed non-use after the maximum metering delay cancels

These are acceptance examples, not a complete test plan for fault injection, persistence, or provider-specific metering.

Personal caps belong after these tests as runaway-spend controls. Caps alone hide first-come allocation; tested holds, guarantees, sharing, and priorities make protected workloads explicit under exhaustion.

Sources¶

FinOps Foundation, State of FinOps 2026, February 19, 2026. Used for 1,192 total respondents, N=693 for the AI-management question, the trend, and the skills result. ↩
GitHub, GitHub Copilot is moving to usage-based billing, April 27, 2026. Used for the monthly-plan transition, Business and Enterprise included amounts, June-August promotion, pooling, and budget hierarchy. ↩↩
GitHub Docs, Usage-based billing for organizations and enterprises, accessed August 9, 2026. Used for exhaustion behavior and optional additional usage. ↩
Kubernetes Documentation, Resource Management for Pods and Containers, accessed August 9, 2026. Used for request, limit, scheduling, and limit-to-request default semantics. ↩↩
SchedMD, Slurm Workload Manager: Fair Tree Fairshare Algorithm, accessed August 9, 2026. Used for fairshare ranking and half-life decay. ↩