Billing for AI Services: Token, GPU, and API Accounting

Note: when an AI platform starts handling thousands of requests per second, billing becomes a bottleneck. LLM tokens are priced using an input/output model, GPU time is billed per second with a minimum charge, and API requests have rate limiting. A configuration error, and the platform could lose up to 10% of revenue. We build billing systems that synchronize these heterogeneous resources into a unified accounting model. Our engineers have 5+ years of experience developing billing for high-load AI services.

How We Ensure Accurate Accounting for Tokens and GPU Hours

The main challenge is combining multiple billing units into a single system. LLM tokens (input/output) are typically priced separately, with progressive discounts for large volumes. GPU hours are billed per second, with a minimum billing period—usually 10 seconds—to avoid microtransactions. API requests often have rate limits with burst quotas. All this must be merged into a consistent invoice that the customer understands. For each resource, we design a separate data model supporting tiered pricing, credits, and minimum charges. The final system uses PostgreSQL for OLTP operations and ClickHouse for analytical aggregations over arbitrary periods. We guarantee transparent accounting with detailed breakdowns by model, endpoint, and time slot.

from dataclasses import dataclass
from enum import Enum
from decimal import Decimal

class BillingUnit(Enum):
    TOKEN = "token"           # 1M tokens
    GPU_SECOND = "gpu_second" # GPU time
    REQUEST = "request"       # Request
    GB_MONTH = "gb_month"     # Storage

@dataclass
class PricingRule:
    resource: BillingUnit
    unit_price: Decimal
    tier_breaks: list  # [(volume_threshold, discounted_price)]
    minimum_charge: Decimal = Decimal('0')

PRICING = {
    "llm_input_token": PricingRule(
        resource=BillingUnit.TOKEN,
        unit_price=Decimal('0.000005'),  # $5 per 1M tokens
        tier_breaks=[
            (10_000_000, Decimal('0.000004')),   # >10M → $4/1M
            (100_000_000, Decimal('0.000003')),  # >100M → $3/1M
        ]
    ),
    "gpu_a100_second": PricingRule(
        resource=BillingUnit.GPU_SECOND,
        unit_price=Decimal('0.00089'),   # ~$3.20/hour per A100
        tier_breaks=[],
        minimum_charge=Decimal('0.01')   # Minimum 10 seconds billing
    ),
}

Comparison: Real-Time Metering vs. Batch Billing

Instead of relying on hourly batching, real-time metering with Redis and Kafka reduces billing latency to 10 ms. This is 50 times faster than traditional approaches, which is especially important under high load where every request costs money. Such an architecture can reduce revenue loss up to 10% and improve budgeting accuracy.

Why Real-Time Metering Is Critical for AI Billing

Without real-time metering, customers can exceed their budget, and the platform can lose money due to billing delays. We use Redis for fast counters (current balance, rate limiting) and Kafka for auditing. P99 event write latency is under 10 ms. Our certified engineers ensure system fault tolerance.

Metering implementation includes two parallel flows: operational counters in Redis for instant limit control and a Kafka event queue for further aggregation. This ensures a balance between speed and reliability.

class UsageMeter:
    def __init__(self, redis_client, kafka_producer):
        self.redis = redis_client
        self.kafka = kafka_producer

    async def record_llm_usage(self, customer_id: str, model_id: str,
                                input_tokens: int, output_tokens: int,
                                request_id: str):
        # 1. Real-time counters in Redis (for rate limiting and balance checks)
        pipe = self.redis.pipeline()
        month_key = f"usage:{customer_id}:{self.current_month()}"
        pipe.hincrby(month_key, f"{model_id}:input_tokens", input_tokens)
        pipe.hincrby(month_key, f"{model_id}:output_tokens", output_tokens)
        pipe.expire(month_key, 60 * 60 * 24 * 40)  # 40 days
        await pipe.execute()

        # 2. Detailed events to Kafka for auditing and billing
        await self.kafka.send("usage_events", {
            "event_type": "llm_inference",
            "customer_id": customer_id,
            "model_id": model_id,
            "request_id": request_id,
            "input_tokens": input_tokens,
            "output_tokens": output_tokens,
            "timestamp": datetime.utcnow().isoformat()
        })

    async def record_gpu_job(self, customer_id: str, job_id: str,
                              gpu_type: str, duration_seconds: float):
        # GPU billing with ceiling rounding to the second
        billable_seconds = max(ceil(duration_seconds), 10)  # Minimum 10 sec

        await self.kafka.send("usage_events", {
            "event_type": "gpu_training",
            "customer_id": customer_id,
            "job_id": job_id,
            "gpu_type": gpu_type,
            "duration_seconds": billable_seconds,
            "timestamp": datetime.utcnow().isoformat()
        })

Apache Kafka is used as a reliable event broker, and Redis for low-latency counters. This combination has been tested in production with loads exceeding 10 thousand events per second.

How We Generate Invoices and Control Budgets

Invoices are built from aggregated events in ClickHouse. We support tiered pricing, credits, and discounts. Budget control includes alerts at thresholds and automatic blocking.

class InvoiceGenerator:
    async def generate_monthly_invoice(self, customer_id: str,
                                        billing_period: str) -> Invoice:
        # Aggregate usage events from ClickHouse
        usage = await self.clickhouse.query("""
            SELECT
                model_id,
                sum(input_tokens) as total_input,
                sum(output_tokens) as total_output,
                sum(gpu_seconds) as total_gpu,
                count() as total_requests
            FROM usage_events
            WHERE customer_id = %(customer_id)s
              AND toYYYYMM(timestamp) = %(period)s
            GROUP BY model_id
        """, {"customer_id": customer_id, "period": billing_period})

        line_items = []
        total = Decimal('0')

        for row in usage:
            input_cost = self.compute_tiered_price(
                row['total_input'], PRICING['llm_input_token']
            )
            output_cost = self.compute_tiered_price(
                row['total_output'], PRICING['llm_output_token']
            )
            line_items.append({
                'description': f"{row['model_id']} - Input tokens",
                'quantity': row['total_input'],
                'unit': "1M tokens",
                'unit_price': float(PRICING['llm_input_token'].unit_price * 1_000_000),
                'amount': float(input_cost)
            })
            total += input_cost + output_cost

        # Apply credits and discounts
        credits = await self.get_customer_credits(customer_id)
        final_amount = max(total - credits, Decimal('0'))

        invoice = Invoice(
            customer_id=customer_id,
            period=billing_period,
            line_items=line_items,
            subtotal=float(total),
            credits_applied=float(min(credits, total)),
            total=float(final_amount)
        )

        # Send via Stripe
        if final_amount > 0:
            await self.stripe.create_invoice(customer_id, invoice)

        return invoice

async def check_budget_alerts(customer_id: str):
    customer = await db.get_customer(customer_id)
    current_spend = await usage_meter.get_current_month_spend(customer_id)

    thresholds = [0.5, 0.8, 0.9, 1.0]  # % of monthly budget
    for threshold in thresholds:
        alert_amount = customer.monthly_budget * threshold
        if current_spend >= alert_amount:
            alert_key = f"budget_alert:{customer_id}:{threshold}:{current_month()}"
            if not await redis.exists(alert_key):
                await send_budget_alert(customer, threshold, current_spend)
                await redis.setex(alert_key, 86400 * 31, "1")

Billing System Development Process

Deliverables include: full API documentation, access to source code, team training (2 hours), and one month of post-release support.

Our experience includes more than 50 completed projects in AI billing. The most common cause of revenue loss in practice is incorrect tiered pricing configuration when crossing price thresholds: without proper calculation, the platform charges less than it should. Another frequent issue is lack of edge case handling: canceled requests after inference starts, interrupted GPU jobs, and retry requests after timeout. Each of these scenarios requires explicit handling in billing logic. We verify the tariff model on synthetic data with boundary cases before launch. Contact us for a preliminary audit. We will assess the complexity of your tariffs and propose timelines.