Tool — Cost & PTU Sizing Calculator

A runnable back-of-envelope for "what will this cost, and how many PTUs do we need?" — the question you'll get in any banking-scale design discussion. Pairs with K00 §3 (Capacity & cost) and the system-design cross-cutting section.

Run it (offline)

pip install pytest                      # only for the tests; the tool is pure stdlib
python cost_sizing.py                                   # worked example
python cost_sizing.py --qps 200 --in 1200 --out 400 --model gpt-4o
pytest -q                               # 6 tests

What it computes

  • Cost per request = (in_tokens × input_price + out_tokens × output_price) / 1e6. Output tokens dominate.
  • Monthly pay-as-you-go cost at a given QPS (modeling non-24×7 traffic).
  • PTU sizing — PTU capacity is gated by output tokens/minute; size to a target utilization (<1) so a spike doesn't throttle.
  • Levers — the monthly cost after prompt caching (discounts the cacheable input prefix) and model tiering/routing (a fraction of traffic to gpt-4o-mini). These are the two biggest cost levers at banking scale.

The numbers to say out loud

  • $/1M tokens is the unit of cost; output costs ~4× input.
  • PAYG for dev/spiky; PTU for steady, latency-sensitive production (reserved capacity, predictable latency).
  • The cheapest levers: tier models (mini for routing/extraction), cache the system-prompt prefix, trim retrieved context, then buy PTU reservations once volume is steady.

⚠️ Prices and PTU throughput in cost_sizing.py are illustrative (~2026) and configurable — verify current figures on the Azure pricing page before quoting a customer. The method is the deliverable, not the constants.