Lab 01 — Unified Model Invocation & the Capacity Model

Phase 24 · Lab 01 · Phase README · Warmup

The problem

Bedrock's pitch is "one API over many foundation models." That is true, but it hides a real design fork you must be able to explain in an interview:

1. InvokeModel is a raw HTTP passthrough. You send provider-native JSON and get back provider-native JSON. Anthropic's Messages format, Amazon Titan's single-inputText format, Meta Llama's raw [INST]-templated prompt string, and Cohere's message + chat_history format are four genuinely different schemas — you must know each one.
2. Converse / ConverseStream is Bedrock's normalized API: one request/response shape (system prompt, multi-turn messages, tool use, streaming) that works identically no matter which model you point it at. Bedrock builds this by translating to/from each provider's native shape under the hood — Converse is not a separate model capability, it is a translation layer sitting on top of the exact same InvokeModel primitive.

That normalization has a real cost: Converse's feature set is the least common denominator across every provider it fronts. Not every model supports every Converse feature (tool use is the sharpest example) — and you build that constraint into the adapters themselves.

Then there is capacity: the three different ways Bedrock schedules and prices inference — On-Demand (token-bucket rate limiting), Provisioned Throughput (a purchased, fixed pool of Model Units that queues instead of throttling), and Batch (an async job with a discount and no real-time SLA) — plus cross-region routing and the cost math to compare them.

What you build

File map

Run it

pytest test_lab.py -v                       # your lab.py (red until you implement)
LAB_MODULE=solution pytest test_lab.py -v   # the reference (green)
python solution.py                          # worked example

Success criteria

• Four provider adapters each round-trip their own native invoke_model schema correctly.
• converse returns the SAME response shape (output/stopReason/usage) for every provider, built entirely on top of invoke_model + per-adapter translation.
• A tool_config against a provider with supports_tools = False raises UnsupportedFeatureError — the least-common-denominator tradeoff, made concrete.
• converse_stream returns a deterministic list of chunk events ending in messageStop + metadata.usage.
• TokenBucket.consume raises ThrottlingException over capacity and never goes negative; advance refills, capped at capacity.
• ProvisionedThroughputPool runs immediately within capacity, queues FIFO once saturated (not throttles), and complete drains the queue in submission order.
• BatchInferenceManager moves SUBMITTED → IN_PROGRESS → COMPLETED/FAILED, processes requests in deterministic submission order, and rejects a re-process of a finished job.
• CrossRegionRouter.route picks the lowest-queue-depth available region, deterministically.
• All 40 tests pass under both lab and solution.

How this maps to the real stack

• InvokeModel is the real low-level Bedrock Runtime API: you build the provider's native JSON body yourself (Anthropic Messages, Titan inputText, Llama's prompt template, Cohere's chat_history) and parse its native response. Every SDK example you've seen that special-cases "if model starts with anthropic.... elif amazon.titan..." is working around exactly this.
• Converse/ConverseStream are the real unified APIs AWS shipped specifically to kill that special-casing: one message/tool/streaming schema across (most) text models. Our to_native_request → invoke_model → from_native_response pipeline is the actual implementation strategy — Converse is Bedrock translating on your behalf, not a different model path. ConverseStream's real transport is Server-Sent Events with contentBlockDelta events; our list-of-events is the deterministic, testable equivalent of the same event shapes.
• Tool use support varies by model in real Bedrock — not every model Converse fronts supports the toolConfig field, and calling it against one that doesn't is a real, documented failure mode. UnsupportedFeatureError on TitanAdapter/LlamaAdapter is that constraint made visible instead of silently ignored.
• On-Demand really is a token-bucket-shaped quota per account/model/region (Service Quotas), and exceeding it really does raise ThrottlingException (HTTP 429) with an SDK-side exponential-backoff retry convention. Provisioned Throughput really is a purchased, time-committed pool of Model Units; requests within capacity never throttle, and AWS's own guidance is that sustained high-volume traffic should move off On-Demand once volume justifies the commitment — the queuing behavior here is the honest mental model of "reserved capacity," even though real Provisioned Throughput will still reject over a hard ceiling in some configurations rather than queue indefinitely. Batch inference really is submitted as an async job over an S3 input/output manifest, processed with no real-time SLA, at a meaningful discount versus On-Demand (our 50% is "on the order of" the real number, not a guaranteed rate card figure).
• Cross-region inference profiles are real: an inference-profile modelId routes a single logical request across a defined set of regions, which both increases effective throughput against per-region quotas and can reach models not yet available in your home region. Real routing is AWS-internal load-based routing; ours is the same idea (route to the least-loaded available candidate) with a simulated queue-depth signal instead of AWS's live telemetry.

Limits of the miniature. Real tokenization is BPE and provider-specific; ours is a crude len(text)//4 estimate — good enough to reason about ratios, not to bill against. Real Provisioned Throughput purchase has commitment terms (1-month/6-month) and per-model Model Unit throughput that varies by model size; ours is a flat capacity_per_unit. Real batch jobs read from and write to S3 and do not guarantee output order matches input order; ours processes in-memory in submission order specifically so the test suite can assert on it.

Extensions (your own machine)

• Add a retry-with-backoff wrapper around TokenBucket.consume that retries on ThrottlingException with a fake, injected clock (never real time.sleep).
• Add a fifth provider adapter (Mistral or AI21) to prove the adapter interface generalizes.
• Model Custom Model Import as an adapter whose native schema you define yourself, to show Converse can front a model Bedrock didn't originally know about.
• Wire estimate_on_demand_cost/estimate_batch_cost/provisioned_breakeven_requests into a small "which capacity mode should I use" calculator that takes a traffic shape and recommends On-Demand, Provisioned Throughput, or Batch.

Interview / resume signal

"Built a miniature Bedrock Runtime: four provider adapters (Anthropic, Titan, Llama, Cohere) each with a genuinely different native request/response schema, unified behind a Converse-style normalization layer that fails closed on unsupported features (tool use) rather than silently dropping them — plus the three capacity models (On-Demand token-bucket throttling, Provisioned Throughput's queue-not-throttle semantics, Batch's async job lifecycle) and the cost math that decides between them."