« Phase 24 README · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 24 Warmup — Amazon Bedrock: The Managed Foundation-Model Platform

Who this is for: someone who has already built agent infrastructure from scratch (Phases 00-17) and framework internals (Phases 18-23, including AgentCore in Phase 20) and now needs to speak fluently about the layer AgentCore and every other framework ultimately calls: the managed model-serving platform. By the end you will be able to explain, from first principles, how Bedrock federates model vendors behind one API, how it prices and schedules inference, how it enforces content safety and access control as separate concerns, how managed RAG actually works under the hood, and how Bedrock stacks up against Azure AI Foundry, Vertex AI, Databricks Mosaic AI, OpenAI directly, and the specialized inference clouds — with a real decision framework, not a feature checklist. No AWS account, no boto3, no network — everything in the labs is mechanism.

What Bedrock actually is: control plane vs data plane
The model catalog: federation and its tradeoff
InvokeModel vs the unified Converse/ConverseStream API
Capacity models in depth: On-Demand, Provisioned Throughput, Batch
Cross-region inference profiles and latency-optimized inference
Guardrails architecture: content safety, not access control
Knowledge Bases: managed RAG-as-a-service
Model customization: fine-tuning, continued pre-training, import, distillation
Model evaluation jobs
The security model
Pricing model deep-dive with worked cost math
Observability: metrics, logs, invocation logging
Competitors: a principal-level comparison
Bedrock vs AgentCore vs Agents for Bedrock
Common misconceptions
Lab walkthrough
Success criteria
Interview Q&A
References

1. What Bedrock actually is: control plane vs data plane

Amazon Bedrock is a fully managed service that exposes foundation models from multiple providers behind one API, plus the platform capabilities every production LLM application needs around those models: capacity management, content-safety guardrails, managed retrieval, model customization, and evaluation. Read that as two separable halves, because the AWS SDK itself enforces the split: boto3.client("bedrock") is a different client from boto3.client("bedrock-runtime"), and that is not an accident of API design — it is the control-plane / data-plane split made literal.

Control plane (bedrock, bedrock-agent) — the slow-moving, administrative surface: ListFoundationModels, GetFoundationModel, creating and versioning a Guardrail, creating a Knowledge Base, purchasing Provisioned Throughput, kicking off a customization or evaluation job. These calls are infrequent, IAM-gated, and typically made by a human or a CI/CD pipeline, not by your application's hot path.
Data plane (bedrock-runtime, bedrock-agent-runtime) — the high-frequency, latency- and cost-sensitive surface your application actually calls per request: InvokeModel, Converse, ConverseStream, ApplyGuardrail, Retrieve, RetrieveAndGenerate. This is the path that gets rate-limited, the path CloudTrail data events log, and the path every dollar of your inference bill flows through.

The mental model worth internalizing: Bedrock is not a model. It is a catalog-as-a-service. The control plane's core object is the model catalog itself — every model has a modelId, provider metadata (input/output modalities, streaming support, whether it supports Converse tool use, whether it can be customized), and a per-account, per-region model access grant you must explicitly enable before the data plane will let you invoke it. That access-grant step exists precisely because Bedrock is reselling third-party providers' models under AWS's own billing and compliance umbrella — which sets up the whole federation story in §2.

2. The model catalog: federation and its tradeoff

Bedrock's catalog spans four categories of model, and knowing the shape of each is a fast seniority signal:

First-party (Amazon) — Titan (Titan Text Express/Lite/Premier, Titan Embeddings G1/V2, Titan Image Generator, Titan Multimodal Embeddings) and the newer Nova family (Nova Micro, Lite, Pro, and multimodal generation models Nova Canvas and Nova Reel), Amazon's own foundation models, priced and supported directly by AWS with the deepest Bedrock-native feature support (Provisioned Throughput availability, fine-tuning, and — for Nova — distillation, §8).
Third-party — Anthropic (Claude), Meta (Llama), Mistral AI, Cohere (Command, Embed, Rerank), AI21 Labs (Jamba), Stability AI (Stable Diffusion / Stable Image). AWS hosts these models' weights (or brokers the inference) but does not own the model itself — feature parity with each vendor's own native API is not guaranteed, and lags the vendor's own platform for brand-new capabilities.
Custom Model Import — you bring your own model weights (for supported open architectures — notably Llama- and Mistral-family checkpoints in Hugging Face safetensors format) and Bedrock hosts them behind the same invocation API, billed as on-demand (serverless) with no Provisioned Throughput purchase required. This is the escape hatch for a fine-tuned open-weight model you trained outside Bedrock.
Fine-tuned / customized models — the output of a customization job (§8) becomes its own invokable modelId, distinct from its base model.

The federation mechanism: every provider's model is wrapped so it is addressable by one modelId string and reachable through the same IAM/VPC/Guardrails/CloudWatch substrate — you get uniform platform behavior (auth, network isolation, logging, safety policy) across every vendor, even though the models underneath remain genuinely different.

The real tradeoff, stated plainly: federation buys you vendor flexibility and a uniform operational envelope — swap Claude for Nova without re-plumbing IAM, VPC, logging, or Guardrails — at the cost of a least-common-denominator feature surface. A capability that only one provider supports (a bleeding-edge context window, a provider-specific tool-calling nuance, a just-released model variant) reaches Bedrock later than it reaches that provider's own first-party API, and Bedrock's own unifying layer (Converse, §3) will not expose it until enough providers support something like it. If your product's edge is a specific provider's frontier capability, calling that provider directly is sometimes the right call — Bedrock is optimized for operational uniformity and multi-vendor optionality, not for being first to any one vendor's newest feature.

3. InvokeModel vs the unified Converse/ConverseStream API

Bedrock actually ships two invocation APIs, and the difference is a favorite interview probe because it reveals whether you've called Bedrock in anger or just read the marketing page.

InvokeModel / InvokeModelWithResponseStream is a thin, provider-native passthrough: you build the JSON body in that specific provider's own schema and get back that provider's own response schema. Anthropic's Messages format, Titan's single inputText completion format, Llama's raw [INST]-templated prompt string, and Cohere's message/chat_history chat format are four genuinely different shapes. Nothing is normalized; you are calling the provider through Bedrock's plumbing, not through a Bedrock abstraction.

Converse / ConverseStream is Bedrock's answer to the resulting special-casing: one request/response shape — system prompt, multi-turn message history, tool use / function calling, and streaming — that works identically regardless of which model you point it at. Under the hood, Bedrock translates your unified request into each provider's native shape, calls that provider, and translates the native response back. Converse is not a different model capability; it is a translation layer built on top of the exact same InvokeModel primitive, and Lab 01 makes you implement precisely that translation for four different provider shapes so the mechanism stops being a black box.

Why this matters in production, concretely:

Portability. Code written against Converse can switch modelId from Claude to Nova to Llama with no schema changes — the entire reason product teams standardize on it.
The least-common-denominator cost is real and specific. Not every model Converse fronts supports every Converse feature. Tool use (toolConfig) is the sharpest example: call it against a model whose native API has no structured tool-calling concept, and Bedrock returns a validation error rather than silently degrading — Lab 01's UnsupportedFeatureError on the Titan and Llama adapters is exactly this failure mode, deliberately surfaced instead of hidden.
InvokeModel still matters when you need a provider-specific feature Converse hasn't normalized yet, or when you're migrating existing provider-native code onto Bedrock without a rewrite.

4. Capacity models in depth: On-Demand, Provisioned Throughput, Batch

This is the section a principal engineer should be able to work through on a whiteboard, with real formulas, because it is a genuine cost/latency/reliability design decision, not a checkbox.

On-Demand

Pay per token, no commitment, subject to a per-account, per-model, per-region quota — a request-rate and token-rate ceiling AWS calls a Service Quota. Mechanically this behaves like a token bucket: you have a maximum burst capacity and a steady refill rate; exceed it and the API returns a ThrottlingException (HTTP 429). The SDK convention is exponential backoff with jitter and retry — you are expected to handle this, not treat it as exceptional. Lab 01's TokenBucket (capacity, refill_per_tick, consume) is this mechanism exactly, minus the wall clock (an injected tick, per this track's determinism rule).

On-Demand is the right default for spiky, unpredictable, or low-to-moderate traffic, because you pay only for what you use and carry zero fixed cost. Its failure mode under sustained high volume is throttling — a real latency and reliability risk if you don't monitor and back off correctly.

Provisioned Throughput

You purchase a fixed number of Model Units — each Model Unit guarantees a certain reserved throughput (a model-specific tokens-per-minute figure that varies by model size and input/output token mix; treat any specific number as "on the order of," AWS publishes current figures per model). Two commitment shapes exist: no commitment (hourly, priced higher, cancel anytime) and term commitments (1-month or 6-month, priced lower per hour in exchange for the commitment). The behavioral difference from On-Demand is the whole lesson: within your reserved capacity, requests never throttle. Once every purchased Model Unit is saturated, additional requests queue rather than reject outright — a fundamentally different failure mode (added latency) than On-Demand's (an error you must retry). Lab 01's ProvisionedThroughputPool (capacity, in_flight, a FIFO _queue, invoke/complete) is this exact queue-not-throttle semantic, proven by a test that shows a saturated pool queues a request instead of raising.

The whiteboard question: at what request volume does Provisioned Throughput start being cheaper than On-Demand? Set the fixed provisioned cost equal to the variable on-demand cost and solve for request count:

\[ N^* = \frac{C_{PT} \times U \times H}{C_{OD}} \]

where $C_{PT}$ is the hourly cost per Model Unit, $U$ is the number of Model Units purchased, $H$ is the number of hours you're comparing over, and $C_{OD}$ is your average On-Demand cost per request. Below $N^*$ requests in that window, On-Demand is cheaper; above it, Provisioned Throughput is. Lab 01's provisioned_breakeven_requests computes exactly this — plug in illustrative numbers ($0.02/request on-demand, $21/hour per Model Unit, 1 unit, 24 hours) and you get roughly 25,200 requests/day as the crossover; below that, stay On-Demand.

Batch

An asynchronous job, not a live request: you submit a batch of prompts, Bedrock processes them on its own schedule with no real-time SLA, and — per AWS's own batch inference announcement — at on the order of a 50% discount versus On-Demand pricing for the same tokens. The status lifecycle is SUBMITTED → IN_PROGRESS → COMPLETED/FAILED (Lab 01's BatchInferenceManager mirrors these exact states). Batch is the right tool for large, non-interactive workloads — nightly re-summarization, bulk classification, offline evaluation — where latency doesn't matter but unit cost does. Real Bedrock batch jobs read input from and write output to S3 and do not guarantee output order matches input order; the lab processes in-memory in submission order specifically so the mechanism is testable, and calls this difference out explicitly.

The decision, in one sentence per mode

On-Demand for unpredictable or low-volume traffic where you'd rather pay per token than commit; Provisioned Throughput once sustained volume crosses the breakeven and you need throttle-free latency guarantees; Batch for large, delay-tolerant workloads where discount beats latency.

5. Cross-region inference profiles and latency-optimized inference

Two more capacity levers sit on top of the three modes above.

Cross-region inference profiles let you invoke a single logical modelId (an "inference profile" ARN, conventionally prefixed like us.anthropic.claude-...) that Bedrock routes across a defined set of regions on your behalf. Two concrete benefits: it raises your effective throughput against per-region On-Demand quotas (your account's us-east-1 quota and us-west-2 quota are now both available to the same logical request stream), and it can let you reach a model not yet available in your home region by routing to one where it is. Lab 01's CrossRegionRouter models the mechanism — pick the available region with the lowest simulated queue depth among the profile's member regions — as the same idea real cross-region routing implements (AWS's own load-based routing is internal and proprietary; the lab's point is the decision shape, not the telemetry).

Latency-optimized inference is a newer Bedrock capability (available for a subset of models) that trades a modest cost premium for materially lower time-to-first-token and higher tokens-per-second, using optimized serving infrastructure for that specific model. It is a per-request opt-in, not a separate capacity mode — you request it alongside a normal Converse call for models that support it, when your product is latency-sensitive (a voice agent, an interactive coding assistant) rather than throughput-sensitive.

6. Guardrails architecture: content safety, not access control

Guardrails is a content-safety policy layer: it decides whether a piece of TEXT is safe to pass through. This is a fundamentally different job from Cedar/IAM access control, which decides whether a principal may take an action on a resource — the job Phase 20's AgentCore Policy lab builds with Cedar's permit/forbid/default-deny/forbid-overrides semantics. Draw this distinction explicitly and you will out-perform most candidates on this exact question: Guardrails gates what an agent is allowed to say; Cedar/IAM gates what an agent is allowed to do. A production agent needs both, they are configured separately, and they fail independently — a Guardrails intervention never authorizes a tool call, and a Cedar permit never excuses toxic or hallucinated text.

Guardrails is a composite of five independent policies, each evaluated at up to two points in the request lifecycle:

Denied topics — you describe a topic in natural language (e.g. "investment advice with guaranteed returns"); Bedrock's classifier blocks any prompt or generation that matches it. This is the only policy that is fundamentally about subject matter rather than tone or content type.
Content filters — six categories: hate, insults, sexual, violence, misconduct, and prompt attack (this last one is a real, named category specifically for detecting jailbreak/prompt-injection attempts). Each category is scored at a severity — None / Low / Medium / High — and you configure a per-category filter strength: how aggressively that category blocks. A HIGH-severity match under a LOW-strength filter config behaves differently than the same match under a HIGH-strength config — the threshold is the tuning knob, not the detector.
Word filters — an exact custom denylist (profanity, competitor names, banned phrases) plus AWS's built-in profanity list.
Sensitive information (PII) filters — detect entity types (email, phone, SSN, credit card, and many more built-in types, plus custom regex-defined entity types) and, per entity type, either block the whole response or mask/anonymize just the matched span. This block-vs-mask choice matters operationally: masking preserves the rest of a useful response (an agent that shares a support article but redacts an accidentally-echoed email address is still useful); blocking a whole response over one PII match can be the wrong tradeoff for a low-sensitivity entity type.
Contextual grounding checks — output-only, and the policy that catches what none of the others can: a fluent, non-toxic, completely fabricated answer. Bedrock scores groundedness (is the claim supported by the source documents you supply?) and relevance (does the answer actually address the query?), each against a configurable threshold. Lab 02's token-overlap scorer is the deterministic, inspectable stand-in for what production Guardrails does with a purpose-built model.

Where each applies in the request lifecycle: most policies (topics, content, words, PII) can run as an input guardrail — before the model is ever invoked — and/or an output guardrail — after generation, before the caller sees the result. Contextual grounding is output-only by construction; there is nothing to ground a bare prompt against. The load-bearing production property, which Lab 02's test suite proves directly: a blocked input guardrail means the model is never called at all. No tokens spent, no risk of denied content ever entering the model's context window — which is also why Guardrails is cheaper, not just safer, than a post-hoc content filter bolted onto the end of a pipeline.

Every intervention — block or mask — produces a trace entry naming the policy, the specific rule, and the action taken; the overall result carries one GuardrailAction: NONE if nothing fired, GUARDRAIL_INTERVENED if anything did. AWS has also previewed Automated Reasoning checks as a further Guardrails safeguard: rather than statistical scoring, it validates factual claims against domain rules encoded as formal logic — a narrower, more rigorous check than the statistical policies above, aimed specifically at hallucination in domains (finance, HR policy, legal) where "probably grounded" isn't good enough.

7. Knowledge Bases: managed RAG-as-a-service

A Knowledge Base is the managed version of the hybrid retriever you built from scratch in Phase 05. The pipeline is the same shape; what "managed" buys you is the orchestration around it.

Ingestion: a data source (S3, a web crawler, Confluence, Salesforce, SharePoint) feeds raw documents through a configurable chunking strategy:

Fixed-size with overlap — the default: split into token windows of a fixed size, with overlapping tokens at each boundary so a sentence that straddles a cut doesn't lose context on either side. Simple, predictable, and what Lab 03's fixed_size_chunk implements.
Semantic chunking — split at sentence boundaries where consecutive-sentence embedding similarity drops below a threshold, so chunks track topic shifts rather than a fixed token count.
Hierarchical (parent-child) chunking — small child chunks are embedded and matched (small chunks make embedding similarity sharper), but the parent chunk containing each matched child is what gets returned for generation (bigger chunks give the model more context to work with). This is the one chunking choice that buys precision AND context instead of trading one for the other, which is why Lab 03 builds it as the second strategy, retrieving a matched child's full parent text via retrieve's parent-lookup.

Chunks are embedded (by a Titan Embeddings or Cohere Embed model, in production) and written into a vector store you provision — Bedrock Knowledge Bases does not ship a proprietary vector database. Supported options include OpenSearch Serverless (the common default), Aurora PostgreSQL with the pgvector extension, Pinecone, Redis Enterprise Cloud, and MongoDB Atlas — you own the store, its scaling, and its cost; Bedrock owns the ingestion pipeline and the query orchestration around it. This BYO-store model is a genuine, deliberate design choice (not a gap): it lets you reuse a vector store you already operate for other workloads, keep data residency inside infrastructure you control, and avoid a second proprietary database to manage.

Retrieval is exposed as two APIs:

Retrieve — returns ranked source chunks (text, score, location/citation metadata) and lets you build the prompt. This is the right choice when the Knowledge Base is one of several context sources feeding a larger agent (Phase 04's context assembler).
RetrieveAndGenerate — retrieval and generation and citation assembly in one managed call: it retrieves, builds a grounded prompt, calls the model, and returns an answer whose citations array ties each part of the response back to the exact source chunk that grounded it. Lab 03's retrieve_and_generate mirrors this response shape: an output.text plus a citations list of retrievedReferences, each carrying the source chunk's content and location.

Metadata filtering on Retrieve lets you tag ingested documents (category, ACL scope, publish date) and constrain retrieval to matching chunks — the mechanism that makes a single Knowledge Base usable for coarse multi-tenant or access-scoped RAG without standing up separate indexes per tenant.

Data-source sync is asynchronous: StartIngestionJob moves through STARTING → IN_PROGRESS → COMPLETE/FAILED, and a production caller polls or reacts to an event rather than blocking — exactly the lifecycle Lab 03's run_ingestion_job models, just made synchronous for testability.

8. Model customization: fine-tuning, continued pre-training, import, distillation

Bedrock offers four distinct paths to a model that behaves differently from the stock catalog entry, and knowing which applies to which situation is a real production judgment call:

Instruction fine-tuning — supervised training on labeled prompt-response pairs to steer a model's behavior on a specific task (a support-ticket classifier, a house style for generated copy). Needs the least data of the three training-based approaches and is the right default when you have a few hundred to a few thousand good examples of the exact input/output behavior you want.
Continued pre-training — further next-token training on your own unlabeled domain corpus (internal documentation, a specialized codebase, a scientific literature set) to shift the model's underlying knowledge and vocabulary toward your domain, without task-specific labels. Needs much more data than fine-tuning and is the right tool when the problem is "the model doesn't know enough about our domain," not "the model doesn't format its answers the way we want."
Custom Model Import — you already trained or fine-tuned a model outside Bedrock (open Llama/Mistral-family architectures, in Hugging Face safetensors format) and want to serve it through Bedrock's invocation API, IAM, VPC, and logging substrate without re-training on Bedrock. Billed on-demand (serverless), no Provisioned Throughput purchase required — the right choice when the training pipeline is already elsewhere and you want Bedrock purely as the serving/ops layer.
Distillation (Nova-specific) — train a smaller, cheaper Nova model to imitate a larger Nova model's outputs on your specific task, using the larger model's responses as training signal. The point is a cost/latency win at inference time once you've validated the larger model's quality on your task: you pay the larger model's cost during distillation, then serve the smaller, cheaper model in production at a fraction of the per-token cost, ideally with a proportionally small quality gap for that narrow task.

The ordering to reach for these in practice: try prompting and RAG first (cheapest, fastest to iterate); reach for fine-tuning when prompting plateaus on a well-defined task; reach for continued pre-training when the gap is domain knowledge, not task format; reach for distillation once you've proven a large model's quality and now want to cut serving cost; reach for Custom Model Import when the training already happened somewhere else.

9. Model evaluation jobs

Before shipping a model choice or a customization result, Bedrock's model evaluation jobs let you score candidate models against a labeled or synthetic dataset without hand-rolling an eval harness:

Automatic (built-in) metrics — standard, model-graded-free metrics like accuracy, robustness, and toxicity, computed directly over model outputs against a reference dataset. Fast and cheap, but limited to what a formula can measure (exact/fuzzy match, classification accuracy) — genuinely poor at judging open-ended generation quality.
LLM-as-judge evaluation — a Bedrock-hosted judge model scores each response against configurable dimensions (helpfulness, correctness, coherence, and others you can weight), the same technique this track builds from scratch in the eval-harness phase. Cheaper and faster than human evaluation, useful for iterating quickly, but inherits every known LLM-judge bias (position bias, verbosity bias, self-preference) if you don't control for it.
Human evaluation — bring your own reviewer team, or use an AWS-managed workforce, to score responses on a rubric. Slowest and most expensive, and still the gold standard for subjective quality, safety edge cases, and anything where you don't trust an automatic or LLM-graded score yet.

The production pattern: automatic metrics as a fast regression gate in CI, LLM-as-judge for rapid iteration across many candidate prompts/models, human evaluation as the final gate before a customer-facing change ships — the same "automatic metrics catch regressions, judges catch quality drift, humans catch what both miss" layering a mature eval program uses everywhere, not just on Bedrock.

10. The security model

Bedrock's trust boundary is architectural, the same discipline as Phase 00 applied to the model-serving layer:

Network isolation — VPC endpoints / PrivateLink. You can create interface VPC endpoints for bedrock, bedrock-runtime, bedrock-agent, and bedrock-agent-runtime so traffic between your VPC and Bedrock never traverses the public internet. This is the standard requirement for regulated workloads and the first thing a security review asks about.
Access control — IAM. Identity-based IAM policies gate who (which role, which principal) may call which Bedrock action against which resource (a specific modelId, a specific Guardrail, a specific Knowledge Base) — this is the Cedar/IAM access-control layer §6 contrasted against Guardrails; they compose (IAM decides who may invoke this model at all, Guardrails decides whether this specific text is safe).
Encryption at rest — KMS. Bedrock resources that persist data (customization job artifacts, Guardrail configurations, Provisioned Throughput metadata, Knowledge Base session data) support encryption with a customer-managed KMS key, not just an AWS-managed default key, for accounts that need to control and audit key usage themselves.
Audit — CloudTrail. Bedrock logs both control-plane calls (creating a Guardrail, purchasing Provisioned Throughput) and, when enabled, data-plane calls — InvokeModel/Converse requests as CloudTrail data events — the specific setting that lets you answer "who called which model, when" for compliance.
Model invocation logging. A separate, opt-in setting (off by default) that writes the full request and response payloads for every invocation to S3 and/or CloudWatch Logs. This is distinct from CloudTrail (which logs that a call happened and by whom) — invocation logging captures the actual prompt/completion content, which is exactly the data you want for debugging, quality analysis, and incident response, and exactly the data you must handle carefully under your compliance regime (PII, secrets in prompts) because it's a durable copy of everything users and your system ever sent the model.

11. Pricing model deep-dive with worked cost math

Bedrock's three capacity modes have three different pricing shapes, not just three different numbers — the shape is what you reason about on a whiteboard, since exact rate-card numbers change over time and vary per model. Treat every dollar figure below as illustrative, sized to be plausible for a mid-tier model, not an official rate card.

On-Demand is priced per 1,000 input tokens and per 1,000 output tokens, independently, and output tokens are typically priced several times higher than input tokens (generation is more expensive than reading context). For an illustrative model at \$0.003 / 1K input tokens and \$0.015 / 1K output tokens:

\[ \text{cost} = \frac{\text{input\_tokens}}{1000}\times 0.003 \;+\; \frac{\text{output\_tokens}}{1000}\times 0.015 \]

2,000,000 input tokens and 500,000 output tokens costs $2000 \times 0.003 + 500 \times 0.015 = \$6.00 + \$7.50 = \$13.50$ — exactly what Lab 01's estimate_on_demand_cost computes for those inputs.

Batch applies a flat discount (on the order of 50%, per AWS's own batch inference announcement) to the same per-token rate: the identical workload above costs roughly \$6.75 in batch. The lever you're pulling is latency tolerance, not workload size — batch doesn't get cheaper the bigger the job, it's a fixed percentage off, so the decision to batch is purely "can this wait."

Provisioned Throughput is priced per Model Unit per hour, a fixed cost independent of how many tokens you actually push through it that hour — which is precisely why the crossover math in §4 matters: below the breakeven request volume you are paying for idle capacity; above it, you're saving versus what the same volume would cost on-demand. The commitment term (none / 1-month / 6-month) trades a lower hourly rate for less flexibility, the same lever as EC2 Reserved Instances or Savings Plans — if you've reasoned about that tradeoff before, you already understand this one.

The whiteboard exercise an interviewer might actually run: "we're doing 40,000 requests/day at \$0.02/request on-demand average cost — does Provisioned Throughput make sense?" Using §4's formula with 1 Model Unit at an illustrative \$21/hour: breakeven is $21 \times 1 \times 24 / 0.02 = 25{,}200$ requests/day. At 40,000/day you're above breakeven — Provisioned Throughput wins, and you can say so with the formula, not a hunch.

12. Observability: metrics, logs, invocation logging

Bedrock publishes CloudWatch metrics under its own namespace, dimensioned by ModelId (and by Guardrail, Knowledge Base, or Provisioned Throughput ARN where applicable) — the metrics you'd actually alarm on include invocation count, invocation latency, input/output token counts, and invocation throttles/errors, giving you the p50/p95/p99 latency and the throttle rate per model without instrumenting anything yourself. Combine that with model invocation logging (§10) — the full request/response payloads, opt-in, to S3 and/or CloudWatch Logs — and CloudTrail data events (who called what, when) and you have the same three observability legs this track builds from scratch in Phase 14 (metrics, structured logs, and traces), provided as managed primitives instead of something you wire up yourself. The practical pattern: CloudWatch metrics for real-time alarming and dashboards, invocation logging for offline quality analysis and incident forensics, CloudTrail for compliance and access auditing — three different questions, three different stores, on purpose.

13. Competitors: a principal-level comparison

Every major cloud and several independents now offer "a unified API over multiple foundation models." The differences that matter for a real platform decision are model breadth vs lock-in, compliance posture, pricing shape, customization depth, and network isolation — not the marketing page.

Azure AI Foundry (the evolution of Azure OpenAI Service plus Azure AI Studio's model catalog) is Bedrock's closest structural peer: a managed model catalog (OpenAI's models with the deepest, often first-to-market integration, plus a broader third-party catalog similar in spirit to Bedrock's), Content Safety filters analogous to Guardrails, and Azure AI Search as the typical (though not exclusive) RAG backing store. Its differentiator is OpenAI model access depth — if your product's edge is a frontier OpenAI capability the day it ships, Azure is usually first, ahead of Bedrock and Vertex. Its enterprise compliance posture is exceptionally strong (Azure Government, extensive FedRAMP/HIPAA coverage) for shops already committed to the Microsoft ecosystem. Pricing shape mirrors Bedrock's (token-based on-demand plus Provisioned Throughput Units, Azure's name for the same reserved-capacity idea).

Google Vertex AI (Model Garden plus native Gemini) has the strongest first-party multimodal story — Gemini's native long-context and multimodal (video, audio) handling is a real technical edge, not just marketing — and Model Garden federates third-party and open models similarly to Bedrock's catalog. Vertex AI's tooling for classical ML (pipelines, feature store, Vertex AI Search) is more mature than either Bedrock's or Azure's if your organization already runs ML workloads on GCP, and its RAG story (Vertex AI Search / grounding) is comparably managed to Knowledge Bases. Compliance posture (FedRAMP, HIPAA, Assured Workloads for government/regulated workloads) is competitive with AWS and Azure without being categorically ahead.

Databricks Mosaic AI (Foundation Model APIs) is the right comparison when the organization's center of gravity is already the lakehouse, not the cloud provider: Foundation Model APIs offer pay-per-token and provisioned-throughput serving similar in shape to Bedrock, but the real draw is governance integration — Unity Catalog lineage, access control, and monitoring apply uniformly to your data pipelines and your model calls, which Bedrock cannot offer unless your data already lives entirely in AWS-native stores. Model breadth is narrower than Bedrock/Azure/ Vertex (fewer first-party frontier models), and it is not a general cloud platform — you are choosing it for lakehouse-native governance, not catalog breadth.

The OpenAI platform directly trades every multi-vendor benefit above for maximum feature depth on one vendor: the newest OpenAI capabilities land there first, full stop, and the API surface (Assistants/Responses API, real-time voice, fine-tuning) is purpose-built rather than translated through a normalization layer the way Converse is. The cost is exactly what you'd expect: no multi-vendor governance, no unified IAM/VPC/Guardrails/CloudTrail story shared with the rest of your cloud infrastructure, and a harder migration if you ever need to add a second model vendor.

Specialized inference clouds (Together AI, Fireworks AI, Groq, and Ray-based platforms like Anyscale) compete on a different axis entirely: raw inference speed and open-model pricing. Groq's custom LPU silicon delivers materially higher tokens-per-second than general-purpose GPU serving for supported open models — the right choice for a latency-critical product built on an open-weight model where Bedrock's latency-optimized inference (§5) doesn't cover your specific model. These platforms generally have thinner enterprise compliance stories (fewer of them carry FedRAMP/HIPAA-equivalent certifications than the big three clouds) and narrower model catalogs focused on popular open-weight families rather than proprietary frontier models — you adopt one for a specific speed or cost win on a specific open model, not as your general-purpose model platform.

The decision framework, stated as a principal engineer would say it out loud: pick Bedrock when you're already an AWS shop and want one operational envelope (IAM, VPC, KMS, CloudTrail, CloudWatch) across every model vendor with strong AWS compliance certifications (including AWS GovCloud for government workloads); pick Azure AI Foundry when OpenAI's frontier models are your product's core dependency and/or your enterprise identity and compliance stack is already Microsoft; pick Vertex AI when Gemini's native multimodal/long-context capability is a hard requirement or your ML platform is already GCP; pick Databricks Mosaic AI when governance over data and models through one lakehouse-native catalog matters more than model breadth; pick OpenAI directly when you need one vendor's frontier capability the instant it ships and multi-cloud governance isn't a near-term requirement; pick a specialized inference cloud when raw tokens-per-second or open-model price/performance is the product's actual bottleneck. None of these is "wrong" — the interview-winning move is naming the axis you're optimizing for and picking accordingly, not declaring a universal favorite.

14. Bedrock vs AgentCore vs Agents for Bedrock

Restated here because it is worth repeating until it is reflexive: Bedrock serves models. AgentCore runs agents. Agents for Bedrock was AWS's first, now largely superseded, attempt to bundle both into one low-code, single-vendor product.

Bedrock (this phase) has no concept of an agent loop at all — it is model catalog, invocation, capacity, Guardrails, Knowledge Bases, customization, and evaluation. There is nothing here that reasons in a loop or calls a tool.
Bedrock AgentCore (Phase 20) is the operational layer that hosts and runs an agent loop — session isolation via microVMs, a Gateway that turns APIs into MCP tools, Cedar policy, memory strategies — and is explicitly framework- and model-agnostic: it can run a LangGraph graph calling Bedrock models, or calling a model that has nothing to do with Bedrock at all. AgentCore's Runtime is a great place to host an agent that calls Bedrock's Converse API, but the two services do not require each other.
Agents for Bedrock ("Bedrock Agents," the older feature) is a single, fully-managed, console-configurable agent: you pick a Bedrock model, write an instruction prompt, define action groups (tools, typically Lambda-backed), and attach Knowledge Bases, and AWS runs the reasoning loop for you, its way. It is convenient and low-code, but you adopt AWS's agent shape — you cannot bring an existing LangGraph graph and run it unchanged the way you can on AgentCore. It predates AgentCore and, for teams building anything beyond a simple single-agent RAG assistant, has been effectively superseded by the AgentCore primitives — know it exists and what it was for, but don't lead with it as the modern answer.

The composition that actually ships in production: an agent framework (LangGraph, CrewAI, a hand-rolled loop) calls Bedrock's Converse API (this phase) for model invocation, backed by Guardrails and a Knowledge Base (this phase), and the whole agent is hosted, isolated, and Cedar-gated by AgentCore (Phase 20). All three AWS products, three different jobs, one stack.

15. Common misconceptions

"Bedrock is a model." It's a platform that serves many models behind one API — the model is a modelId string you pick from a catalog, not something Bedrock is.
"Converse replaces InvokeModel." Converse is built on top of InvokeModel per-provider translation; InvokeModel still exists and is still the only path to a provider-native feature Converse hasn't normalized yet.
"Every Bedrock model supports every Converse feature." No — tool use in particular is provider-dependent; calling toolConfig against an unsupported model is a real, documented failure mode, not something Converse silently papers over.
"Guardrails and IAM do the same job." Guardrails is content safety (is this text OK to show); IAM/Cedar is access control (is this principal allowed to do this action). An agent needs both, configured separately.
"PII detection always blocks." Per-entity-type, it can mask (redact just the span, keep the rest of the response) or block (replace the whole response) — a real configuration choice with real UX tradeoffs, not a fixed behavior.
"Provisioned Throughput guarantees unlimited scale." It guarantees throttle-free service up to your purchased capacity; beyond it, requests queue (or, in some configurations, still reject) — it is reserved capacity, not infinite capacity.
"Batch inference is just a discount." It is a discount and an async job model with no real-time SLA and no output-order guarantee — you are trading latency and ordering for cost, not getting the discount for free.
"Bedrock Knowledge Bases includes its own vector database." It does not — you provision OpenSearch Serverless, Aurora pgvector, Pinecone, Redis Enterprise Cloud, or MongoDB Atlas, and Bedrock manages the pipeline around whichever you pick.
"AgentCore is just the new name for Bedrock Agents." No — Agents for Bedrock is one managed agent AWS owns the loop for; AgentCore is composable operational primitives around an agent loop you own, in any framework, optionally calling Bedrock models or not.

16. Lab walkthrough

Build the three miniatures in order; each isolates one Bedrock capability and injects the model (and, in Lab 03, the embedder) as a pure function so everything stays deterministic and offline.

Lab 01 — Unified Invocation & Capacity. Implement four provider adapters (AnthropicAdapter, TitanAdapter, LlamaAdapter, CohereAdapter) each with a genuinely different native schema, a BedrockRuntime exposing both raw invoke_model and normalized converse/converse_stream, a TokenBucket for On-Demand throttling, a ProvisionedThroughputPool that queues instead of throttling, a BatchInferenceManager job lifecycle, a CrossRegionRouter, and the cost-math functions from §11. 40 tests.
Lab 02 — Guardrails. Implement denied topics, content filters with configurable per-category severity thresholds, word filters, PII detection with block-or-mask actions, and a contextual grounding check, wired as both an input guardrail (proving the model is never invoked on blocked input) and an output guardrail (proving PII is masked while ungrounded generations are blocked). 26 tests.
Lab 03 — Knowledge Bases. Implement fixed-size and hierarchical parent-child chunking, the deterministic hashing bag-of-words embedder (matching Phase 05's convention), an in-memory vector index with metadata filtering, the sync job lifecycle, and retrieve/retrieve_and_generate with citations. 29 tests.

Run each with LAB_MODULE=solution pytest test_lab.py -v first (green reference), then fill your lab.py to match, then read solution.py's main() output.

17. Success criteria

You can state, in one sentence, what Bedrock's control plane does versus its data plane.
You can name all four categories in Bedrock's model catalog and the tradeoff federation buys you.
You can explain why Converse exists and name a real feature (tool use) it cannot normalize across every provider.
You can do the On-Demand-vs-Provisioned-Throughput breakeven calculation from memory.
You can state the difference between Guardrails and Cedar/IAM without hesitating.
You can explain why Knowledge Bases needs a BYO vector store and name at least three supported options.
You can compare Bedrock to at least three competitors on model breadth, compliance, and pricing shape, and say when you'd pick each.
You can draw the Bedrock / AgentCore / Agents-for-Bedrock triangle without confusing them.
All three labs pass under both lab and solution (95 tests total).

18. Interview Q&A

Q: What's the difference between Bedrock and Bedrock AgentCore? A: Different layers. Bedrock is the model-serving platform — catalog, invocation (InvokeModel/Converse), capacity, Guardrails, Knowledge Bases, customization, evaluation. There is no agent loop in Bedrock at all. AgentCore is the operational layer that hosts and runs an agent loop — session isolation, Gateway/MCP tools, Cedar policy, memory — and is model-agnostic; it can run an agent that calls Bedrock models, or one that doesn't. If someone frames it as "Bedrock vs AgentCore," they have the mental model wrong — a real system typically uses both together.

Q: Why does Converse exist if InvokeModel already works? A: InvokeModel is a raw, provider-native passthrough — every provider (Anthropic, Titan, Llama, Cohere, ...) has its own request/response schema, so calling multiple models means special-casing each one. Converse is a translation layer built on top of InvokeModel: one normalized request/response shape (system prompt, multi-turn messages, tool use, streaming) that Bedrock translates to/from each provider's native format. The cost is that Converse's feature set is only as rich as the least-capable provider it fronts — tool use, for instance, isn't supported by every model, and calling it against an unsupported one is a real validation error, not silently ignored.

Q: Walk me through the difference between On-Demand, Provisioned Throughput, and Batch. A: On-Demand is pay-per-token with no commitment, rate-limited by a token-bucket-shaped quota that throttles (429, retry with backoff) when exceeded. Provisioned Throughput is a purchased, fixed pool of Model Units; within that capacity requests never throttle, and once saturated, requests queue rather than reject — a different failure mode entirely. Batch is an asynchronous job (SUBMITTED → IN_PROGRESS → COMPLETED/FAILED) with no real-time SLA, at roughly a 50% discount, for large delay-tolerant workloads. The choice comes down to traffic predictability (On-Demand for spiky/low volume), sustained volume above the cost breakeven (Provisioned Throughput), and latency tolerance (Batch).

Q: How do you decide if Provisioned Throughput is worth it? A: Solve for the request volume where the fixed hourly Model Unit cost equals your average on-demand spend over the same window: breakeven requests = (hourly Model Unit cost × units × hours) / average on-demand cost per request. Below that volume, On-Demand is cheaper; above it, Provisioned Throughput is, and you also get throttle-free latency as a bonus once you're paying for reserved capacity anyway.

Q: What's the difference between Guardrails and IAM/Cedar policy? A: Guardrails is content safety — it evaluates TEXT (a prompt or a generation) for denied topics, harmful content, PII, or hallucination, and blocks or masks it. IAM/Cedar (AgentCore's Policy service) is access control — it evaluates whether a PRINCIPAL may take an ACTION on a RESOURCE, like whether an agent may call a specific tool. They're independent layers with independent failure modes: a Guardrails intervention never authorizes a tool call, and a Cedar permit never excuses unsafe text. A production agent needs both.

Q: When would PII in a response get masked versus blocked? A: It's configured per entity type. Masking replaces just the matched span (an email address becomes a placeholder) and lets the rest of the response through — the right choice for lower-sensitivity types where the rest of the answer is still useful. Blocking replaces the entire response with a canned message — the right choice for high-sensitivity types (an SSN) where partial disclosure is still a problem, or where you'd rather fail loud than risk any leak.

Q: How does Knowledge Bases handle the vector store? A: It doesn't ship one — you provision OpenSearch Serverless, Aurora PostgreSQL with pgvector, Pinecone, Redis Enterprise Cloud, or MongoDB Atlas, and Bedrock manages ingestion (chunking, embedding, writing to your store) and retrieval on top of it. That's a deliberate BYO-store design: you can reuse infrastructure you already operate and keep data residency inside your own footprint, at the cost of having to provision and scale that store yourself.

Q: What's hierarchical chunking and why would you use it? A: Small child chunks are embedded and matched (small chunks make embedding similarity sharper and more precise); when a child matches, the larger parent chunk containing it is returned for generation (bigger context for the model to reason over). It's the one chunking strategy that buys precision AND context instead of trading one against the other — fixed-size-with-overlap has to pick a single chunk size that compromises between both goals.

Q: How do Retrieve and RetrieveAndGenerate differ? A: Retrieve returns ranked source chunks and metadata and lets you build the prompt yourself — the right call when the Knowledge Base is one of several context sources feeding a larger system. RetrieveAndGenerate does retrieval, prompt construction, generation, and citation assembly in one managed call, returning an answer whose citations tie back to the exact chunks that grounded it — the right call for a straightforward "answer this from our docs, with sources" feature.

Q: How would you pick between Bedrock, Azure AI Foundry, and Vertex AI for a new project? A: It comes down to what you're optimizing for, not a universal ranking. Already an AWS shop wanting one IAM/VPC/compliance envelope across vendors → Bedrock. Need OpenAI's frontier capabilities the day they ship, or already standardized on Microsoft identity/compliance → Azure AI Foundry. Need Gemini's native long-context/multimodal handling, or your ML platform is already GCP → Vertex AI. I'd also ask about existing cloud commitment and compliance certifications required (FedRAMP, HIPAA, GovCloud) before recommending, since those can be the deciding factor over any feature difference.

Q: What's the honest limitation of Bedrock's "unified API" pitch? A: Federation buys operational uniformity — the same IAM, VPC, Guardrails, and CloudWatch across every vendor — but the model catalog is only as fresh as Bedrock's integration work, and Converse's feature surface is the least common denominator across providers. If your product's edge depends on a specific vendor's newest capability the week it ships, that vendor's own API will usually get it first. Bedrock is optimized for multi-vendor flexibility and operational consistency, not for being the fastest path to any single vendor's frontier feature.

19. References

Amazon Bedrock — What is Amazon Bedrock (User Guide). https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html
Amazon Bedrock — product overview page. https://aws.amazon.com/bedrock/
Amazon Bedrock — Converse API (unified inference: system prompts, multi-turn, tool use, streaming). https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html
Amazon Bedrock — Guardrails for Amazon Bedrock (denied topics, content filters, word filters, sensitive information filters, contextual grounding checks). https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html
Amazon Bedrock — Knowledge Bases for Amazon Bedrock (ingestion, chunking strategies, supported vector stores, Retrieve/RetrieveAndGenerate). https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html
Amazon Bedrock — Provisioned Throughput. https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html
Amazon Bedrock — Batch inference. https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html
Amazon Bedrock — Increase throughput with cross-region inference. https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html
Amazon Bedrock — Custom Model Import. https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html
Amazon Bedrock — Model customization (fine-tuning, continued pre-training). https://docs.aws.amazon.com/bedrock/latest/userguide/custom-models.html
Amazon Bedrock — Model evaluation jobs (automatic metrics, human evaluation, LLM-as-judge). https://docs.aws.amazon.com/bedrock/latest/userguide/model-evaluation.html
Amazon Bedrock — Pricing. https://aws.amazon.com/bedrock/pricing/
Amazon Nova — foundation models overview. https://aws.amazon.com/ai/generative-ai/nova/
Agents for Amazon Bedrock (the older, fully-managed single-agent product) — for the §14 contrast. https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html
Amazon Bedrock AgentCore — for the §14 contrast; see also Phase 20 of this track. https://docs.aws.amazon.com/bedrock-agentcore/
Cedar policy language (the access-control contrast in §6). https://www.cedarpolicy.com/ and https://docs.cedarpolicy.com/
Azure AI Foundry documentation. https://learn.microsoft.com/en-us/azure/ai-foundry/
Google Cloud Vertex AI — Model Garden. https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models
Databricks Mosaic AI — Foundation Model APIs. https://docs.databricks.com/en/machine-learning/foundation-models/index.html
OWASP Top 10 for Large Language Model Applications (the prompt-attack content-filter category in §6). https://owasp.org/www-project-top-10-for-large-language-model-applications/

Agentic AI Engineer — Agent Infrastructure, Platforms & Production Systems