Build a generative AI application on AWS without managing model infrastructure.
Amazon Bedrock - fully managed access to foundation models (Anthropic Claude, Meta Llama, Amazon Titan, Stability, AI21, Mistral, Cohere) via a single API.
Why: No GPU provisioning, no model hosting; pay per token. SageMaker JumpStart is the alternative when you need a self-hosted endpoint in your VPC.
Large model pre-trained on diverse, mostly unlabeled data; adaptable to many downstream tasks via prompting, fine-tuning, or RAG.
Estimate how much input fits in one prompt and what drives inference cost.
Tokens are sub-word units. Context window = max tokens per request (input + output). Inference cost is roughly proportional to tokens processed.
Why: Token count, not request count, drives Bedrock pricing. If a long doc exceeds context window, chunk it or pick a larger-window model.
Pick output style: deterministic vs creative.
Low temperature (~0.0-0.3) β deterministic, repeatable. High temperature (~0.7-1.0) β creative, varied. Use 0 for classification or sentiment to get consistent labels.
Restrict the candidate token pool beyond temperature.
Top-K = consider only the K most-likely tokens. Top-P (nucleus) = consider tokens until cumulative probability reaches P.
Why: Top-P adapts the candidate set size to the distribution shape; Top-K is fixed-width.
Get LLM output in a specific style, length, or language.
Prompt engineering. Add explicit instructions ("Respond in French, under 50 words, formal tone").
Why: Cheaper and faster than fine-tuning, retraining, or changing model size for stylistic control.
Improve LLM accuracy on a specific task without retraining.
Few-shot prompting - embed 2-5 labeled input/output examples in the prompt before the new input.
Why: In-context learning lets the model pattern-match against examples without weight updates.
LLM gives wrong answers on multi-step reasoning problems.
Chain-of-thought prompting - instruct the model to walk through reasoning steps before the final answer ("Let's think step by step").
LLM generates text that sounds plausible but is factually wrong or fabricated.
Hallucination. Mitigate with RAG (ground in retrieved facts), Bedrock Guardrails, lower temperature, and human review of high-stakes outputs.
Power semantic search, clustering, or RAG retrieval over text or multimodal data.
Use an embedding model (e.g. Titan Embeddings, Cohere Embed) to convert content into dense vectors. Store and query in a vector DB.
Why: Embeddings capture semantic meaning so similar items land near each other in vector space (cosine / dot-product similarity).
Teach a foundation model specialized vocabulary (medical, legal, scientific) using lots of unlabeled domain text.
Continued pre-training on the unlabeled domain corpus.
Why: Continued pre-training updates the model's understanding of vocabulary and concepts; instruction fine-tuning teaches task behavior. Different goal, different data shape.
Managed RAG β Bedrock Knowledge Bases (handles vector store automatically). Custom vector DB β OpenSearch Service (k-NN), Aurora PostgreSQL with pgvector, Neptune Analytics, or RDS for PostgreSQL with pgvector.
Why: OpenSearch is the default for high-scale k-NN; pgvector reuses an existing relational DB.
Stakeholders or regulators require an explanation of how the model reaches its predictions.
Use interpretable models when possible (decision trees, linear/logistic regression). For complex models, use Partial Dependence Plots, SHAP feature importance via SageMaker Clarify, or SageMaker Model Cards.
Why: PDPs show the marginal effect of each feature; SHAP attributes contribution per prediction; model cards capture the whole story for audit.
Generative AI output may reproduce copyrighted material or be passed off as human-authored work.
Plagiarism / IP-infringement risk. Mitigate with citation requirements, content provenance tracking, watermarking where supported, human review, and clear AI-content disclosure policies.
Security, Compliance, and Governance for AI Solutions
Foundation-model app must keep prompts and responses on the AWS network - no public internet egress.
Bedrock with VPC endpoints (PrivateLink) for the runtime API. Block public Bedrock endpoints with SCPs at the org level.
Why: PrivateLink keeps requests private and avoids data leaving the VPC; SCPs enforce the rule across all accounts.
Capture Bedrock activity for monitoring, debugging, audit, and compliance.
Two complementary services. CloudTrail = who/when/from-where for every API call (identity, timestamp, source IP). Bedrock model invocation logging = the actual prompt/response payload, written to CloudWatch Logs or S3. Enable both.
Malicious user input tries to override system prompt, exfiltrate data, or trigger unintended actions.
Defense in depth: Bedrock Guardrails for content filtering, prompt templates that detect/ignore override patterns, input length limits, output validation, and least-privilege tool permissions for agents.
Why: No single mitigation is sufficient; combine input filtering, output filtering, and capability limits.
A custom model was trained on confidential data that shouldn't leak into responses.
Delete the model, scrub the confidential records from the training set, and retrain. Output filtering is not sufficient.
Why: Knowledge embedded in model weights cannot be reliably masked at inference; only retraining without that data removes it.
Decide what AWS secures vs. what the customer secures for an AI workload.
AWS Shared Responsibility Model: AWS = security OF the cloud (hardware, hypervisor, regions). Customer = security IN the cloud (data, IAM, KMS keys, network, app config).