You need a single place to discover models, deploy them, and build AI apps on Azure.
Use the Microsoft Foundry portal - it hosts the model catalog, deployments, playground, and agent tooling.
Why: Foundry is the unified hub; individual Azure AI services exist but Foundry is where you compose and deploy solutions.
Reference
You want the model to always answer as a polite support agent regardless of the question asked.
Set behavior and persona in the system prompt; put the specific question in the user prompt.
Why: The system prompt frames overall behavior and rules; the user prompt is the per-turn request.
Reference
You picked a model in the catalog and need it callable from an app.
Create a deployment for the model in the Foundry portal, which gives an endpoint and key.
Why: A model in the catalog isn't usable until deployed; the deployment exposes the callable endpoint.
Reference
You want to test prompts and tune temperature before writing any code.
Use the chat playground in the Foundry portal to interact with the deployed model and adjust parameters.
Why: The playground lets you iterate on prompts and settings interactively; no SDK needed to experiment.
You need to call the deployed chat model from application code.
Use the Foundry (Azure AI) SDK to create a chat client that sends messages to the deployment endpoint.
Why: The SDK wraps the endpoint with a typed client; you pass system and user messages and read the completion.
Reference
Your app must authenticate to the deployed model.
Use the deployment's endpoint URL with an API key or Microsoft Entra ID (Azure AD) credential.
Why: Key-based auth is simplest; Entra ID is more secure and avoids embedding secrets in code.
You want an AI assistant that follows instructions and uses tools, built without much code.
Create a single agent in the Foundry portal - define its instructions, model, and tools (the Agent Service).
Why: The portal agent builder configures behavior and tools declaratively; you don't hand-write the orchestration loop.
Reference
Your agent should always cite sources and refuse off-topic requests.
Encode these rules in the agent's instructions (its system-level guidance).
Why: Agent instructions steer consistent behavior across turns, similar to a system prompt for a plain chat model.
Your agent must answer from your company documents, not just its training data.
Give the agent a knowledge/grounding tool (e.g., file search or Azure AI Search) so it retrieves your data.
Why: Grounding/RAG supplies current, private context; without it the model can hallucinate or use stale knowledge.
You need a custom app to drive a Foundry agent programmatically.
Build an agent client app with the Foundry SDK - create a thread, add messages, run the agent, read responses.
Why: The SDK exposes threads, runs, and messages so your app can integrate the agent into any workflow.
Reference
You must build an app that extracts sentiment and entities from incoming text.
Use Azure AI Language (text analysis) via the SDK or REST, accessed through Foundry, calling sentiment and NER features.
Why: For classic NLP tasks, the Language service is purpose-built and cheaper than prompting a general LLM.
Reference
A user wants to speak a question and have a deployed model answer it.
Send the audio to a multimodal model that accepts speech input, or transcribe first then prompt the model.
Why: Multimodal models can take audio directly; otherwise use speech-to-text to feed a text model.
Your app needs high-quality transcription and natural spoken output.
Use Azure AI Speech within Foundry Tools for speech-to-text and text-to-speech.
Why: The Speech service offers tuned recognition and lifelike neural voices, beyond what a chat model alone provides.
Reference
You need the app to read responses aloud in a natural-sounding voice.
Use Azure AI Speech text-to-speech with a neural voice; control prosody with SSML if needed.
Why: Neural voices sound natural; SSML lets you tune pace, pitch, and pronunciation.
An app must describe what is happening in a user-supplied photo and answer questions about it.
Send the image to a multimodal model in Foundry and prompt it with the question.
Why: Multimodal LLMs reason over image content; the classic Vision service only returns fixed tags and captions.
Reference
An app must produce images from text descriptions on demand.
Deploy a text-to-image model (e.g., a DALL-E / image generation model) in Foundry and call it from your app.
Why: Image generation models create visuals from prompts; a vision model only analyzes existing images.
Reference
You need an app that classifies images and reads printed text from them.
Build a vision app using Azure AI Vision (image analysis and OCR) accessed through Foundry.
Why: Azure AI Vision provides ready image analysis and OCR; you don't need to train a model for common tasks.
Reference
An app must extract printed and handwritten text from scanned pages.
Use the OCR (Read) capability of Azure AI Vision to return the recognized text and its location.
Why: OCR returns raw text with coordinates; structured-field extraction needs Content Understanding instead.
You must extract structured fields (totals, dates, line items) from invoices and forms.
Use Azure AI Content Understanding in Foundry Tools to extract structured data from documents and forms.
Why: Content Understanding pulls labeled fields; plain OCR only returns unstructured text.
Reference
You need structured descriptions and metadata extracted from a batch of images.
Use Azure AI Content Understanding to analyze images and return structured output.
Why: Content Understanding produces consistent structured results across content types, beyond a free-text caption.
You must turn call recordings into structured summaries with key data points.
Use Azure AI Content Understanding on the audio to transcribe and extract structured fields.
Why: Content Understanding combines transcription with extraction; Speech alone only gives the transcript.
You need scenes, topics, and key fields pulled from training videos.
Use Azure AI Content Understanding for video to extract structured insights across modalities.
Why: It analyzes audio and visual streams together to produce structured output, not just a transcript.
Reference
You must add your company's private FAQ knowledge to model answers, with minimal effort.
Ground the model with retrieval (RAG) over your documents rather than fine-tuning.
Why: RAG injects current data at query time and is simpler/cheaper; fine-tuning changes behavior, not knowledge freshness.
You must block harmful or unsafe text and image outputs from a deployed model.
Enable Azure AI Content Safety filters on the deployment to detect and block harmful content.
Why: Content Safety enforces responsible-AI guardrails at runtime; the base model alone isn't guaranteed safe.
Reference
After deploying, you need to measure response quality and watch for drift.
Use Foundry evaluation and monitoring tools to score outputs and track metrics over time.
Why: Evaluation quantifies quality (groundedness, relevance); monitoring catches regressions in production.
You need to organize models, agents, and connections for one application.
Create a Foundry project, which groups deployments, connected resources, and tools for that solution.
Why: A project is the workspace boundary; connections link external resources like Azure AI Search or storage.
Reference