Last reviewed: June 2026
A scannable reference of architectural patterns the NCA-GENM exam tests. Read top-to-bottom, or jump to a section.
Diffusion outputs ignore the prompt; raising fidelity to text without wrecking image quality.
Increase classifier-free guidance scale; watch for over-saturation/artifacts and back off.
Why: Higher CFG tightens prompt adherence but too high causes burnt colors and unnatural detail — it is a tradeoff, not a free lever.
Diffusion sampling is too slow for an interactive demo; cut steps without obvious quality loss.
Switch to a faster ODE sampler (DPM-Solver++ / Euler) and reduce steps; validate with FID, not eyeballing.
Why: Modern samplers reach comparable quality in far fewer steps than ancestral DDPM sampling.
A multimodal pipeline has many moving parts and one weak result; deciding what to change next.
Run a controlled ablation — change one component at a time and measure against a fixed eval set.
Why: Changing several knobs at once makes the result uninterpretable; isolate the cause before scaling up.
Generation results vary run to run and you cannot compare two prompt variants fairly.
Fix the random seed (and sampler) so the only difference is the variable under test.
Why: Diffusion is stochastic; without a fixed seed you are comparing noise, not your change.
Generated images keep including an unwanted element (e.g. text, watermark, extra limbs).
Add a negative prompt describing what to exclude; combine with CFG.
Why: Negative prompting steers the unconditional branch away from named concepts — cheaper than retraining.
Picking the right metric to drive a text-to-image experiment.
Use FID for distributional image quality, CLIPScore for prompt-image alignment, and human preference for the final call.
Why: A single metric misleads: a model can score great FID while ignoring the prompt. Use both axes.
A vision-language model captioning task gives inconsistent, hallucinated captions.
Lower decoding temperature / use greedy or low top-p for factual captioning.
Why: High temperature increases creativity and hallucination; captioning wants determinism and grounding.
Iterating on conditioning is slow because each round evaluates the whole dataset.
Build a small, representative golden eval set for fast iteration; run full eval only on candidates.
Why: Tight feedback loops beat exhaustive but slow ones for the experimentation phase.
Need generated images to follow a precise pose, depth, or edge layout.
Add structural conditioning (ControlNet-style: pose/depth/canny) on top of the text prompt.
Why: Text prompts cannot specify exact spatial structure; an auxiliary conditioning map can.
Two checkpoints score nearly identical FID/CLIPScore; choosing which to ship.
Run a blind A/B human preference test on a held-out prompt set.
Why: Automated metrics saturate; human preference is the tiebreaker for generative quality.
Model looks great on the prompts you tuned on but poor on fresh prompts.
Hold out a separate prompt set never used during tuning and report on it.
Why: Tuning against your eval prompts overfits the experiment, not the model.
Outputs are close to target style but not quite; deciding between prompt tricks and training.
Exhaust prompting/conditioning and LoRA-style light fine-tune before full retraining.
Why: Cheapest intervention first — full retraining is rarely justified by a stylistic gap.
Explaining how a diffusion model generates an image.
Forward process adds noise to data; the model learns the reverse, denoising from pure noise to a sample.
Why: Generation is iterative denoising — the network predicts noise (or velocity) at each step.
Why high-resolution diffusion runs efficiently instead of operating on raw pixels.
Latent diffusion runs the diffusion process in a VAE's compressed latent space, then decodes to pixels.
Why: Operating in latent space cuts compute massively vs. pixel space for the same fidelity.
How a model learns to match images and text without per-pixel labels.
Contrastive pretraining (CLIP-style) pulls matched image-text pairs together and pushes mismatches apart in a shared embedding space.
Why: The shared space is what enables zero-shot classification and cross-modal retrieval.
Core mechanism that lets transformers relate tokens across a sequence or modalities.
Self/cross-attention computes weighted relevance among tokens; cross-attention conditions one modality on another.
Why: Cross-attention is how a diffusion U-Net injects text conditioning into image generation.
How a vision transformer turns an image into tokens.
Split the image into fixed patches, linearly embed each patch, add positional encodings.
Why: Patches are the visual analog of word tokens — that is what makes a unified transformer backbone possible.
Picking an architecture for image captioning vs. open-ended text-to-image chat.
Encoder-decoder (vision encoder + text decoder) for captioning; decoder-only multimodal LLM for flexible generation.
Why: The task shape — fixed input to text output vs. interleaved generation — drives the architecture.
How a single model consumes text and image together.
Project each modality into a shared token space and feed the combined sequence to one transformer.
Why: Token-level fusion lets attention reason across modalities jointly rather than late-fusing outputs.
Role of the VAE in a latent diffusion image generator.
The VAE encoder compresses images to latents for diffusion; its decoder reconstructs pixels at the end.
Why: Quality of the VAE caps the final image quality regardless of the diffusion model.
How audio enters a neural model for speech or audio generation.
Convert the waveform to a mel spectrogram (time-frequency image); models operate on that, then a vocoder reconstructs audio.
Why: Spectrograms make audio tractable for image-like and sequence models.
Why cross-modal search (text query, image results) works at all.
Both modalities are embedded into one aligned vector space; retrieval is nearest-neighbor across modalities.
Why: Alignment from contrastive training is the precondition — without it the spaces are not comparable.
Training a vision-language model and captions are noisy or weakly related to images.
Filter pairs by CLIP similarity threshold and re-caption low-alignment images.
Why: Poor caption-image alignment in data directly limits prompt adherence downstream.
Large scraped image-text corpus risks memorization and skewed eval.
Deduplicate near-identical images (perceptual hashing / embedding similarity) before training.
Why: Duplicates inflate memorization and leak into eval, overstating quality.
ASR training data mixes 8kHz phone audio and 44.1kHz studio audio.
Resample all clips to the model's expected sample rate (commonly 16kHz for ASR) and normalize loudness.
Why: Mismatched sample rates and levels corrupt spectrogram features and hurt recognition.
Diffusion training images vary wildly in size and aspect ratio.
Bucket by aspect ratio and resize/crop within buckets to the training resolution.
Why: Aspect-ratio bucketing avoids distortion from forcing everything square while keeping batches uniform.
Preparing a web-scraped multimodal corpus for a production model.
Run NSFW/CSAM and license/consent filtering before training; log provenance.
Why: Generative models reproduce training content — unsafe or unlicensed data becomes a legal and safety liability.
Short, sparse captions limit prompt diversity the model can handle.
Augment with synthetic detailed captions from a strong VLM, then quality-filter them.
Why: Richer captions broaden the prompt distribution the model learns to follow.
Video clips are long; deciding how to feed them to a multimodal model.
Sample frames at a fixed rate (or keyframes) plus aligned audio/transcript segments.
Why: Dense frame sampling is wasteful; aligned sparse sampling preserves temporal signal at lower cost.
Deploying a generative model as a production-ready, scalable inference endpoint on NVIDIA GPUs.
Serve it as an NVIDIA NIM microservice — prebuilt, optimized, OpenAI-compatible container.
Why: NIM packages the engine, runtime, and API so you skip hand-building TensorRT/Triton plumbing.
Need production ASR and TTS for a multimodal voice pipeline on NVIDIA hardware.
Use NVIDIA Riva for GPU-accelerated speech recognition and synthesis.
Why: Riva is the NVIDIA-stack answer for streaming, low-latency speech — not a general LLM tool.
Customizing or fine-tuning a foundation model within the NVIDIA ecosystem.
Use NVIDIA NeMo for training, fine-tuning (incl. PEFT/LoRA), and data curation.
Why: NeMo is the build/customize layer; NIM is the serve layer — keep the roles distinct.
Serving multiple models (vision encoder + LLM + vocoder) behind one inference server.
Use Triton Inference Server with model ensembles to chain them in one request path.
Why: Triton handles multi-framework, multi-model, and ensemble pipelines with dynamic batching.
Inference latency on a deployed model is too high for the target SLA.
Compile to TensorRT (with quantization where acceptable) for kernel-fused, lower-precision execution.
Why: TensorRT optimizes the graph for the specific GPU — the standard NVIDIA latency lever.
Building retrieval-augmented generation over a mixed image-and-text knowledge base.
Embed both modalities into a shared vector store, retrieve cross-modally, then ground the generator on the hits.
Why: Multimodal RAG needs a shared embedding space and a retriever, not just an LLM call.
Adding programmable input/output safety rails to a deployed multimodal app.
Wrap the model with NeMo Guardrails to enforce topic, safety, and grounding policies.
Why: Guardrails sit around the model as a policy layer rather than being baked into weights.
Generated outputs are biased toward one content type that dominates the dataset.
Profile the dataset distribution and rebalance or reweight underrepresented categories.
Why: Generative models mirror their data distribution — imbalance becomes output bias.
Understanding structure and coverage of a multimodal dataset before training.
Embed samples and inspect clusters (UMAP/t-SNE) to find gaps, dupes, and outliers.
Why: Embedding-space EDA surfaces coverage holes that raw counts miss.
A deployed multimodal model degrades on new production data.
Compare production embedding distribution to training; flag drift and trigger re-curation.
Why: Distribution shift, not model decay, is the usual cause of silent quality loss.
Captioning quality is poor and you suspect the data, not the model.
Compute caption-image CLIPScore distribution; a low-mean tail confirms a data alignment problem.
Why: Quantifying alignment separates a data issue from a modeling issue.
FID dropped but reviewers say images look worse; reconciling the contradiction.
Cross-check with CLIPScore and human eval; FID alone can be gamed by distributional tricks.
Why: No single metric is sufficient — interpret them together against ground truth.
A text-to-image model produces stereotyped depictions for occupation prompts.
Audit outputs across demographic axes; rebalance data and add prompt/guardrail mitigations.
Why: Representational harm is a first-class risk in generative media, not an edge case.
A feature can synthesize a real person's likeness or voice.
Require explicit consent, restrict to authorized identities, and watermark outputs.
Why: Likeness and voice cloning raise consent and impersonation harms that policy must gate.
Downstream consumers need to tell AI-generated media from real media.
Embed provenance metadata (C2PA-style) and/or an invisible watermark at generation time.
Why: Provenance signaling is the standard mitigation for synthetic-media misuse.
A multimodal RAG assistant confidently describes content not present in the retrieved image.
Constrain generation to retrieved evidence and add a grounding/citation check.
Why: Ungrounded multimodal output is hallucination — tie claims back to the source.
Preventing a deployed image generator from producing unsafe content.
Apply input-prompt and output-image safety classifiers plus a denylist; block and log violations.
Why: Safety must be enforced at both prompt and output stages — one side alone leaks.
Enforcing topic and safety policy on a multimodal chat app at runtime.
Use NeMo Guardrails for programmable input, output, and topical rails around the model.
Why: Guardrails give an auditable policy layer independent of model weights.
Stakeholders ask whether the model could reproduce copyrighted or private images.
Document data sources/licenses, deduplicate to limit memorization, and test for verbatim regeneration.
Why: Memorization risk is a trust and legal issue — transparency and dedup are the controls.