Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest diagnose choice for this specific question. Inspect each judge's required inputs, reference fields, rubric and expected failure behavior. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — A medical expert flags unsafe triage advice, but engineering discards the feedback because an LLM judge rates it highly. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect inference table availability, payload schemas, trace IDs and user-feedback mapping. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — A medical expert flags unsafe triage advice, but engineering discards the feedback because an LLM judge rates it highly. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect production trace quality, error types, latency percentiles, user feedback and alert routes. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A medical expert flags unsafe triage advice, but engineering discards the feedback because an LLM judge rates it highly. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect scorer rubric, available trace spans, model response and known evaluation references. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect inference table availability, payload schemas, trace IDs and user-feedback mapping. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect production trace quality, error types, latency percentiles, user feedback and alert routes. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Add groundedness and policy-error monitoring with escalation thresholds alongside latency. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A policy assistant begins making incorrect claims after the source index updates, but no live records show which index answered. — is testing Inference tables and Agent Monitoring feedback loop. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Enable authorized usage/inference tracking and per-principal attribution with clear quota alerts. This action primarily relates to AI Gateway rate limits, usage tables and inference telemetry: AI Gateway offers controlled access, supported usage tracking and rate limiting to protect shared endpoints and measure consumption. However, the described case — A policy assistant begins making incorrect claims after the source index updates, but no live records show which index answered. — is testing Inference tables and Agent Monitoring feedback loop. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Enable approved inference tracing/logging with request IDs, redaction and retrieval metadata. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — A policy assistant begins making incorrect claims after the source index updates, but no live records show which index answered. — is testing Inference tables and Agent Monitoring feedback loop. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect changed prompt text, outcome metrics, case distribution and any regressed examples. This action primarily relates to Prompt changes evaluated against baseline: A prompt improvement should be measured against a fixed baseline set, not accepted because a few outputs sound better. However, the described case — A multilingual agent is proposed using a model card that reports poor performance on the target language. — is testing Model cards and task-specific limitations. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect trace context passages, token distribution, coverage of necessary conditions and retrieval rank. This action primarily relates to Chunk quality evaluation and boundary splitting: Retrieval failure can arise from overly small chunks that omit needed context or overly large chunks that dilute relevance. However, the described case — A multilingual agent is proposed using a model card that reports poor performance on the target language. — is testing Model cards and task-specific limitations. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect request/response sensitive spans, redaction logs, moderation findings and business exceptions. This action primarily relates to PII masking and output moderation as complementary guardrails: Input redaction and output checks serve distinct purposes; retain only necessary identifiers and validate actions separately. However, the described case — A multilingual agent is proposed using a model card that reports poor performance on the target language. — is testing Model cards and task-specific limitations. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not assume a managed Foundation Model endpoint grants unrestricted training rights or satisfies data residency automatically. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — A deploy pipeline overwrites the production prompt text each release and cannot restore yesterday's known-good response. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Enforce reproducible environment manifests and API compatibility checks in CI/CD. This action primarily relates to Deployed model dependency drift and environment compatibility: Models logged in experiments can fail in serving if Python dependencies or inference signatures differ between build and target. However, the described case — A deploy pipeline overwrites the production prompt text each release and cannot restore yesterday's known-good response. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Never rely on interactive notebook state or unpinned runtime packages when promoting a served model. This action primarily relates to MLflow pyfunc packaging for pre- and post-processing: A custom MLflow pyfunc model wraps request parsing, retriever/model invocation and output formatting inside a reproducible predict interface. However, the described case — A deploy pipeline overwrites the production prompt text each release and cannot restore yesterday's known-good response. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Regenerate embeddings from newly chunked source records and verify index synchronization. This action primarily relates to Embedding model swap and reindex compatibility: Changed embedding dimensions or model semantics typically require re-embedding and index consistency before comparing search quality. However, the described case — An invoice workflow needs exact structured amounts and dates but developers choose an unconstrained chat generator only. — is testing Specialized model versus general LLM. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Extract model/country fields and apply them as verified metadata filters to approved retrieval. This action primarily relates to Query-field driven prompt and retrieval filters: User intent and structured fields such as account_id should drive authorized SQL lookups or retrieval filters, not be ignored in free text. However, the described case — An invoice workflow needs exact structured amounts and dates but developers choose an unconstrained chat generator only. — is testing Specialized model versus general LLM. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Benchmark lower-cost dimension options and choose one meeting retrieval recall and latency goals. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — An invoice workflow needs exact structured amounts and dates but developers choose an unconstrained chat generator only. — is testing Specialized model versus general LLM. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Inject adversarial retrieved passages and verify they cannot cause unauthorized actions or data disclosure. This action primarily relates to Prompt-injection defenses for retrieved context: Retrieved content is lower-trust data; embedded instructions must not trigger privileged tool calls or override assistant rules. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Reproduce a slow/error case and confirm each relevant stage is observable without sensitive tokens. This action primarily relates to MLflow tracing in an Agent Framework workflow: MLflow tracing records agent steps, tool calls and latency; integration must retain child spans and inputs without leaking secrets. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Use the same quality rubrics for pre-release gates and production monitoring with separate datasets. This action primarily relates to Offline evaluation versus ongoing production monitoring: Offline test datasets catch regressions pre-release; production monitoring detects drift, real-user failures and evolving prompts. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not equate registering a model with approving its business safety or access to all governed tables. This action primarily relates to Unity Catalog registered model version and privileges: MLflow-logged models can be registered in Unity Catalog, with governed names, permissions and traceable version lineage. However, the described case — A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. — is testing Foundation Model API versus custom serving choices. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep integration tests on simple chains before adding unrelated agent orchestration complexity. This action primarily relates to Simple prompt-retrieval-generation chain assembly: A minimal chain composes a retriever, prompt, LLM and output parser, preserving required variables and failure states. However, the described case — A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. — is testing Foundation Model API versus custom serving choices. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Never rely on interactive notebook state or unpinned runtime packages when promoting a served model. This action primarily relates to MLflow pyfunc packaging for pre- and post-processing: A custom MLflow pyfunc model wraps request parsing, retriever/model invocation and output formatting inside a reproducible predict interface. However, the described case — A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. — is testing Foundation Model API versus custom serving choices. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Revise few-shot examples for format compliance and rerun previously passing examples. This action primarily relates to Prompt changes evaluated against baseline: A prompt improvement should be measured against a fixed baseline set, not accepted because a few outputs sound better. However, the described case — A RAG assistant passes an internal evaluation set but receives complaints about new policy terms after launch. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Reject unauthorized tool requests through server-side permissions even if the model produces a valid tool call. This action primarily relates to Unsafe model output and jailbreak response diagnosis: User-provided instructions may attempt to bypass agent policies or cause data leakage; classification and tool boundaries need separate tests. However, the described case — A RAG assistant passes an internal evaluation set but receives complaints about new policy terms after launch. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Classify current turn intent and route balances to authorized structured lookup rather than policy-only search. This action primarily relates to Conversation context and intent routing: Multi-turn assistants need explicit conversational state while routing each new request to the correct tool or topic. However, the described case — A RAG assistant passes an internal evaluation set but receives complaints about new policy terms after launch. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect logged model artifacts, environment requirements, signature and source index identifiers. This action primarily relates to RAG production bundle metadata and dependencies: A deployable RAG application needs tracked model code, retriever config, embedding model, dependency versions, signature and example input. However, the described case — A maintenance assistant queries a vector index by exact serial number without preserving metadata filters. — is testing Vector Search retrieval workflow and query integration. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect source table, index type, embedding model, index status and last successful sync. This action primarily relates to Databricks Vector Search index from governed Delta data: A vector index serves embeddings of chunked records maintained in a Unity Catalog governed Delta source table. However, the described case — A maintenance assistant queries a vector index by exact serial number without preserving metadata filters. — is testing Vector Search retrieval workflow and query integration. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect UC model names, signatures, version metadata, lineage and relevant ACLs. This action primarily relates to Unity Catalog registered model version and privileges: MLflow-logged models can be registered in Unity Catalog, with governed names, permissions and traceable version lineage. However, the described case — A maintenance assistant queries a vector index by exact serial number without preserving metadata filters. — is testing Vector Search retrieval workflow and query integration. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect p95 latency, queue depth, rate limits, cost and per-version error traces. This action primarily relates to Endpoint scale concurrency and operational rollout: Serving latency and reliability depend on provisioned capacity, request concurrency, quotas and controlled rollout. However, the described case — A team promotes a new prompt without rerunning citation accuracy or tool-permission regression cases. — is testing CI/CD pipelines for retrieval and agent components. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect prompt names, version history, aliases, tests and active referenced URI. This action primarily relates to MLflow Prompt Registry version aliases and promotion: Prompt Registry tracks immutable prompt versions with mutable aliases for staging and production, enabling change control and rollback. However, the described case — A team promotes a new prompt without rerunning citation accuracy or tool-permission regression cases. — is testing CI/CD pipelines for retrieval and agent components. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect runtime invocation order, prompt input names, stage outputs and errors. This action primarily relates to Simple prompt-retrieval-generation chain assembly: A minimal chain composes a retriever, prompt, LLM and output parser, preserving required variables and failure states. However, the described case — A team promotes a new prompt without rerunning citation accuracy or tool-permission regression cases. — is testing CI/CD pipelines for retrieval and agent components. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.