Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest diagnose choice for this specific question. Inspect which indicators are measured before deployment versus on live traffic and feedback. This action primarily relates to Offline evaluation versus ongoing production monitoring: Offline test datasets catch regressions pre-release; production monitoring detects drift, real-user failures and evolving prompts. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect vector count, dimension, encoder throughput, per-query latency and measured quality curves. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect experiment runs, source versions, parameter drift and metric definitions. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Start a clean container smoke test and exercise both old and new client payloads where compatibility is promised. This action primarily relates to Deployed model dependency drift and environment compatibility: Models logged in experiments can fail in serving if Python dependencies or inference signatures differ between build and target. However, the described case — A deploy pipeline overwrites the production prompt text each release and cannot restore yesterday's known-good response. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Load the logged pyfunc model in an isolated environment and test the declared signature on sample requests. This action primarily relates to MLflow pyfunc packaging for pre- and post-processing: A custom MLflow pyfunc model wraps request parsing, retriever/model invocation and output formatting inside a reproducible predict interface. However, the described case — A deploy pipeline overwrites the production prompt text each release and cannot restore yesterday's known-good response. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test model invocation under realistic throughput and verify output schema, quota and governance constraints. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — A deploy pipeline overwrites the production prompt text each release and cannot restore yesterday's known-good response. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test representative queries needing missing records and confirm either accurate retrieval or abstention. This action primarily relates to Authoritative source coverage for factual answers: A RAG model cannot answer accurately if the required authoritative record is missing or stale in its index. However, the described case — A parts catalog stores component numbers as strings while retrieval loses manufacturer and model columns. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Check that an exception and its definition are available to the generator after retrieval. This action primarily relates to Parent-child retrieval and hierarchical structure: Parent-child retrieval indexes targeted sections but expands to their broader parent context for generation. However, the described case — A parts catalog stores component numbers as strings while retrieval loses manufacturer and model columns. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Compare hit recall and answer grounding for sample chunk sizes with a fixed query set. This action primarily relates to Chunk size and overlap tradeoffs: Chunk size determines the number of vectors and coherence; overlap reduces boundary loss but increases index size and cost. However, the described case — A parts catalog stores component numbers as strings while retrieval loses manufacturer and model columns. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect experiment runs, source versions, parameter drift and metric definitions. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A medical summarization product considers a model evaluated only on short English news articles. — is testing Model cards and task-specific limitations. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect required output type, input modality and measured task capability rather than model size alone. This action primarily relates to Specialized model versus general LLM: Embedding, reranking, extraction and generative models solve different stages and should be evaluated for their actual role. However, the described case — A medical summarization product considers a model evaluated only on short English news articles. — is testing Model cards and task-specific limitations. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect trace context passages, token distribution, coverage of necessary conditions and retrieval rank. This action primarily relates to Chunk quality evaluation and boundary splitting: Retrieval failure can arise from overly small chunks that omit needed context or overly large chunks that dilute relevance. However, the described case — A medical summarization product considers a model evaluated only on short English news articles. — is testing Model cards and task-specific limitations. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Maintain an auditable source-rights register and recheck rights when a corpus is refreshed. This action primarily relates to Training and retrieval corpus license rights: Public accessibility does not automatically authorize copying, embedding or using content in commercial RAG services. However, the described case — An imported policy library contains a deliberately altered obsolete security procedure mixed into current instructions. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Redact or pseudonymize identity fields before vector indexing and enforce source access rules. This action primarily relates to Sensitive-field masking before embedding: Masking or tokenization can remove unnecessary PII from chunks before embedding and external model calls while retaining useful meaning. However, the described case — An imported policy library contains a deliberately altered obsolete security procedure mixed into current instructions. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not assert every index feature or hybrid/rerank option exists in every SKU, workspace or region. This action primarily relates to Standard versus storage-optimized Vector Search index: Index choice trades capacity, update speed, latency and storage economics; availability and feature support may vary by cloud/region. However, the described case — A prototype RAG agent works on one notebook cluster but fails because its deployment package omits the vector search client dependency. — is testing RAG production bundle metadata and dependencies. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep integration tests on simple chains before adding unrelated agent orchestration complexity. This action primarily relates to Simple prompt-retrieval-generation chain assembly: A minimal chain composes a retriever, prompt, LLM and output parser, preserving required variables and failure states. However, the described case — A prototype RAG agent works on one notebook cluster but fails because its deployment package omits the vector search client dependency. — is testing RAG production bundle metadata and dependencies. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Treat prompt alias updates as change-controlled deployments with regression gates and auditing. This action primarily relates to MLflow Prompt Registry version aliases and promotion: Prompt Registry tracks immutable prompt versions with mutable aliases for staging and production, enabling change control and rollback. However, the described case — A prototype RAG agent works on one notebook cluster but fails because its deployment package omits the vector search client dependency. — is testing RAG production bundle metadata and dependencies. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect experiment runs, source versions, parameter drift and metric definitions. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A product assistant must answer only for the model and country selected in the user form. — is testing Query-field driven prompt and retrieval filters. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect runnable input/output schemas, prompt variables and invocation traces around the failing stage. This action primarily relates to LangChain runnable composition and invocation: Composable runnable chains pass typed values between retrieval, prompt and model stages; each binding must preserve expected input keys. However, the described case — A product assistant must answer only for the model and country selected in the user form. — is testing Query-field driven prompt and retrieval filters. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect agent routing to Genie, space permissions, metric definitions and tool-returned values. This action primarily relates to Multi-agent Genie Space tool routing and governed answers: Genie Spaces can provide governed natural-language access to structured business metrics; an agent should choose authorized Genie tools over guessing SQL facts. However, the described case — A product assistant must answer only for the model and country selected in the user form. — is testing Query-field driven prompt and retrieval filters. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Start a clean container smoke test and exercise both old and new client payloads where compatibility is promised. This action primarily relates to Deployed model dependency drift and environment compatibility: Models logged in experiments can fail in serving if Python dependencies or inference signatures differ between build and target. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Query a new source chunk and a retired chunk to confirm expected inclusion and removal. This action primarily relates to Databricks Vector Search index from governed Delta data: A vector index serves embeddings of chunked records maintained in a Unity Catalog governed Delta source table. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Exercise failed tests and confirm release is blocked, then deploy and verify a passing version. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Sign in as two users with different UC rights and confirm their answers cannot expose each other's data. This action primarily relates to Databricks Apps user-facing agent interface and OAuth identities: Databricks Apps supports authenticated interactive frontends; app credentials and user credentials have different scopes. However, the described case — An analyst must classify millions of rows overnight inside a governed Databricks SQL pipeline. — is testing SQL ai_query for governed batch inference. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test model invocation under realistic throughput and verify output schema, quota and governance constraints. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — An analyst must classify millions of rows overnight inside a governed Databricks SQL pipeline. — is testing SQL ai_query for governed batch inference. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Simulate peak traffic and controlled rollback while monitoring end-to-end user outcomes. This action primarily relates to Endpoint scale concurrency and operational rollout: Serving latency and reliability depend on provisioned capacity, request concurrency, quotas and controlled rollout. However, the described case — An analyst must classify millions of rows overnight inside a governed Databricks SQL pipeline. — is testing SQL ai_query for governed batch inference. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Monitor tail latency, timeout rate and affected user experience under representative load. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — An invoice extraction agent passes generic relevance judges but loses an essential invoice currency field. — is testing MLflow GenAI tracing, built-in and custom scorers. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not treat request logging or gateway authentication as proof that underlying data access is authorized. This action primarily relates to AI Gateway rate limits, usage tables and inference telemetry: AI Gateway offers controlled access, supported usage tracking and rate limiting to protect shared endpoints and measure consumption. However, the described case — An invoice extraction agent passes generic relevance judges but loses an essential invoice currency field. — is testing MLflow GenAI tracing, built-in and custom scorers. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect required model type, deployment control, supported region, endpoint quotas and operational burden. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — A chain logged under Python package version A fails to deserialize when the model serving image ships incompatible version B. — is testing Deployed model dependency drift and environment compatibility. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect logged model artifacts, environment requirements, signature and source index identifiers. This action primarily relates to RAG production bundle metadata and dependencies: A deployable RAG application needs tracked model code, retriever config, embedding model, dependency versions, signature and example input. However, the described case — A chain logged under Python package version A fails to deserialize when the model serving image ships incompatible version B. — is testing Deployed model dependency drift and environment compatibility. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect UC model names, signatures, version metadata, lineage and relevant ACLs. This action primarily relates to Unity Catalog registered model version and privileges: MLflow-logged models can be registered in Unity Catalog, with governed names, permissions and traceable version lineage. However, the described case — A chain logged under Python package version A fails to deserialize when the model serving image ships incompatible version B. — is testing Deployed model dependency drift and environment compatibility. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.