Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest implement choice for this specific question. Trace retriever and generator spans, score passage relevance and answer faithfulness separately. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — An agent endpoint gets accuracy complaints but serving logs capture only HTTP status, hiding response content and retrieval IDs. — is testing Inference logging and production trace privacy. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Enable authorized usage/inference tracking and per-principal attribution with clear quota alerts. This action primarily relates to AI Gateway rate limits, usage tables and inference telemetry: AI Gateway offers controlled access, supported usage tracking and rate limiting to protect shared endpoints and measure consumption. However, the described case — An agent endpoint gets accuracy complaints but serving logs capture only HTTP status, hiding response content and retrieval IDs. — is testing Inference logging and production trace privacy. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Monitor tail latency, timeout rate and affected user experience under representative load. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — An agent endpoint gets accuracy complaints but serving logs capture only HTTP status, hiding response content and retrieval IDs. — is testing Inference logging and production trace privacy. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Use budget limits and retry semantics when applying per-row model calls to large tables. This action primarily relates to SQL ai_query for governed batch inference: `ai_query()` invokes an available serving endpoint from SQL; suitability depends on supported runtimes, endpoint and workload. However, the described case — An agent produces raw document objects but the serving client expects a JSON list of cited answers. — is testing MLflow pyfunc packaging for pre- and post-processing. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Require approval/rollback for prompt and index changes that alter user-facing model behavior. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — An agent produces raw document objects but the serving client expects a JSON list of cited answers. — is testing MLflow pyfunc packaging for pre- and post-processing. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Treat prompt alias updates as change-controlled deployments with regression gates and auditing. This action primarily relates to MLflow Prompt Registry version aliases and promotion: Prompt Registry tracks immutable prompt versions with mutable aliases for staging and production, enabling change control and rollback. However, the described case — An agent produces raw document objects but the serving client expects a JSON list of cited answers. — is testing MLflow pyfunc packaging for pre- and post-processing. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Enable approved inference tracing/logging with request IDs, redaction and retrieval metadata. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Never label judge outputs as factual correctness when the required reference evidence is absent. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Check embedding dimensions or extracted-field validation against task-specific acceptance tests. This action primarily relates to Specialized model versus general LLM: Embedding, reranking, extraction and generative models solve different stages and should be evaluated for their actual role. However, the described case — A multi-step customer agent calls Genie and Vector Search but no span shows which returned an incorrect fact. — is testing MLflow tracing in an Agent Framework workflow. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run paired representative requests and record quality versus p95 latency and cost tradeoffs. This action primarily relates to Model selection by quality latency and budget: Select the smallest suitable model that meets a task's accuracy and latency constraints under realistic workload, not marketing rank. However, the described case — A multi-step customer agent calls Genie and Vector Search but no span shows which returned an incorrect fact. — is testing MLflow tracing in an Agent Framework workflow. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Reproduce representative task evaluation in the intended language and document limitations. This action primarily relates to Model cards and task-specific limitations: A model card records intended uses, supported language/context, safety limitations and evaluation scope relevant to selection. However, the described case — A multi-step customer agent calls Genie and Vector Search but no span shows which returned an incorrect fact. — is testing MLflow tracing in an Agent Framework workflow. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test extracted schema completeness or supervisor routing correctness on representative multi-step cases. This action primarily relates to Agent Bricks Information Extraction versus Multiagent Supervisor: Information Extraction maps source documents to structured fields; Multiagent Supervisor coordinates specialized capabilities for compound workflows. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Simulate a failed eligibility lookup and verify no payment or booking action occurs. This action primarily relates to Tool ordering and error isolation in multi-stage reasoning: An agent should retrieve authoritative context before taking consequential actions and validate tool outcomes at each stage. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test malformed inputs and inspect whether every response satisfies the parser and required field contract. This action primarily relates to Structured JSON schema prompt and format compliance: A model can produce plausible prose rather than machine-readable JSON unless the output contract and validation are explicit. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect task-appropriate quality metrics, inference cost, p95 latency and error segments. This action primarily relates to Model selection from quantitative task metrics: Select candidate LLMs by measured task success, latency, safety and token cost, not parameter count or leaderboard rank. However, the described case — A deployed claims assistant has 99.99% uptime but increasing incorrect eligibility advice. — is testing Production performance KPIs and incident thresholds. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect scorer rubric, available trace spans, model response and known evaluation references. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — A deployed claims assistant has 99.99% uptime but increasing incorrect eligibility advice. — is testing Production performance KPIs and incident thresholds. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect each judge's required inputs, reference fields, rubric and expected failure behavior. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — A deployed claims assistant has 99.99% uptime but increasing incorrect eligibility advice. — is testing Production performance KPIs and incident thresholds. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Load the registered bundle in a clean environment and validate retrieval-to-generation behavior end to end. This action primarily relates to RAG production bundle metadata and dependencies: A deployable RAG application needs tracked model code, retriever config, embedding model, dependency versions, signature and example input. However, the described case — New policies appear in Delta but a manually managed embedding index still returns retired versions. — is testing Databricks Vector Search index from governed Delta data. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Sign in as two users with different UC rights and confirm their answers cannot expose each other's data. This action primarily relates to Databricks Apps user-facing agent interface and OAuth identities: Databricks Apps supports authenticated interactive frontends; app credentials and user credentials have different scopes. However, the described case — New policies appear in Delta but a manually managed embedding index still returns retired versions. — is testing Databricks Vector Search index from governed Delta data. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Execute a small sample query then validate throughput, cost, failed rows and result parsing. This action primarily relates to SQL ai_query for governed batch inference: `ai_query()` invokes an available serving endpoint from SQL; suitability depends on supported runtimes, endpoint and workload. However, the described case — New policies appear in Delta but a manually managed embedding index still returns retired versions. — is testing Databricks Vector Search index from governed Delta data. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Simulate quality and latency threshold breaches and confirm responsible owners receive alerts. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — Four reviewers rate the same policy answers inconsistently and the team cannot determine whether prompt changes help. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Reproduce a failing request and verify a sanitized trace contains enough evidence to diagnose it. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — Four reviewers rate the same policy answers inconsistently and the team cannot determine whether prompt changes help. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run paired experiments with consistent evaluation data and evaluate cost-quality Pareto tradeoffs. This action primarily relates to Model selection from quantitative task metrics: Select candidate LLMs by measured task success, latency, safety and token cost, not parameter count or leaderboard rank. However, the described case — Four reviewers rate the same policy answers inconsistently and the team cannot determine whether prompt changes help. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect cited spans, source freshness, answer claims and missing-reference patterns. This action primarily relates to Hallucinated facts and unsupported citations: Fluent responses can invent facts and citations; faithfulness must be checked against supplied evidence, not writing style. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect model version, vector dimensions, index schema, pipeline lineage and freshness. This action primarily relates to Embedding model swap and reindex compatibility: Changed embedding dimensions or model semantics typically require re-embedding and index consistency before comparing search quality. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect whether the task requires real external state, permitted action or just text transformation. This action primarily relates to Agent tool wrappers versus direct LLM responses: A tool-using agent needs explicit callable interfaces and typed tool results, whereas plain generation requires no action execution. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Version embeddings with source and model identifiers to prevent silently mixing incompatible representations. This action primarily relates to Embedding model swap and reindex compatibility: Changed embedding dimensions or model semantics typically require re-embedding and index consistency before comparing search quality. However, the described case — A vendor proposes to copy managed model weights into an on-prem product contrary to provider terms. — is testing Open model license versus managed endpoint obligations. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Select a suitable embedding model rather than treating generated prose as a numeric embedding. This action primarily relates to Specialized model versus general LLM: Embedding, reranking, extraction and generative models solve different stages and should be evaluated for their actual role. However, the described case — A vendor proposes to copy managed model weights into an on-prem product contrary to provider terms. — is testing Open model license versus managed endpoint obligations. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect deployment manifests, tests, aliases, index versions and rollback dependency matrix. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect query embedding provenance, index schema, similarity results and selected context window. This action primarily relates to Vector Search retrieval workflow and query integration: Semantic retrieval uses an embedding model to query a compatible index, returns ranked passages and metadata for grounded generation. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect serving endpoint permissions, token path, user/app identity and downstream Unity Catalog access. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.