Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest diagnose choice for this specific question. Inspect scorer rubric, available trace spans, model response and known evaluation references. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — A production team logs raw access tokens and complete customer chats when adding observability. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect production trace quality, error types, latency percentiles, user feedback and alert routes. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A production team logs raw access tokens and complete customer chats when adding observability. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect task-appropriate quality metrics, inference cost, p95 latency and error segments. This action primarily relates to Model selection from quantitative task metrics: Select candidate LLMs by measured task success, latency, safety and token cost, not parameter count or leaderboard rank. However, the described case — A production team logs raw access tokens and complete customer chats when adding observability. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Split by task-relevant sections and filter top passages before prompt assembly. This action primarily relates to Chunk quality evaluation and boundary splitting: Retrieval failure can arise from overly small chunks that omit needed context or overly large chunks that dilute relevance. However, the described case — A team wants to deploy a new prompt based solely on improving live thumbs-up counts. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Review model license and select an authorized commercial alternative before deployment. This action primarily relates to Open model license versus managed endpoint obligations: Model availability on a hub does not itself confer commercial training, redistribution or hosting rights. However, the described case — A team wants to deploy a new prompt based solely on improving live thumbs-up counts. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Compare compact versus larger embeddings on labeled retrieval quality and total vector-index footprint. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — A team wants to deploy a new prompt based solely on improving live thumbs-up counts. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect encoder token limits, chunk length percentiles, truncation counters and source suffix omissions. This action primarily relates to Embedding context window and chunk token distribution: The embedding encoder must accept the actual tokenized chunk length; silently truncating chunks can drop relevant suffix facts. However, the described case — A troubleshooting assistant retrieves a part number but omits the note that the part is discontinued. — is testing Chunk quality evaluation and boundary splitting. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect tool-call traces, retrieved source text and permission decisions for instruction-origin confusion. This action primarily relates to Prompt-injection defenses for retrieved context: Retrieved content is lower-trust data; embedded instructions must not trigger privileged tool calls or override assistant rules. However, the described case — A troubleshooting assistant retrieves a part number but omits the note that the part is discontinued. — is testing Chunk quality evaluation and boundary splitting. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect vector count, dimension, encoder throughput, per-query latency and measured quality curves. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — A troubleshooting assistant retrieves a part number but omits the note that the part is discontinued. — is testing Chunk quality evaluation and boundary splitting. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect the business decision, data owners, actor permissions, known inputs and success criteria. This action primarily relates to Translate use-case requirements into pipeline inputs and outputs: Business acceptance criteria must map to concrete source inputs, generated fields, correctness measures and permission constraints. However, the described case — A corporate help desk must answer policy questions from a vetted HR handbook and cite the passage used. — is testing Agent Bricks Knowledge Assistant suitability. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Identify whether the output is a category, extractive field, free-form summary or grounded answer. This action primarily relates to Task choice: classification versus generative summarization: Classification predicts a category whereas summarization compresses meaning; the desired output determines the appropriate task. However, the described case — A corporate help desk must answer policy questions from a vetted HR handbook and cite the passage used. — is testing Agent Bricks Knowledge Assistant suitability. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Determine whether output is fixed fields, document answers or orchestration of multiple tasks with tool calls. This action primarily relates to Agent Bricks Information Extraction versus Multiagent Supervisor: Information Extraction maps source documents to structured fields; Multiagent Supervisor coordinates specialized capabilities for compound workflows. However, the described case — A corporate help desk must answer policy questions from a vetted HR handbook and cite the passage used. — is testing Agent Bricks Knowledge Assistant suitability. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep each specialist's permissions bounded and audit handoffs rather than granting a coordinator unrestricted tools. This action primarily relates to Agent Bricks Information Extraction versus Multiagent Supervisor: Information Extraction maps source documents to structured fields; Multiagent Supervisor coordinates specialized capabilities for compound workflows. However, the described case — An IT assistant can read service status but its generic tool credentials also allow deletion of production tables. — is testing Agent tool schemas, least privilege and retries. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Treat absence of relevant retrieved evidence as an abstention condition, not a reason to invent missing facts. This action primarily relates to Retriever-generator chain component boundaries: A basic RAG chain separates user input, query transformation, retrieval, prompt assembly, model generation and output validation. However, the described case — An IT assistant can read service status but its generic tool credentials also allow deletion of production tables. — is testing Agent tool schemas, least privilege and retries. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Revisit cost/latency budgets whenever context window or retrieval size changes significantly. This action primarily relates to Latency, cost and retrieval targets as design requirements: A valid pipeline design has measurable quality, latency, token and throughput constraints before model selection. However, the described case — An IT assistant can read service status but its generic tool credentials also allow deletion of production tables. — is testing Agent tool schemas, least privilege and retries. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Regenerate embeddings from newly chunked source records and verify index synchronization. This action primarily relates to Embedding model swap and reindex compatibility: Changed embedding dimensions or model semantics typically require re-embedding and index consistency before comparing search quality. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Reject unauthorized tool requests through server-side permissions even if the model produces a valid tool call. This action primarily relates to Unsafe model output and jailbreak response diagnosis: User-provided instructions may attempt to bypass agent policies or cause data leakage; classification and tool boundaries need separate tests. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Remove unused side-effect tools and use a plain summarization chain with bounded output. This action primarily relates to Agent tool wrappers versus direct LLM responses: A tool-using agent needs explicit callable interfaces and typed tool results, whereas plain generation requires no action execution. However, the described case — An employee asks first about leave policy, then about their leave balance, and the chain keeps searching policy text only. — is testing Conversation context and intent routing. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Reproduce representative task evaluation in the intended language and document limitations. This action primarily relates to Model cards and task-specific limitations: A model card records intended uses, supported language/context, safety limitations and evaluation scope relevant to selection. However, the described case — A maintenance chatbot fabricates a firmware version not found in its manual index. — is testing Hallucinated facts and unsupported citations. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test data questions for correct Genie routing, SQL-backed values and denial of unauthorized tables. This action primarily relates to Multi-agent Genie Space tool routing and governed answers: Genie Spaces can provide governed natural-language access to structured business metrics; an agent should choose authorized Genie tools over guessing SQL facts. However, the described case — A maintenance chatbot fabricates a firmware version not found in its manual index. — is testing Hallucinated facts and unsupported citations. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Embed test chunks with relevant facts at the end and confirm they remain retrievable. This action primarily relates to Embedding context window and chunk token distribution: The embedding encoder must accept the actual tokenized chunk length; silently truncating chunks can drop relevant suffix facts. However, the described case — A maintenance chatbot fabricates a firmware version not found in its manual index. — is testing Hallucinated facts and unsupported citations. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. A/B test chunk segmentation on frozen questions and verify evidence sufficiency and prompt-budget compliance. This action primarily relates to Chunk quality evaluation and boundary splitting: Retrieval failure can arise from overly small chunks that omit needed context or overly large chunks that dilute relevance. However, the described case — A multilingual agent is proposed using a model card that reports poor performance on the target language. — is testing Model cards and task-specific limitations. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run equivalent retrieval tests including storage and refresh cost for candidate encoders. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — A multilingual agent is proposed using a model card that reports poor performance on the target language. — is testing Model cards and task-specific limitations. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Rerun a controlled experiment and confirm metrics match within expected stochastic tolerance. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A multilingual agent is proposed using a model card that reports poor performance on the target language. — is testing Model cards and task-specific limitations. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Compare outputs against task-specific labeled examples and inspect mismatches between required and produced formats. This action primarily relates to Task choice: classification versus generative summarization: Classification predicts a category whereas summarization compresses meaning; the desired output determines the appropriate task. However, the described case — An IT assistant can read service status but its generic tool credentials also allow deletion of production tables. — is testing Agent tool schemas, least privilege and retries. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Simulate a failed eligibility lookup and verify no payment or booking action occurs. This action primarily relates to Tool ordering and error isolation in multi-stage reasoning: An agent should retrieve authoritative context before taking consequential actions and validate tool outcomes at each stage. However, the described case — An IT assistant can read service status but its generic tool credentials also allow deletion of production tables. — is testing Agent tool schemas, least privilege and retries. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Check answer fidelity, citation paths and forbidden document access across test identities. This action primarily relates to Agent Bricks Knowledge Assistant suitability: A governed knowledge assistant grounds conversational responses in curated documents and access-controlled retrieval. However, the described case — An IT assistant can read service status but its generic tool credentials also allow deletion of production tables. — is testing Agent tool schemas, least privilege and retries. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Register immutable prompt versions, evaluate staging and move the approved production alias. This action primarily relates to MLflow Prompt Registry version aliases and promotion: Prompt Registry tracks immutable prompt versions with mutable aliases for staging and production, enabling change control and rollback. However, the described case — An agent must search an external research API requiring a secret, while internal company data uses managed connectors. — is testing Managed versus external versus custom MCP server integration. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Move to a supported SQL warehouse/runtime and verify endpoint region/permission prerequisites. This action primarily relates to SQL ai_query for governed batch inference: `ai_query()` invokes an available serving endpoint from SQL; suitability depends on supported runtimes, endpoint and workload. However, the described case — An agent must search an external research API requiring a secret, while internal company data uses managed connectors. — is testing Managed versus external versus custom MCP server integration. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Build a CI gate for prompt schema, retrieval relevance, agent tools and controlled promotion across environments. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — An agent must search an external research API requiring a secret, while internal company data uses managed connectors. — is testing Managed versus external versus custom MCP server integration. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect task-appropriate quality metrics, inference cost, p95 latency and error segments. This action primarily relates to Model selection from quantitative task metrics: Select candidate LLMs by measured task success, latency, safety and token cost, not parameter count or leaderboard rank. However, the described case — A team measures 'answer correctness' with a golden-answer comparator but its evaluation data includes only user questions. — is testing Ground-truth requirement for correctness versus reference-free judges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect logging configuration, sampling policy, trace privacy, missing identifiers and audit destinations. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — A team measures 'answer correctness' with a golden-answer comparator but its evaluation data includes only user questions. — is testing Ground-truth requirement for correctness versus reference-free judges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect token volumes, QPS, endpoint pricing, cache hit rate and user/workload attribution. This action primarily relates to Databricks usage and token cost controls: Reduce LLM spend through token budgets, caching, smaller appropriate models and AI Gateway quotas while preserving acceptance quality. However, the described case — A team measures 'answer correctness' with a golden-answer comparator but its evaluation data includes only user questions. — is testing Ground-truth requirement for correctness versus reference-free judges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.