Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest verify choice for this specific question. Rerun a controlled experiment and confirm metrics match within expected stochastic tolerance. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A compliance summary app adds examples but stops honoring the one-line required answer format. — is testing Prompt changes evaluated against baseline. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Use the same quality rubrics for pre-release gates and production monitoring with separate datasets. This action primarily relates to Offline evaluation versus ongoing production monitoring: Offline test datasets catch regressions pre-release; production monitoring detects drift, real-user failures and evolving prompts. However, the described case — A compliance summary app adds examples but stops honoring the one-line required answer format. — is testing Prompt changes evaluated against baseline. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Check embedding dimensions or extracted-field validation against task-specific acceptance tests. This action primarily relates to Specialized model versus general LLM: Embedding, reranking, extraction and generative models solve different stages and should be evaluated for their actual role. However, the described case — A compliance summary app adds examples but stops honoring the one-line required answer format. — is testing Prompt changes evaluated against baseline. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep monitoring schemas and retention governance versioned across model deployment changes. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not equate an LLM judge score with unquestionable ground truth; calibrate against reviewed labels. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Never assume diagnostics justify indefinite storage of unrestricted personal data or secrets. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Bind question and retrieval context into a complete prompt before invoking the model and parsing output. This action primarily relates to Simple prompt-retrieval-generation chain assembly: A minimal chain composes a retriever, prompt, LLM and output parser, preserving required variables and failure states. However, the described case — A team persists all raw conversations in an unprotected shared cache to preserve agent memory. — is testing Persistent conversation memory with authorized datastore. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Deploy an authenticated Databricks App backend that invokes the agent and enforces user-specific data permissions. This action primarily relates to Databricks Apps user-facing agent interface and OAuth identities: Databricks Apps supports authenticated interactive frontends; app credentials and user credentials have different scopes. However, the described case — A team persists all raw conversations in an unprotected shared cache to preserve agent memory. — is testing Persistent conversation memory with authorized datastore. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Restrict endpoint CAN QUERY access to approved principals and test permission boundaries. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — A team persists all raw conversations in an unprotected shared cache to preserve agent memory. — is testing Persistent conversation memory with authorized datastore. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect whether the task is grounded document Q&A, extraction, tool coordination or cross-agent orchestration. This action primarily relates to Agent Bricks Knowledge Assistant suitability: A governed knowledge assistant grounds conversational responses in curated documents and access-controlled retrieval. However, the described case — A policy assistant produces unsupported answers because its chain sends only the user question to the LLM. — is testing Retriever-generator chain component boundaries. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect the business decision, data owners, actor permissions, known inputs and success criteria. This action primarily relates to Translate use-case requirements into pipeline inputs and outputs: Business acceptance criteria must map to concrete source inputs, generated fields, correctness measures and permission constraints. However, the described case — A policy assistant produces unsupported answers because its chain sends only the user question to the LLM. — is testing Retriever-generator chain component boundaries. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect the prompt hierarchy, few-shot coverage, untrusted-data delimiters and observed format deviations. This action primarily relates to Instruction hierarchy and bounded prompt examples: Few-shot examples and explicit boundaries can clarify expected outputs, but embedded source documents should not become new system instructions. However, the described case — A policy assistant produces unsupported answers because its chain sends only the user question to the LLM. — is testing Retriever-generator chain component boundaries. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Enforce policy-layer refusals and prevent disclosure through prompt and output checks without relying on wording alone. This action primarily relates to Unsafe model output and jailbreak response diagnosis: User-provided instructions may attempt to bypass agent policies or cause data leakage; classification and tool boundaries need separate tests. However, the described case — A benefits RAG assistant cites a real policy page but its claimed deadline does not appear in the cited passage. — is testing Hallucinated facts and unsupported citations. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Compare both models on the same held-out support questions, citations, cost and safety cases. This action primarily relates to Model evaluation with paired benchmarks: Accuracy, faithfulness, relevance and refusal behavior require task-specific scoring, and a model switch needs common test conditions. However, the described case — A benefits RAG assistant cites a real policy page but its claimed deadline does not appear in the cited passage. — is testing Hallucinated facts and unsupported citations. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Treat the page as untrusted evidence and enforce tool authorization and destination allowlists on the server. This action primarily relates to Prompt-injection defenses for retrieved context: Retrieved content is lower-trust data; embedded instructions must not trigger privileged tool calls or override assistant rules. However, the described case — A benefits RAG assistant cites a real policy page but its claimed deadline does not appear in the cited passage. — is testing Hallucinated facts and unsupported citations. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Enforce reproducible environment manifests and API compatibility checks in CI/CD. This action primarily relates to Deployed model dependency drift and environment compatibility: Models logged in experiments can fail in serving if Python dependencies or inference signatures differ between build and target. However, the described case — A maintenance assistant queries a vector index by exact serial number without preserving metadata filters. — is testing Vector Search retrieval workflow and query integration. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Never rely on interactive notebook state or unpinned runtime packages when promoting a served model. This action primarily relates to MLflow pyfunc packaging for pre- and post-processing: A custom MLflow pyfunc model wraps request parsing, retriever/model invocation and output formatting inside a reproducible predict interface. However, the described case — A maintenance assistant queries a vector index by exact serial number without preserving metadata filters. — is testing Vector Search retrieval workflow and query integration. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not assume a managed Foundation Model endpoint grants unrestricted training rights or satisfies data residency automatically. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — A maintenance assistant queries a vector index by exact serial number without preserving metadata filters. — is testing Vector Search retrieval workflow and query integration. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Reproduce a failing request and verify a sanitized trace contains enough evidence to diagnose it. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run `mlflow.genai.evaluate()` on a fixed set with documented built-in and custom scorer outputs. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test a missing-ground-truth record to verify the judge abstains or uses the appropriate alternate scorer. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — A batch classification job spikes costs because every row invokes a large general-purpose model. — is testing Databricks usage and token cost controls. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect user conversation state, current intent, entity resolution and selected tool trace. This action primarily relates to Conversation context and intent routing: Multi-turn assistants need explicit conversational state while routing each new request to the correct tool or topic. However, the described case — A brainstorming app wants creative draft ideas but is incorrectly evaluated on exact-string matching to one reference. — is testing Temperature and deterministic format tradeoffs. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect cited spans, source freshness, answer claims and missing-reference patterns. This action primarily relates to Hallucinated facts and unsupported citations: Fluent responses can invent facts and citations; faithfulness must be checked against supplied evidence, not writing style. However, the described case — A brainstorming app wants creative draft ideas but is incorrectly evaluated on exact-string matching to one reference. — is testing Temperature and deterministic format tradeoffs. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect task difficulty, expected concurrency, context size, throughput and model benchmark evidence. This action primarily relates to Model selection by quality latency and budget: Select the smallest suitable model that meets a task's accuracy and latency constraints under realistic workload, not marketing rank. However, the described case — A brainstorming app wants creative draft ideas but is incorrectly evaluated on exact-string matching to one reference. — is testing Temperature and deterministic format tradeoffs. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect which indicators are measured before deployment versus on live traffic and feedback. This action primarily relates to Offline evaluation versus ongoing production monitoring: Offline test datasets catch regressions pre-release; production monitoring detects drift, real-user failures and evolving prompts. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect required output type, input modality and measured task capability rather than model size alone. This action primarily relates to Specialized model versus general LLM: Embedding, reranking, extraction and generative models solve different stages and should be evaluated for their actual role. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect model version, vector dimensions, index schema, pipeline lineage and freshness. This action primarily relates to Embedding model swap and reindex compatibility: Changed embedding dimensions or model semantics typically require re-embedding and index consistency before comparing search quality. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Configure approved inference tables and trace metadata to correlate outputs with deployed model/index versions. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — Four reviewers rate the same policy answers inconsistently and the team cannot determine whether prompt changes help. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Add validated reference answers or use a reference-free groundedness criterion with clear scope. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — Four reviewers rate the same policy answers inconsistently and the team cannot determine whether prompt changes help. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Enable authorized usage/inference tracking and per-principal attribution with clear quota alerts. This action primarily relates to AI Gateway rate limits, usage tables and inference telemetry: AI Gateway offers controlled access, supported usage tracking and rate limiting to protect shared endpoints and measure consumption. However, the described case — Four reviewers rate the same policy answers inconsistently and the team cannot determine whether prompt changes help. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect scorer rubric, available trace spans, model response and known evaluation references. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — A production team logs raw access tokens and complete customer chats when adding observability. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect production trace quality, error types, latency percentiles, user feedback and alert routes. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A production team logs raw access tokens and complete customer chats when adding observability. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect task-appropriate quality metrics, inference cost, p95 latency and error segments. This action primarily relates to Model selection from quantitative task metrics: Select candidate LLMs by measured task success, latency, safety and token cost, not parameter count or leaderboard rank. However, the described case — A production team logs raw access tokens and complete customer chats when adding observability. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.