Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest diagnose choice for this specific question. Inspect whether the task is grounded document Q&A, extraction, tool coordination or cross-agent orchestration. This action primarily relates to Agent Bricks Knowledge Assistant suitability: A governed knowledge assistant grounds conversational responses in curated documents and access-controlled retrieval. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Identify whether the output is a category, extractive field, free-form summary or grounded answer. This action primarily relates to Task choice: classification versus generative summarization: Classification predicts a category whereas summarization compresses meaning; the desired output determines the appropriate task. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect request volume, concurrency, response deadline, token budget and evaluation quality targets. This action primarily relates to Latency, cost and retrieval targets as design requirements: A valid pipeline design has measurable quality, latency, token and throughput constraints before model selection. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Enable approved inference tracing/logging with request IDs, redaction and retrieval metadata. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — A team measures 'answer correctness' with a golden-answer comparator but its evaluation data includes only user questions. — is testing Ground-truth requirement for correctness versus reference-free judges. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Configure approved inference tables and trace metadata to correlate outputs with deployed model/index versions. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — A team measures 'answer correctness' with a golden-answer comparator but its evaluation data includes only user questions. — is testing Ground-truth requirement for correctness versus reference-free judges. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Apply supported AI Gateway endpoint quotas or rate limits by workload and inspect usage traces. This action primarily relates to AI Gateway rate limits, usage tables and inference telemetry: AI Gateway offers controlled access, supported usage tracking and rate limiting to protect shared endpoints and measure consumption. However, the described case — A team measures 'answer correctness' with a golden-answer comparator but its evaluation data includes only user questions. — is testing Ground-truth requirement for correctness versus reference-free judges. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Compare both models on the same held-out support questions, citations, cost and safety cases. This action primarily relates to Model evaluation with paired benchmarks: Accuracy, faithfulness, relevance and refusal behavior require task-specific scoring, and a model switch needs common test conditions. However, the described case — A team declares a new assistant better but cannot identify which prompt or model version generated the winning score. — is testing Experiment tracking and version reproducibility. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Split by task-relevant sections and filter top passages before prompt assembly. This action primarily relates to Chunk quality evaluation and boundary splitting: Retrieval failure can arise from overly small chunks that omit needed context or overly large chunks that dilute relevance. However, the described case — A team declares a new assistant better but cannot identify which prompt or model version generated the winning score. — is testing Experiment tracking and version reproducibility. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Design a precise instruction on evidence and uncertainty, then compare against baseline using the same evaluation set. This action primarily relates to Prompt changes evaluated against baseline: A prompt improvement should be measured against a fixed baseline set, not accepted because a few outputs sound better. However, the described case — A team declares a new assistant better but cannot identify which prompt or model version generated the winning score. — is testing Experiment tracking and version reproducibility. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Trace agent reasoning/tool interactions with source/result provenance to isolate the faulty component. This action primarily relates to MLflow tracing in an Agent Framework workflow: MLflow tracing records agent steps, tool calls and latency; integration must retain child spans and inputs without leaking secrets. However, the described case — A public application generates harmful abusive content after an unusual user prompt despite sanitized inputs. — is testing PII masking and output moderation as complementary guardrails. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Pin the retrieval index version and trace reference set in experiments before comparing runs. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A public application generates harmful abusive content after an unusual user prompt despite sanitized inputs. — is testing PII masking and output moderation as complementary guardrails. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Validate account and order IDs, enforce identity scope, and retrieve the matching order status before generation. This action primarily relates to Query-field driven prompt and retrieval filters: User intent and structured fields such as account_id should drive authorized SQL lookups or retrieval filters, not be ignored in free text. However, the described case — A public application generates harmful abusive content after an unusual user prompt despite sanitized inputs. — is testing PII masking and output moderation as complementary guardrails. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Version preprocessing rules and sample exceptions so cleansing never strips legal requirements. This action primarily relates to Boilerplate and navigation filtering in source corpora: Duplicated headers, disclaimers and navigation menus contaminate embeddings and cause irrelevant hits. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Select chunk granularity by measured retrieval results, not solely by minimizing index records. This action primarily relates to Chunk size and overlap tradeoffs: Chunk size determines the number of vectors and coherence; overlap reduces boundary loss but increases index size and cost. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Retain source offsets and section IDs so citations remain traceable after rechunking. This action primarily relates to Semantic boundaries versus fixed-window chunks: Headings, paragraphs and tables can require structure-aware splitting rather than arbitrary token windows. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect tool-call traces, retrieved source text and permission decisions for instruction-origin confusion. This action primarily relates to Prompt-injection defenses for retrieved context: Retrieved content is lower-trust data; embedded instructions must not trigger privileged tool calls or override assistant rules. However, the described case — A support assistant retrieves a full 60-page manual and overwhelms the model prompt with unrelated sections. — is testing Chunk quality evaluation and boundary splitting. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect encoder token limits, chunk length percentiles, truncation counters and source suffix omissions. This action primarily relates to Embedding context window and chunk token distribution: The embedding encoder must accept the actual tokenized chunk length; silently truncating chunks can drop relevant suffix facts. However, the described case — A support assistant retrieves a full 60-page manual and overwhelms the model prompt with unrelated sections. — is testing Chunk quality evaluation and boundary splitting. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect vector count, dimension, encoder throughput, per-query latency and measured quality curves. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — A support assistant retrieves a full 60-page manual and overwhelms the model prompt with unrelated sections. — is testing Chunk quality evaluation and boundary splitting. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Sanitize sensitive fields and restrict access/retention in the logging pipeline. This action primarily relates to Inference logging and production trace privacy: Inference logs and tracing permit diagnosis of real issues but must be sampled, retained and redacted safely. However, the described case — A helpdesk agent receives frequent negative user feedback but no monitoring process turns it into reproducible examples. — is testing Inference tables and Agent Monitoring feedback loop. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Define scoring anchors, calibrate experts on disputed samples and use aligned feedback as reliable evaluation labels. This action primarily relates to Human SME feedback rubric and reviewer calibration: Domain experts should use a shared rubric, calibrate disagreements and maintain adjudicated examples to improve model evaluation. However, the described case — A helpdesk agent receives frequent negative user feedback but no monitoring process turns it into reproducible examples. — is testing Inference tables and Agent Monitoring feedback loop. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Monitor tail latency, timeout rate and affected user experience under representative load. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A helpdesk agent receives frequent negative user feedback but no monitoring process turns it into reproducible examples. — is testing Inference tables and Agent Monitoring feedback loop. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Log model/prompt/data identifiers and metrics to a run with reproducible evaluation artifacts. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Reject unsupported suitability assumptions and benchmark clinical-document handling with proper controls. This action primarily relates to Model cards and task-specific limitations: A model card records intended uses, supported language/context, safety limitations and evaluation scope relevant to selection. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Monitor sampled production traces and maintain the offline regression set for controlled fixes. This action primarily relates to Offline evaluation versus ongoing production monitoring: Offline test datasets catch regressions pre-release; production monitoring detects drift, real-user failures and evolving prompts. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not store sensitive chat traces indefinitely merely because longer memory improves convenience. This action primarily relates to Persistent conversation memory with authorized datastore: Transient prompt context differs from persistent user state; durable memory needs explicit storage, identity scope and retention. However, the described case — A deployed agent hardcodes prompt version 1, so validated updates never reach the application. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep integration tests on simple chains before adding unrelated agent orchestration complexity. This action primarily relates to Simple prompt-retrieval-generation chain assembly: A minimal chain composes a retriever, prompt, LLM and output parser, preserving required variables and failure states. However, the described case — A deployed agent hardcodes prompt version 1, so validated updates never reach the application. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Set load thresholds and versioned fallback rather than assuming a single demo proves production scalability. This action primarily relates to Endpoint scale concurrency and operational rollout: Serving latency and reliability depend on provisioned capacity, request concurrency, quotas and controlled rollout. However, the described case — A deployed agent hardcodes prompt version 1, so validated updates never reach the application. — is testing MLflow Prompt Registry version aliases and promotion. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run cases with complete and missing required input fields and verify schema, access scope and usefulness. This action primarily relates to Translate use-case requirements into pipeline inputs and outputs: Business acceptance criteria must map to concrete source inputs, generated fields, correctness measures and permission constraints. However, the described case — A travel agent books a flight before checking ticket rules and traveler approval. — is testing Tool ordering and error isolation in multi-stage reasoning. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Inspect a full trace to confirm retrieved evidence and user intent reach the model and that citations point to real chunks. This action primarily relates to Retriever-generator chain component boundaries: A basic RAG chain separates user input, query transformation, retrieval, prompt assembly, model generation and output validation. However, the described case — A travel agent books a flight before checking ticket rules and traveler approval. — is testing Tool ordering and error isolation in multi-stage reasoning. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Load-test p95 latency and per-request cost against realistic payload sizes and concurrency. This action primarily relates to Latency, cost and retrieval targets as design requirements: A valid pipeline design has measurable quality, latency, token and throughput constraints before model selection. However, the described case — A travel agent books a flight before checking ticket rules and traveler approval. — is testing Tool ordering and error isolation in multi-stage reasoning. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test a missing-ground-truth record to verify the judge abstains or uses the appropriate alternate scorer. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Query sampled live traces and demonstrate a complaint can be reproduced and scored on an offline test. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Re-evaluate previously disputed cases and confirm inter-rater agreement improves after calibration. This action primarily relates to Human SME feedback rubric and reviewer calibration: Domain experts should use a shared rubric, calibrate disagreements and maintain adjudicated examples to improve model evaluation. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.