Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest safeguard choice for this specific question. Do not hide expert disagreement by excluding difficult cases or averaging away severe failures. This action primarily relates to Human SME feedback rubric and reviewer calibration: Domain experts should use a shared rubric, calibrate disagreements and maintain adjudicated examples to improve model evaluation. However, the described case — A support RAG app fails some answers, but team cannot tell whether retrieval or generation caused each error. — is testing MLflow GenAI tracing, built-in and custom scorers. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Use usage monitoring and budgets to detect regressions when prompts, indexes or models change. This action primarily relates to Databricks usage and token cost controls: Reduce LLM spend through token budgets, caching, smaller appropriate models and AI Gateway quotas while preserving acceptance quality. However, the described case — A support RAG app fails some answers, but team cannot tell whether retrieval or generation caused each error. — is testing MLflow GenAI tracing, built-in and custom scorers. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Revisit the choice as workload and evaluation data change instead of permanently trusting benchmark ranking. This action primarily relates to Model selection from quantitative task metrics: Select candidate LLMs by measured task success, latency, safety and token cost, not parameter count or leaderboard rank. However, the described case — A support RAG app fails some answers, but team cannot tell whether retrieval or generation caused each error. — is testing MLflow GenAI tracing, built-in and custom scorers. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect token distribution, overlap proportion, expected retrieval granularity and vector count. This action primarily relates to Chunk size and overlap tradeoffs: Chunk size determines the number of vectors and coherence; overlap reduces boundary loss but increases index size and cost. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect section relationships, chunk hierarchy and whether cross-references survive extraction. This action primarily relates to Parent-child retrieval and hierarchical structure: Parent-child retrieval indexes targeted sections but expands to their broader parent context for generation. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect labeled query/chunk relevance, false positive hits, top-k settings and evaluation coverage. This action primarily relates to Retrieval recall precision and ranking diagnostics: Measure hit recall, precision, MRR or nDCG on labeled query-to-relevant-chunk pairs before tuning retrieval. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Order identity verification, purchase lookup, policy check, user confirmation and transaction tool invocation. This action primarily relates to Tool ordering and error isolation in multi-stage reasoning: An agent should retrieve authoritative context before taking consequential actions and validate tool outcomes at each stage. However, the described case — A benefits intake app requires a JSON object with claim_id, decision and reason but the assistant returns Markdown paragraphs. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Separate low-trust quoted input from higher-priority task instructions and include an example that preserves the schema. This action primarily relates to Instruction hierarchy and bounded prompt examples: Few-shot examples and explicit boundaries can clarify expected outputs, but embedded source documents should not become new system instructions. However, the described case — A benefits intake app requires a JSON object with claim_id, decision and reason but the assistant returns Markdown paragraphs. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Split read-only diagnostic tools from authorized destructive operations with separate permissions. This action primarily relates to Agent tool schemas, least privilege and retries: Well-described tool schemas constrain parameters, and idempotent retries prevent duplicate side effects. However, the described case — A benefits intake app requires a JSON object with claim_id, decision and reason but the assistant returns Markdown paragraphs. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not treat a model hub popularity score as certification of deployment fitness. This action primarily relates to Model cards and task-specific limitations: A model card records intended uses, supported language/context, safety limitations and evaluation scope relevant to selection. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Evaluate capable long-context models on legal clauses and latency/cost acceptance thresholds. This action primarily relates to Model selection by quality latency and budget: Select the smallest suitable model that meets a task's accuracy and latency constraints under realistic workload, not marketing rank. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep component roles explicit so updates to a generation model do not silently alter retrieval vectors. This action primarily relates to Specialized model versus general LLM: Embedding, reranking, extraction and generative models solve different stages and should be evaluated for their actual role. However, the described case — An Agent Framework service reports only end-to-end duration, hiding which tool call causes a 12-second delay. — is testing MLflow tracing in an Agent Framework workflow. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not mistake text masking for permission checks, lawful data use or complete safety assurance. This action primarily relates to PII masking and output moderation as complementary guardrails: Input redaction and output checks serve distinct purposes; retain only necessary identifiers and validate actions separately. However, the described case — An Agent Framework service reports only end-to-end duration, hiding which tool call causes a 12-second delay. — is testing MLflow tracing in an Agent Framework workflow. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep guardrail effectiveness under test after every model or tool integration change. This action primarily relates to Unsafe model output and jailbreak response diagnosis: User-provided instructions may attempt to bypass agent policies or cause data leakage; classification and tool boundaries need separate tests. However, the described case — An Agent Framework service reports only end-to-end duration, hiding which tool call causes a 12-second delay. — is testing MLflow tracing in an Agent Framework workflow. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Ingest a current price source with source timestamps and governed refresh frequency. This action primarily relates to Authoritative source coverage for factual answers: A RAG model cannot answer accurately if the required authoritative record is missing or stale in its index. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Strip template boilerplate, repeated navigation and low-information text before embedding. This action primarily relates to Boilerplate and navigation filtering in source corpora: Duplicated headers, disclaimers and navigation menus contaminate embeddings and cause irrelevant hits. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Use document-aware sections plus parent-child metadata to connect symptom to resolution. This action primarily relates to Semantic boundaries versus fixed-window chunks: Headings, paragraphs and tables can require structure-aware splitting rather than arbitrary token windows. However, the described case — A scanned JPEG safety certificate yields no extractable text in the pipeline. — is testing Format-specific document extraction and OCR needs. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Compare intended answer fields to current indexed sources and freshness requirements. This action primarily relates to Authoritative source coverage for factual answers: A RAG model cannot answer accurately if the required authoritative record is missing or stale in its index. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect section relationships, chunk hierarchy and whether cross-references survive extraction. This action primarily relates to Parent-child retrieval and hierarchical structure: Parent-child retrieval indexes targeted sections but expands to their broader parent context for generation. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect table extraction, repeated column headings, units, row identity and exact filter fields. This action primarily relates to Table-aware chunking and metadata-rich retrieval: Tabular rows need headers, units and column semantics preserved when transformed into retrieval documents. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run an incremental update and verify new, changed and deleted source chunks are reflected exactly once. This action primarily relates to Delta and Unity Catalog pipeline for chunk tables: A governed retrieval corpus stores chunk text, IDs, source lineage and metadata in managed Delta/Unity Catalog tables before indexing. However, the described case — An energy tariff PDF is chunked by arbitrary text lines and rate values lose the currency and zone column. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Check that an exception and its definition are available to the generator after retrieval. This action primarily relates to Parent-child retrieval and hierarchical structure: Parent-child retrieval indexes targeted sections but expands to their broader parent context for generation. However, the described case — An energy tariff PDF is chunked by arbitrary text lines and rate values lose the currency and zone column. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Verify that target retrieval returns complete condition-and-exception evidence for representative questions. This action primarily relates to Semantic boundaries versus fixed-window chunks: Headings, paragraphs and tables can require structure-aware splitting rather than arbitrary token windows. However, the described case — An energy tariff PDF is chunked by arbitrary text lines and rate values lose the currency and zone column. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Simulate quality and latency threshold breaches and confirm responsible owners receive alerts. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A data extraction agent has identical field-F1 across two architectures but one exceeds the production latency budget. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Benchmark per-answer cost and quality on representative traffic before promoting optimizations. This action primarily relates to Databricks usage and token cost controls: Reduce LLM spend through token budgets, caching, smaller appropriate models and AI Gateway quotas while preserving acceptance quality. However, the described case — A data extraction agent has identical field-F1 across two architectures but one exceeds the production latency budget. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run `mlflow.genai.evaluate()` on a fixed set with documented built-in and custom scorer outputs. This action primarily relates to MLflow GenAI tracing, built-in and custom scorers: MLflow GenAI traces capture components, and scorers/judges can evaluate a full answer or individual retrieval and tool spans. However, the described case — A data extraction agent has identical field-F1 across two architectures but one exceeds the production latency budget. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Never equate a successful embedding API call with a governed and reproducible source-data pipeline. This action primarily relates to Delta and Unity Catalog pipeline for chunk tables: A governed retrieval corpus stores chunk text, IDs, source lineage and metadata in managed Delta/Unity Catalog tables before indexing. However, the described case — An energy tariff PDF is chunked by arbitrary text lines and rate values lose the currency and zone column. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Never assume embedding similarity alone is a security boundary or reliable exact identifier lookup. This action primarily relates to Hybrid sparse and dense retrieval with filters: Hybrid search combines lexical exact-match and semantic similarity; metadata filtering narrows authorized relevant content. However, the described case — An energy tariff PDF is chunked by arbitrary text lines and rate values lose the currency and zone column. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Route documents by actual format and flag low-confidence extraction rather than silently indexing blank text. This action primarily relates to Format-specific document extraction and OCR needs: Extract text with a parser suited to source format; image-only scans require OCR rather than HTML or plain-text parsing. However, the described case — An energy tariff PDF is chunked by arbitrary text lines and rate values lose the currency and zone column. — is testing Table-aware chunking and metadata-rich retrieval. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect adversarial prompt examples, tool permissions and whether refusals are enforced outside the model. This action primarily relates to Unsafe model output and jailbreak response diagnosis: User-provided instructions may attempt to bypass agent policies or cause data leakage; classification and tool boundaries need separate tests. However, the described case — A RAG assistant passes an internal evaluation set but receives complaints about new policy terms after launch. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect trace context passages, token distribution, coverage of necessary conditions and retrieval rank. This action primarily relates to Chunk quality evaluation and boundary splitting: Retrieval failure can arise from overly small chunks that omit needed context or overly large chunks that dilute relevance. However, the described case — A RAG assistant passes an internal evaluation set but receives complaints about new policy terms after launch. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect user conversation state, current intent, entity resolution and selected tool trace. This action primarily relates to Conversation context and intent routing: Multi-turn assistants need explicit conversational state while routing each new request to the correct tool or topic. However, the described case — A RAG assistant passes an internal evaluation set but receives complaints about new policy terms after launch. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.