Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest verify choice for this specific question. Test a missing-ground-truth record to verify the judge abstains or uses the appropriate alternate scorer. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — A medical expert flags unsafe triage advice, but engineering discards the feedback because an LLM judge rates it highly. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Simulate quality and latency threshold breaches and confirm responsible owners receive alerts. This action primarily relates to Production performance KPIs and incident thresholds: Monitoring should include answer correctness, retrieval coverage, safety, latency, drift and per-user harm indicators, not just availability. However, the described case — A medical expert flags unsafe triage advice, but engineering discards the feedback because an LLM judge rates it highly. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Query sampled live traces and demonstrate a complaint can be reproduced and scored on an offline test. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — A medical expert flags unsafe triage advice, but engineering discards the feedback because an LLM judge rates it highly. — is testing Human SME feedback rubric and reviewer calibration. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Compare fixed evaluation queries on baseline and candidate retrieval with confidence in differences. This action primarily relates to Retrieval recall precision and ranking diagnostics: Measure hit recall, precision, MRR or nDCG on labeled query-to-relevant-chunk pairs before tuning retrieval. However, the described case — A crawler fetches HTML product manuals but outputs menu text instead of article content. — is testing Format-specific document extraction and OCR needs. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Retrieve rare table values and confirm the matching row, units and header context are intact. This action primarily relates to Table-aware chunking and metadata-rich retrieval: Tabular rows need headers, units and column semantics preserved when transformed into retrieval documents. However, the described case — A crawler fetches HTML product manuals but outputs menu text instead of article content. — is testing Format-specific document extraction and OCR needs. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Verify that target retrieval returns complete condition-and-exception evidence for representative questions. This action primarily relates to Semantic boundaries versus fixed-window chunks: Headings, paragraphs and tables can require structure-aware splitting rather than arbitrary token windows. However, the described case — A crawler fetches HTML product manuals but outputs menu text instead of article content. — is testing Format-specific document extraction and OCR needs. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Build a relevance-labeled query set and measure recall@k and rank quality on actual returned chunks. This action primarily relates to Retrieval recall precision and ranking diagnostics: Measure hit recall, precision, MRR or nDCG on labeled query-to-relevant-chunk pairs before tuning retrieval. However, the described case — A compliance RAG app splits a numbered contract clause so exclusions are separated from the obligation. — is testing Semantic boundaries versus fixed-window chunks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Include part number and model metadata in each searchable row and filter by approved product line. This action primarily relates to Table-aware chunking and metadata-rich retrieval: Tabular rows need headers, units and column semantics preserved when transformed into retrieval documents. However, the described case — A compliance RAG app splits a numbered contract clause so exclusions are separated from the obligation. — is testing Semantic boundaries versus fixed-window chunks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Use an HTML-aware parser and targeted DOM content extraction rather than image OCR. This action primarily relates to Format-specific document extraction and OCR needs: Extract text with a parser suited to source format; image-only scans require OCR rather than HTML or plain-text parsing. However, the described case — A compliance RAG app splits a numbered contract clause so exclusions are separated from the obligation. — is testing Semantic boundaries versus fixed-window chunks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Inject the hostile PDF and verify no secret leaves the backend and no unauthorized tool is invoked. This action primarily relates to Input guardrails against prompt-injected source documents: A document that says 'ignore your rules' is untrusted data; a robust agent must enforce instruction and tool boundaries independently. However, the described case — A support bot incorporates competitor FAQs with usage terms that forbid automated redistribution. — is testing Training and retrieval corpus license rights. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test harmful queries and verify contaminated passages are blocked without discarding legitimate safety instructions. This action primarily relates to Toxic, inaccurate and adversarial corpus cleaning: Unreliable source text can teach unsafe behavior through retrieval; quarantine and source review matter before embeddings. However, the described case — A support bot incorporates competitor FAQs with usage terms that forbid automated redistribution. — is testing Training and retrieval corpus license rights. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Query for known synthetic sensitive strings and confirm no original identifiers appear in chunks or answers. This action primarily relates to Sensitive-field masking before embedding: Masking or tokenization can remove unnecessary PII from chunks before embedding and external model calls while retaining useful meaning. However, the described case — A support bot incorporates competitor FAQs with usage terms that forbid automated redistribution. — is testing Training and retrieval corpus license rights. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Strip template boilerplate, repeated navigation and low-information text before embedding. This action primarily relates to Boilerplate and navigation filtering in source corpora: Duplicated headers, disclaimers and navigation menus contaminate embeddings and cause irrelevant hits. However, the described case — A regulatory standard contains nested headings, and isolated subclauses lose defined terms from the section header. — is testing Parent-child retrieval and hierarchical structure. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Score retrieved documents separately from generation using graded relevance and ranked-hit metrics. This action primarily relates to Retrieval recall precision and ranking diagnostics: Measure hit recall, precision, MRR or nDCG on labeled query-to-relevant-chunk pairs before tuning retrieval. However, the described case — A regulatory standard contains nested headings, and isolated subclauses lose defined terms from the section header. — is testing Parent-child retrieval and hierarchical structure. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Apply tenant identity metadata filters before or during retrieval and use hybrid ranking within that scope. This action primarily relates to Hybrid sparse and dense retrieval with filters: Hybrid search combines lexical exact-match and semantic similarity; metadata filtering narrows authorized relevant content. However, the described case — A regulatory standard contains nested headings, and isolated subclauses lose defined terms from the section header. — is testing Parent-child retrieval and hierarchical structure. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Require approval/rollback for prompt and index changes that alter user-facing model behavior. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — A new model signature expects messages array but the existing app sends only a prompt string. — is testing Deployed model dependency drift and environment compatibility. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Register the MLflow model as a named Unity Catalog model version and grant only approved use/manage rights. This action primarily relates to Unity Catalog registered model version and privileges: MLflow-logged models can be registered in Unity Catalog, with governed names, permissions and traceable version lineage. However, the described case — A new model signature expects messages array but the existing app sends only a prompt string. — is testing Deployed model dependency drift and environment compatibility. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect p95 latency, queue depth, rate limits, cost and per-version error traces. This action primarily relates to Endpoint scale concurrency and operational rollout: Serving latency and reliability depend on provisioned capacity, request concurrency, quotas and controlled rollout. However, the described case — A product catalog has 85 million vectors with stringent p95 search latency and frequent incremental updates. — is testing Standard versus storage-optimized Vector Search index. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect storage key design, user isolation, TTL/retention and retrieval authorization. This action primarily relates to Persistent conversation memory with authorized datastore: Transient prompt context differs from persistent user state; durable memory needs explicit storage, identity scope and retention. However, the described case — A product catalog has 85 million vectors with stringent p95 search latency and frequent incremental updates. — is testing Standard versus storage-optimized Vector Search index. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect logged model artifacts, environment requirements, signature and source index identifiers. This action primarily relates to RAG production bundle metadata and dependencies: A deployable RAG application needs tracked model code, retriever config, embedding model, dependency versions, signature and example input. However, the described case — A product catalog has 85 million vectors with stringent p95 search latency and frequent incremental updates. — is testing Standard versus storage-optimized Vector Search index. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Deploy synchronized index definitions and validate source freshness during production cutover. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — A company must deploy a custom fine-tuned checkpoint with nonstandard preprocessing not offered by managed endpoints. — is testing Foundation Model API versus custom serving choices. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not equate registering a model with approving its business safety or access to all governed tables. This action primarily relates to Unity Catalog registered model version and privileges: MLflow-logged models can be registered in Unity Catalog, with governed names, permissions and traceable version lineage. However, the described case — A company must deploy a custom fine-tuned checkpoint with nonstandard preprocessing not offered by managed endpoints. — is testing Foundation Model API versus custom serving choices. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Simulate a traffic burst and confirm rate limiting protects other clients while usage reporting remains accurate. This action primarily relates to AI Gateway rate limits, usage tables and inference telemetry: AI Gateway offers controlled access, supported usage tracking and rate limiting to protect shared endpoints and measure consumption. However, the described case — An internal assistant sends 40,000 irrelevant tokens to the LLM for every short FAQ question. — is testing Databricks usage and token cost controls. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test a missing-ground-truth record to verify the judge abstains or uses the appropriate alternate scorer. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — An internal assistant sends 40,000 irrelevant tokens to the LLM for every short FAQ question. — is testing Databricks usage and token cost controls. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Re-evaluate previously disputed cases and confirm inter-rater agreement improves after calibration. This action primarily relates to Human SME feedback rubric and reviewer calibration: Domain experts should use a shared rubric, calibrate disagreements and maintain adjudicated examples to improve model evaluation. However, the described case — An internal assistant sends 40,000 irrelevant tokens to the LLM for every short FAQ question. — is testing Databricks usage and token cost controls. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Pass both the user question and selected passages into the prompt template before invoking the model. This action primarily relates to Retriever-generator chain component boundaries: A basic RAG chain separates user input, query transformation, retrieval, prompt assembly, model generation and output validation. However, the described case — A benefits intake app requires a JSON object with claim_id, decision and reason but the assistant returns Markdown paragraphs. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep each specialist's permissions bounded and audit handoffs rather than granting a coordinator unrestricted tools. This action primarily relates to Agent Bricks Information Extraction versus Multiagent Supervisor: Information Extraction maps source documents to structured fields; Multiagent Supervisor coordinates specialized capabilities for compound workflows. However, the described case — A benefits intake app requires a JSON object with claim_id, decision and reason but the assistant returns Markdown paragraphs. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect warehouse type, runtime compatibility, endpoint status, SQL arguments and batch size. This action primarily relates to SQL ai_query for governed batch inference: `ai_query()` invokes an available serving endpoint from SQL; suitability depends on supported runtimes, endpoint and workload. However, the described case — A company must deploy a custom fine-tuned checkpoint with nonstandard preprocessing not offered by managed endpoints. — is testing Foundation Model API versus custom serving choices. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect predict entrypoint, model artifacts, dependency environment, inputs and output serialization. This action primarily relates to MLflow pyfunc packaging for pre- and post-processing: A custom MLflow pyfunc model wraps request parsing, retriever/model invocation and output formatting inside a reproducible predict interface. However, the described case — A company must deploy a custom fine-tuned checkpoint with nonstandard preprocessing not offered by managed endpoints. — is testing Foundation Model API versus custom serving choices. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect p95 latency, queue depth, rate limits, cost and per-version error traces. This action primarily relates to Endpoint scale concurrency and operational rollout: Serving latency and reliability depend on provisioned capacity, request concurrency, quotas and controlled rollout. However, the described case — A company must deploy a custom fine-tuned checkpoint with nonstandard preprocessing not offered by managed endpoints. — is testing Foundation Model API versus custom serving choices. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.