Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest verify choice for this specific question. Compare precision at k before and after cleaning while checking no required policy sentences were removed. This action primarily relates to Boilerplate and navigation filtering in source corpora: Duplicated headers, disclaimers and navigation menus contaminate embeddings and cause irrelevant hits. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Verify that target retrieval returns complete condition-and-exception evidence for representative questions. This action primarily relates to Semantic boundaries versus fixed-window chunks: Headings, paragraphs and tables can require structure-aware splitting rather than arbitrary token windows. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Compare fixed evaluation queries on baseline and candidate retrieval with confidence in differences. This action primarily relates to Retrieval recall precision and ranking diagnostics: Measure hit recall, precision, MRR or nDCG on labeled query-to-relevant-chunk pairs before tuning retrieval. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test session resume for the same user and deny cross-user recall of private ticket details. This action primarily relates to Persistent conversation memory with authorized datastore: Transient prompt context differs from persistent user state; durable memory needs explicit storage, identity scope and retention. However, the described case — A public browser sends a long-lived PAT directly to a model serving endpoint for each chat request. — is testing Model Serving endpoint permissions and caller identity. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Load an explicitly registered version and confirm its dependencies and outputs match the approved evaluation run. This action primarily relates to Unity Catalog registered model version and privileges: MLflow-logged models can be registered in Unity Catalog, with governed names, permissions and traceable version lineage. However, the described case — A public browser sends a long-lived PAT directly to a model serving endpoint for each chat request. — is testing Model Serving endpoint permissions and caller identity. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Load the registered bundle in a clean environment and validate retrieval-to-generation behavior end to end. This action primarily relates to RAG production bundle metadata and dependencies: A deployable RAG application needs tracked model code, retriever config, embedding model, dependency versions, signature and example input. However, the described case — A public browser sends a long-lived PAT directly to a model serving endpoint for each chat request. — is testing Model Serving endpoint permissions and caller identity. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep long-lived credentials out of clients and distinguish shared app authorization from user impersonation. This action primarily relates to Databricks Apps user-facing agent interface and OAuth identities: Databricks Apps supports authenticated interactive frontends; app credentials and user credentials have different scopes. However, the described case — A public browser sends a long-lived PAT directly to a model serving endpoint for each chat request. — is testing Model Serving endpoint permissions and caller identity. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not assume a managed Foundation Model endpoint grants unrestricted training rights or satisfies data residency automatically. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — A public browser sends a long-lived PAT directly to a model serving endpoint for each chat request. — is testing Model Serving endpoint permissions and caller identity. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Set load thresholds and versioned fallback rather than assuming a single demo proves production scalability. This action primarily relates to Endpoint scale concurrency and operational rollout: Serving latency and reliability depend on provisioned capacity, request concurrency, quotas and controlled rollout. However, the described case — A public browser sends a long-lived PAT directly to a model serving endpoint for each chat request. — is testing Model Serving endpoint permissions and caller identity. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Sample indexed chunks against a rights manifest and confirm prohibited sources are excluded. This action primarily relates to Training and retrieval corpus license rights: Public accessibility does not automatically authorize copying, embedding or using content in commercial RAG services. However, the described case — A customer-support RAG source includes cardholder names and full card numbers even though questions concern card dispute policy. — is testing Sensitive-field masking before embedding. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test harmful queries and verify contaminated passages are blocked without discarding legitimate safety instructions. This action primarily relates to Toxic, inaccurate and adversarial corpus cleaning: Unreliable source text can teach unsafe behavior through retrieval; quarantine and source review matter before embeddings. However, the described case — A customer-support RAG source includes cardholder names and full card numbers even though questions concern card dispute policy. — is testing Sensitive-field masking before embedding. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Inject the hostile PDF and verify no secret leaves the backend and no unauthorized tool is invoked. This action primarily relates to Input guardrails against prompt-injected source documents: A document that says 'ignore your rules' is untrusted data; a robust agent must enforce instruction and tool boundaries independently. However, the described case — A customer-support RAG source includes cardholder names and full card numbers even though questions concern card dispute policy. — is testing Sensitive-field masking before embedding. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Parse and chunk documents, write keyed chunk rows with provenance to a governed Delta table, then index them. This action primarily relates to Delta and Unity Catalog pipeline for chunk tables: A governed retrieval corpus stores chunk text, IDs, source lineage and metadata in managed Delta/Unity Catalog tables before indexing. However, the described case — An internal parts bot misses exact serial number ZX-410 because embeddings match only conceptual part descriptions. — is testing Hybrid sparse and dense retrieval with filters. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Choose coherent chunks large enough to keep cause and remedy together without exceeding embedding limits. This action primarily relates to Chunk size and overlap tradeoffs: Chunk size determines the number of vectors and coherence; overlap reduces boundary loss but increases index size and cost. However, the described case — An internal parts bot misses exact serial number ZX-410 because embeddings match only conceptual part descriptions. — is testing Hybrid sparse and dense retrieval with filters. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Split by clause and heading boundaries with a token-aware fallback to preserve the rule and exception together. This action primarily relates to Semantic boundaries versus fixed-window chunks: Headings, paragraphs and tables can require structure-aware splitting rather than arbitrary token windows. However, the described case — An internal parts bot misses exact serial number ZX-410 because embeddings match only conceptual part descriptions. — is testing Hybrid sparse and dense retrieval with filters. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Create a compatible AI Search/Vector Search index over the approved source and configured embedding columns. This action primarily relates to Databricks Vector Search index from governed Delta data: A vector index serves embeddings of chunked records maintained in a Unity Catalog governed Delta source table. However, the described case — A prototype RAG agent works on one notebook cluster but fails because its deployment package omits the vector search client dependency. — is testing RAG production bundle metadata and dependencies. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Deploy synchronized index definitions and validate source freshness during production cutover. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — A prototype RAG agent works on one notebook cluster but fails because its deployment package omits the vector search client dependency. — is testing RAG production bundle metadata and dependencies. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Query the relevant index with a compatible query embedding, select authorized matches and pass them to the prompt. This action primarily relates to Vector Search retrieval workflow and query integration: Semantic retrieval uses an embedding model to query a compatible index, returns ranked passages and metadata for grounded generation. However, the described case — A prototype RAG agent works on one notebook cluster but fails because its deployment package omits the vector search client dependency. — is testing RAG production bundle metadata and dependencies. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect task-appropriate quality metrics, inference cost, p95 latency and error segments. This action primarily relates to Model selection from quantitative task metrics: Select candidate LLMs by measured task success, latency, safety and token cost, not parameter count or leaderboard rank. However, the described case — An agent endpoint gets accuracy complaints but serving logs capture only HTTP status, hiding response content and retrieval IDs. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect token volumes, QPS, endpoint pricing, cache hit rate and user/workload attribution. This action primarily relates to Databricks usage and token cost controls: Reduce LLM spend through token budgets, caching, smaller appropriate models and AI Gateway quotas while preserving acceptance quality. However, the described case — An agent endpoint gets accuracy complaints but serving logs capture only HTTP status, hiding response content and retrieval IDs. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect each judge's required inputs, reference fields, rubric and expected failure behavior. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — An agent endpoint gets accuracy complaints but serving logs capture only HTTP status, hiding response content and retrieval IDs. — is testing Inference logging and production trace privacy. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run paired representative requests and record quality versus p95 latency and cost tradeoffs. This action primarily relates to Model selection by quality latency and budget: Select the smallest suitable model that meets a task's accuracy and latency constraints under realistic workload, not marketing rank. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run side-by-side evaluations for baseline and candidate prompts with format, groundedness and latency metrics. This action primarily relates to Prompt changes evaluated against baseline: A prompt improvement should be measured against a fixed baseline set, not accepted because a few outputs sound better. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run reproducible pairwise experiments with confidence and segment-level result analysis. This action primarily relates to Model evaluation with paired benchmarks: Accuracy, faithfulness, relevance and refusal behavior require task-specific scoring, and a model switch needs common test conditions. However, the described case — A team wants to reuse the same retrieval stage with two different LLMs while benchmarking quality. — is testing LangChain runnable composition and invocation. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. A/B test chunk segmentation on frozen questions and verify evidence sufficiency and prompt-budget compliance. This action primarily relates to Chunk quality evaluation and boundary splitting: Retrieval failure can arise from overly small chunks that omit needed context or overly large chunks that dilute relevance. However, the described case — A team wants to deploy a new prompt based solely on improving live thumbs-up counts. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Audit answer-supportedness using labeled trace examples and a human review of fabricated citations. This action primarily relates to Hallucinated facts and unsupported citations: Fluent responses can invent facts and citations; faithfulness must be checked against supplied evidence, not writing style. However, the described case — A team wants to deploy a new prompt based solely on improving live thumbs-up counts. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Verify matched query and document vectors use the approved version and retrieval improves on held-out examples. This action primarily relates to Embedding model swap and reindex compatibility: Changed embedding dimensions or model semantics typically require re-embedding and index consistency before comparing search quality. However, the described case — A team wants to deploy a new prompt based solely on improving live thumbs-up counts. — is testing Offline evaluation versus ongoing production monitoring. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Evaluate a summarization model against faithful concise summaries with source references. This action primarily relates to Task choice: classification versus generative summarization: Classification predicts a category whereas summarization compresses meaning; the desired output determines the appropriate task. However, the described case — A shipping chatbot returns a list with inconsistent property names, breaking the downstream parcel status renderer. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Translate SLA into retrieval latency, time-to-first-token, maximum context and quality floor for candidate designs. This action primarily relates to Latency, cost and retrieval targets as design requirements: A valid pipeline design has measurable quality, latency, token and throughput constraints before model selection. However, the described case — A shipping chatbot returns a list with inconsistent property names, breaking the downstream parcel status renderer. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Order identity verification, purchase lookup, policy check, user confirmation and transaction tool invocation. This action primarily relates to Tool ordering and error isolation in multi-stage reasoning: An agent should retrieve authoritative context before taking consequential actions and validate tool outcomes at each stage. However, the described case — A shipping chatbot returns a list with inconsistent property names, breaking the downstream parcel status renderer. — is testing Structured JSON schema prompt and format compliance. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect deployment manifests, tests, aliases, index versions and rollback dependency matrix. This action primarily relates to CI/CD pipelines for retrieval and agent components: Production promotions should test source index, prompt, model packaging and tool behavior independently before release. However, the described case — A production team passes around unversioned notebook checkpoints and cannot reproduce the active agent package. — is testing Unity Catalog registered model version and privileges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect serving endpoint permissions, token path, user/app identity and downstream Unity Catalog access. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — A production team passes around unversioned notebook checkpoints and cannot reproduce the active agent package. — is testing Unity Catalog registered model version and privileges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect logged model artifacts, environment requirements, signature and source index identifiers. This action primarily relates to RAG production bundle metadata and dependencies: A deployable RAG application needs tracked model code, retriever config, embedding model, dependency versions, signature and example input. However, the described case — A production team passes around unversioned notebook checkpoints and cannot reproduce the active agent package. — is testing Unity Catalog registered model version and privileges. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.