Databricks Certified Generative AI Engineer Associate Bundle

Not the strongest verify choice for this specific question. Test a missing-ground-truth record to verify the judge abstains or uses the appropriate alternate scorer. This action primarily relates to Ground-truth requirement for correctness versus reference-free judges: A correctness comparison often needs an expected answer; other judges such as format or safety can operate without a reference. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Query sampled live traces and demonstrate a complaint can be reproduced and scored on an offline test. This action primarily relates to Inference tables and Agent Monitoring feedback loop: Inference tables and agent monitoring capture production request/response signals and traces for quality investigations. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Re-evaluate previously disputed cases and confirm inter-rater agreement improves after calibration. This action primarily relates to Human SME feedback rubric and reviewer calibration: Domain experts should use a shared rubric, calibrate disagreements and maintain adjudicated examples to improve model evaluation. However, the described case — An operations assistant benchmark favors a huge model on fluency, but a smaller model has higher citation correctness at lower cost. — is testing Model selection from quantitative task metrics. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Keep long-lived credentials out of clients and distinguish shared app authorization from user impersonation. This action primarily relates to Databricks Apps user-facing agent interface and OAuth identities: Databricks Apps supports authenticated interactive frontends; app credentials and user credentials have different scopes. However, the described case — New policies appear in Delta but a manually managed embedding index still returns retired versions. — is testing Databricks Vector Search index from governed Delta data. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Use `ai_query()` against a supported endpoint in a controlled batch-inference query with result checks. This action primarily relates to SQL ai_query for governed batch inference: `ai_query()` invokes an available serving endpoint from SQL; suitability depends on supported runtimes, endpoint and workload. However, the described case — New policies appear in Delta but a manually managed embedding index still returns retired versions. — is testing Databricks Vector Search index from governed Delta data. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Reproduce a slow/error case and confirm each relevant stage is observable without sensitive tokens. This action primarily relates to MLflow tracing in an Agent Framework workflow: MLflow tracing records agent steps, tool calls and latency; integration must retain child spans and inputs without leaking secrets. However, the described case — A legal contract review agent needs nuanced long-context reasoning but can tolerate higher response time. — is testing Model selection by quality latency and budget. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Document legal clearance and confirm the deployed artifact matches the approved license/version. This action primarily relates to Open model license versus managed endpoint obligations: Model availability on a hub does not itself confer commercial training, redistribution or hosting rights. However, the described case — A legal contract review agent needs nuanced long-context reasoning but can tolerate higher response time. — is testing Model selection by quality latency and budget. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Reproduce representative task evaluation in the intended language and document limitations. This action primarily relates to Model cards and task-specific limitations: A model card records intended uses, supported language/context, safety limitations and evaluation scope relevant to selection. However, the described case — A legal contract review agent needs nuanced long-context reasoning but can tolerate higher response time. — is testing Model selection by quality latency and budget. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect extraction fields, masking policy, downstream embedded text and sensitive-data exposure paths. This action primarily relates to Sensitive-field masking before embedding: Masking or tokenization can remove unnecessary PII from chunks before embedding and external model calls while retaining useful meaning. However, the described case — A partner PDF includes text telling a RAG assistant to reveal its secret connection string in the answer. — is testing Input guardrails against prompt-injected source documents. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect filter decisions, action permissions, identity propagation and resulting audit events. This action primarily relates to Prompt safety filtering and tool allowlists: Content filters and action allowlists are complementary; a moderation score is not a substitute for authorization of external effects. However, the described case — A partner PDF includes text telling a RAG assistant to reveal its secret connection string in the answer. — is testing Input guardrails against prompt-injected source documents. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect which fields are necessary, masking false positives and any leaked sensitive fields. This action primarily relates to Contextual masking versus destructive data removal: Masking must preserve information needed for the task, such as non-sensitive dates or amounts, and cannot replace access control. However, the described case — A partner PDF includes text telling a RAG assistant to reveal its secret connection string in the answer. — is testing Input guardrails against prompt-injected source documents. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Validate tool discovery and a permitted request for both services, including denied action testing. This action primarily relates to Managed versus external versus custom MCP server integration: MCP standardizes tool discovery; managed servers reduce hosting, external services need credentials and custom servers wrap bespoke actions. However, the described case — A research archive contains hundreds of millions of mostly static embeddings and storage cost is the main constraint. — is testing Standard versus storage-optimized Vector Search index. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test model invocation under realistic throughput and verify output schema, quota and governance constraints. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — A research archive contains hundreds of millions of mostly static embeddings and storage cost is the main constraint. — is testing Standard versus storage-optimized Vector Search index. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Test allowed and denied principals against both endpoint invocation and protected retrieval data. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — A research archive contains hundreds of millions of mostly static embeddings and storage cost is the main constraint. — is testing Standard versus storage-optimized Vector Search index. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Revisit cost/latency budgets whenever context window or retrieval size changes significantly. This action primarily relates to Latency, cost and retrieval targets as design requirements: A valid pipeline design has measurable quality, latency, token and throughput constraints before model selection. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Join authorized structured values and retrieved policy context through a controlled composition stage. This action primarily relates to Parallel retrieval and deterministic chain postprocessing: Combining structured lookup and semantic retrieval requires explicit routing, result joining, and downstream schema checks. However, the described case — An extraction model treats a quoted customer complaint as a command to ignore the required response shape. — is testing Instruction hierarchy and bounded prompt examples. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect data schema, primary identifiers, change detection, UC permissions and ingestion checkpoint behavior. This action primarily relates to Delta and Unity Catalog pipeline for chunk tables: A governed retrieval corpus stores chunk text, IDs, source lineage and metadata in managed Delta/Unity Catalog tables before indexing. However, the described case — A RAG benchmark reports fluent final answers but no evidence of whether the right passage was retrieved. — is testing Retrieval recall precision and ranking diagnostics. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect candidate-set recall, relevance scores, top-k cutoffs and added ranking latency. This action primarily relates to Cross-encoder reranking after candidate retrieval: A reranker scores query-document relevance over a candidate set; it improves ordering but does not add missing documents. However, the described case — A RAG benchmark reports fluent final answers but no evidence of whether the right passage was retrieved. — is testing Retrieval recall precision and ranking diagnostics. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect token distribution, overlap proportion, expected retrieval granularity and vector count. This action primarily relates to Chunk size and overlap tradeoffs: Chunk size determines the number of vectors and coherence; overlap reduces boundary loss but increases index size and cost. However, the described case — A RAG benchmark reports fluent final answers but no evidence of whether the right passage was retrieved. — is testing Retrieval recall precision and ranking diagnostics. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Isolate source text in retrieval context and use backend authorization for ticket actions. This action primarily relates to Input guardrails against prompt-injected source documents: A document that says 'ignore your rules' is untrusted data; a robust agent must enforce instruction and tool boundaries independently. However, the described case — An imported policy library contains a deliberately altered obsolete security procedure mixed into current instructions. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Review license and reuse conditions and use authorized sources or link without copying protected text. This action primarily relates to Training and retrieval corpus license rights: Public accessibility does not automatically authorize copying, embedding or using content in commercial RAG services. However, the described case — An imported policy library contains a deliberately altered obsolete security procedure mixed into current instructions. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest implement choice for this specific question. Mask account identifiers selectively while retaining required monetary values and verifying extraction accuracy. This action primarily relates to Contextual masking versus destructive data removal: Masking must preserve information needed for the task, such as non-sensitive dates or amounts, and cannot replace access control. However, the described case — An imported policy library contains a deliberately altered obsolete security procedure mixed into current instructions. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run adversarial quoted-text tests and compare output structure on representative and edge inputs. This action primarily relates to Instruction hierarchy and bounded prompt examples: Few-shot examples and explicit boundaries can clarify expected outputs, but embedded source documents should not become new system instructions. However, the described case — A live call-center assistant must respond under three seconds and keep per-call model cost bounded. — is testing Latency, cost and retrieval targets as design requirements. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Run cases with complete and missing required input fields and verify schema, access scope and usefulness. This action primarily relates to Translate use-case requirements into pipeline inputs and outputs: Business acceptance criteria must map to concrete source inputs, generated fields, correctness measures and permission constraints. However, the described case — A live call-center assistant must respond under three seconds and keep per-call model cost bounded. — is testing Latency, cost and retrieval targets as design requirements. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest verify choice for this specific question. Withhold one source during testing and confirm the chain degrades safely rather than inventing a missing record. This action primarily relates to Parallel retrieval and deterministic chain postprocessing: Combining structured lookup and semantic retrieval requires explicit routing, result joining, and downstream schema checks. However, the described case — A live call-center assistant must respond under three seconds and keep per-call model cost bounded. — is testing Latency, cost and retrieval targets as design requirements. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not treat a model hub popularity score as certification of deployment fitness. This action primarily relates to Model cards and task-specific limitations: A model card records intended uses, supported language/context, safety limitations and evaluation scope relevant to selection. However, the described case — A code assistant looks fast in demos but has low pass rate on the organization's test suite. — is testing Model evaluation with paired benchmarks. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Do not call a one-time benchmark a complete monitoring program for a deployed agent. This action primarily relates to Offline evaluation versus ongoing production monitoring: Offline test datasets catch regressions pre-release; production monitoring detects drift, real-user failures and evolving prompts. However, the described case — A code assistant looks fast in demos but has low pass rate on the organization's test suite. — is testing Model evaluation with paired benchmarks. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest safeguard choice for this specific question. Version embeddings with source and model identifiers to prevent silently mixing incompatible representations. This action primarily relates to Embedding model swap and reindex compatibility: Changed embedding dimensions or model semantics typically require re-embedding and index consistency before comparing search quality. However, the described case — A code assistant looks fast in demos but has low pass rate on the organization's test suite. — is testing Model evaluation with paired benchmarks. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect which indicators are measured before deployment versus on live traffic and feedback. This action primarily relates to Offline evaluation versus ongoing production monitoring: Offline test datasets catch regressions pre-release; production monitoring detects drift, real-user failures and evolving prompts. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect vector count, dimension, encoder throughput, per-query latency and measured quality curves. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Not the strongest diagnose choice for this specific question. Inspect experiment runs, source versions, parameter drift and metric definitions. This action primarily relates to Experiment tracking and version reproducibility: MLflow experiments should retain prompt, model, dataset, parameters and evaluation outputs to support a defensible comparison. However, the described case — A proposed LLM wins on general benchmark score but fails grounded FAQ questions more frequently than incumbent. — is testing Model evaluation with paired benchmarks. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.