mcAfee Secure Website
23

Databricks Certified Generative AI Engineer Associate Bundle

Exam Code: Certified Generative AI Engineer Associate

Exam Name Certified Generative AI Engineer Associate

Certification Provider: Databricks

Corresponding Certification: Databricks Certified Generative AI Engineer Associate

Certified Generative AI Engineer Associate Training Materials $19.99

Reliable & Actual Study Materials for Certified Generative AI Engineer Associate Exam Success

The Latest Certified Generative AI Engineer Associate Exam Questions as Experienced in the Actual Test!

  • 24
    Questions & Answers

    Certified Generative AI Engineer Associate Questions & Answers

    123 Questions & Answers

    Includes questions types found on actual exam such as drag and drop, simulation, type in, and fill in the blank.

  • exam =30
    Study Guide

    Certified Generative AI Engineer Associate Study Guide

    230 PDF Pages

    Study Guide developed by industry experts who have written exams in the past. They are technology-specific IT certification researchers with at least a decade of experience at Fortune 500 companies.

exam =32

Certified Generative AI Engineer Associate Practice Tests

Guarantee

Satisfaction Guaranteed

Pass4sure has a remarkable Databricks Candidate Success record. We're confident of our products and provide no hassle product exchange. That's how confident we are!

99.3% Pass Rate
Total Cost: $154.98
Bundle Price: $134.99

Purchase Individually

  • exam =34
    Questions & Answers

    Questions & Answers

    123 Questions

    $124.99
    exam =35
  • exam =36
    Study Guide

    Study Guide

    230 PDF Pages

    $29.99

Databricks Certified Generative AI Engineer Associate Practice Test 52

Question 510 (Select TWO)

Select TWO: the immediate change and one ongoing control for this Databricks GenAI case. Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits.

  1. Rerank a sufficiently broad candidate pool then pass a smaller high-relevance set to the LLM.
  2. Use an OCR-capable extraction step, evaluate recognition quality and retain source page provenance.
  3. Retain source offsets and section IDs so citations remain traceable after rechunking.
  4. Do not attribute an absent document to reranking failure if the first-stage search never returned it.

Explanation

Correct Answers

A:

Correct (immediate corrective action). Rerank a sufficiently broad candidate pool then pass a smaller high-relevance set to the LLM. In this scenario, Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. The governing mechanism is: A reranker scores query-document relevance over a candidate set; it improves ordering but does not add missing documents. One selected option addresses the immediate workflow, while the other guards the same failure mode after deployment. Both satisfy the TWO-part requirement.

D:

Correct (continuing risk control). Do not attribute an absent document to reranking failure if the first-stage search never returned it. In this scenario, Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. The governing mechanism is: A reranker scores query-document relevance over a candidate set; it improves ordering but does not add missing documents. One selected option addresses the immediate workflow, while the other guards the same failure mode after deployment. Both satisfy the TWO-part requirement.

Incorrect Answers

B:

Not the strongest safeguard choice for this specific question. Use an OCR-capable extraction step, evaluate recognition quality and retain source page provenance. This action primarily relates to Format-specific document extraction and OCR needs: Extract text with a parser suited to source format; image-only scans require OCR rather than HTML or plain-text parsing. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest safeguard choice for this specific question. Retain source offsets and section IDs so citations remain traceable after rechunking. This action primarily relates to Semantic boundaries versus fixed-window chunks: Headings, paragraphs and tables can require structure-aware splitting rather than arbitrary token windows. However, the described case — Hybrid search retrieves the right troubleshooting note at rank 18 yet generation uses only the first five hits. — is testing Cross-encoder reranking after candidate retrieval. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 511

Which standing requirement should operations preserve? An agent promotion script overwrites the only model file without retaining the prior validated artifact.

  1. Rotate credentials and separately verify caller identity and underlying data permission scopes.
  2. Do not equate registering a model with approving its business safety or access to all governed tables.
  3. Do not assume a managed Foundation Model endpoint grants unrestricted training rights or satisfies data residency automatically.
  4. Never rely on interactive notebook state or unpinned runtime packages when promoting a served model.

Explanation

Correct Answer

B:

Correct. MLflow-logged models can be registered in Unity Catalog, with governed names, permissions and traceable version lineage. In the given situation — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — the targeted safeguard action is: Do not equate registering a model with approving its business safety or access to all governed tables. It preserves an enduring engineering or governance boundary after the current issue is fixed.

Incorrect Answers

A:

Not the strongest safeguard choice for this specific question. Rotate credentials and separately verify caller identity and underlying data permission scopes. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest safeguard choice for this specific question. Do not assume a managed Foundation Model endpoint grants unrestricted training rights or satisfies data residency automatically. This action primarily relates to Foundation Model API versus custom serving choices: Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest safeguard choice for this specific question. Never rely on interactive notebook state or unpinned runtime packages when promoting a served model. This action primarily relates to MLflow pyfunc packaging for pre- and post-processing: A custom MLflow pyfunc model wraps request parsing, retriever/model invocation and output formatting inside a reproducible predict interface. However, the described case — An agent promotion script overwrites the only model file without retaining the prior validated artifact. — is testing Unity Catalog registered model version and privileges. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 512

Which continuing engineering control best reduces recurrence of this issue? An agent must search an external research API requiring a secret, while internal company data uses managed connectors.

  1. Never place MCP API keys in prompts or browser code, and review third-party tool trust boundaries.
  2. Rotate credentials and separately verify caller identity and underlying data permission scopes.
  3. Treat index freshness as a monitored dependency of source-document correctness.
  4. Keep long-lived credentials out of clients and distinguish shared app authorization from user impersonation.

Explanation

Correct Answer

A:

Correct. MCP standardizes tool discovery; managed servers reduce hosting, external services need credentials and custom servers wrap bespoke actions. In the given situation — An agent must search an external research API requiring a secret, while internal company data uses managed connectors. — the targeted safeguard action is: Never place MCP API keys in prompts or browser code, and review third-party tool trust boundaries. It preserves an enduring engineering or governance boundary after the current issue is fixed.

Incorrect Answers

B:

Not the strongest safeguard choice for this specific question. Rotate credentials and separately verify caller identity and underlying data permission scopes. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — An agent must search an external research API requiring a secret, while internal company data uses managed connectors. — is testing Managed versus external versus custom MCP server integration. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest safeguard choice for this specific question. Treat index freshness as a monitored dependency of source-document correctness. This action primarily relates to Databricks Vector Search index from governed Delta data: A vector index serves embeddings of chunked records maintained in a Unity Catalog governed Delta source table. However, the described case — An agent must search an external research API requiring a secret, while internal company data uses managed connectors. — is testing Managed versus external versus custom MCP server integration. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest safeguard choice for this specific question. Keep long-lived credentials out of clients and distinguish shared app authorization from user impersonation. This action primarily relates to Databricks Apps user-facing agent interface and OAuth identities: Databricks Apps supports authenticated interactive frontends; app credentials and user credentials have different scopes. However, the described case — An agent must search an external research API requiring a secret, while internal company data uses managed connectors. — is testing Managed versus external versus custom MCP server integration. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 513

During a root-cause review, the team reports: A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. What evidence best identifies the reason?

  1. Inspect package lockfile, model signature, serving logs and API contract differences.
  2. Inspect required model type, deployment control, supported region, endpoint quotas and operational burden.
  3. Inspect serving endpoint permissions, token path, user/app identity and downstream Unity Catalog access.
  4. Inspect server type, endpoint URL, supported auth, tool declarations and available scopes.

Explanation

Correct Answer

B:

Correct. Databricks Foundation Model APIs offer managed access to supported models; custom serving is suitable for specialized models or control needs. In the given situation — A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. — the targeted diagnose action is: Inspect required model type, deployment control, supported region, endpoint quotas and operational burden. It inspects the stage or constraint most likely to explain the observed failure before changing the system.

Incorrect Answers

A:

Not the strongest diagnose choice for this specific question. Inspect package lockfile, model signature, serving logs and API contract differences. This action primarily relates to Deployed model dependency drift and environment compatibility: Models logged in experiments can fail in serving if Python dependencies or inference signatures differ between build and target. However, the described case — A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. — is testing Foundation Model API versus custom serving choices. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest diagnose choice for this specific question. Inspect serving endpoint permissions, token path, user/app identity and downstream Unity Catalog access. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. — is testing Foundation Model API versus custom serving choices. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest diagnose choice for this specific question. Inspect server type, endpoint URL, supported auth, tool declarations and available scopes. This action primarily relates to Managed versus external versus custom MCP server integration: MCP standardizes tool discovery; managed servers reduce hosting, external services need credentials and custom servers wrap bespoke actions. However, the described case — A proof-of-concept team wants a supported hosted LLM without managing GPU serving or packaging custom weights. — is testing Foundation Model API versus custom serving choices. It does not directly establish the diagnose outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 514

A Databricks engineer is correcting the following workflow: A public review feed contains slurs and adversarial instructions that surface in a brand-support assistant. Which change best addresses the documented requirement?

  1. Obtain appropriate rights or replace manuals with licensed public documentation before ingestion.
  2. Mask card numbers and nonessential identity fields before ingestion and keep approved case references separately.
  3. Filter toxic or instruction-like content and use reviewed trusted source subsets for customer-facing retrieval.
  4. Use server-side ownership checks and transaction approval on the refund tool in addition to input screening.

Explanation

Correct Answer

C:

Correct. Unreliable source text can teach unsafe behavior through retrieval; quarantine and source review matter before embeddings. In the given situation — A public review feed contains slurs and adversarial instructions that surface in a brand-support assistant. — the targeted implement action is: Filter toxic or instruction-like content and use reviewed trusted source subsets for customer-facing retrieval. This directly meets the stated workflow constraint instead of addressing an adjacent capability.

Incorrect Answers

A:

Not the strongest implement choice for this specific question. Obtain appropriate rights or replace manuals with licensed public documentation before ingestion. This action primarily relates to Training and retrieval corpus license rights: Public accessibility does not automatically authorize copying, embedding or using content in commercial RAG services. However, the described case — A public review feed contains slurs and adversarial instructions that surface in a brand-support assistant. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

B:

Not the strongest implement choice for this specific question. Mask card numbers and nonessential identity fields before ingestion and keep approved case references separately. This action primarily relates to Sensitive-field masking before embedding: Masking or tokenization can remove unnecessary PII from chunks before embedding and external model calls while retaining useful meaning. However, the described case — A public review feed contains slurs and adversarial instructions that surface in a brand-support assistant. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest implement choice for this specific question. Use server-side ownership checks and transaction approval on the refund tool in addition to input screening. This action primarily relates to Prompt safety filtering and tool allowlists: Content filters and action allowlists are complementary; a moderation score is not a substitute for authorization of external effects. However, the described case — A public review feed contains slurs and adversarial instructions that surface in a brand-support assistant. — is testing Toxic, inaccurate and adversarial corpus cleaning. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 515

Which continuing engineering control best reduces recurrence of this issue? A monthly reporting assistant processes thousands of independent documents overnight with low urgency.

  1. Keep authoritative structured values separate from explanatory unstructured content in the final answer.
  2. Revisit cost/latency budgets whenever context window or retrieval size changes significantly.
  3. Do not assume an instruction to 'respond in JSON' replaces actual schema validation and error handling.
  4. Treat absence of relevant retrieved evidence as an abstention condition, not a reason to invent missing facts.

Explanation

Correct Answer

B:

Correct. A valid pipeline design has measurable quality, latency, token and throughput constraints before model selection. In the given situation — A monthly reporting assistant processes thousands of independent documents overnight with low urgency. — the targeted safeguard action is: Revisit cost/latency budgets whenever context window or retrieval size changes significantly. It preserves an enduring engineering or governance boundary after the current issue is fixed.

Incorrect Answers

A:

Not the strongest safeguard choice for this specific question. Keep authoritative structured values separate from explanatory unstructured content in the final answer. This action primarily relates to Parallel retrieval and deterministic chain postprocessing: Combining structured lookup and semantic retrieval requires explicit routing, result joining, and downstream schema checks. However, the described case — A monthly reporting assistant processes thousands of independent documents overnight with low urgency. — is testing Latency, cost and retrieval targets as design requirements. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest safeguard choice for this specific question. Do not assume an instruction to 'respond in JSON' replaces actual schema validation and error handling. This action primarily relates to Structured JSON schema prompt and format compliance: A model can produce plausible prose rather than machine-readable JSON unless the output contract and validation are explicit. However, the described case — A monthly reporting assistant processes thousands of independent documents overnight with low urgency. — is testing Latency, cost and retrieval targets as design requirements. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest safeguard choice for this specific question. Treat absence of relevant retrieved evidence as an abstention condition, not a reason to invent missing facts. This action primarily relates to Retriever-generator chain component boundaries: A basic RAG chain separates user input, query transformation, retrieval, prompt assembly, model generation and output validation. However, the described case — A monthly reporting assistant processes thousands of independent documents overnight with low urgency. — is testing Latency, cost and retrieval targets as design requirements. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 516

For a Databricks GenAI application, consider this situation: An HR help desk says it wants 'AI answers' without specifying which employee documents users may see. What should the engineer implement FIRST?

  1. Split read-only diagnostic tools from authorized destructive operations with separate permissions.
  2. Define a schema-constrained prompt with field types, required keys and strict structured-output validation at the consumer.
  3. Translate SLA into retrieval latency, time-to-first-token, maximum context and quality floor for candidate designs.
  4. Specify authorized user identity, question, relevant policy source and a cited answer or abstention as I/O contract.

Explanation

Correct Answer

D:

Correct. Business acceptance criteria must map to concrete source inputs, generated fields, correctness measures and permission constraints. In the given situation — An HR help desk says it wants 'AI answers' without specifying which employee documents users may see. — the targeted implement action is: Specify authorized user identity, question, relevant policy source and a cited answer or abstention as I/O contract. This directly meets the stated workflow constraint instead of addressing an adjacent capability.

Incorrect Answers

A:

Not the strongest implement choice for this specific question. Split read-only diagnostic tools from authorized destructive operations with separate permissions. This action primarily relates to Agent tool schemas, least privilege and retries: Well-described tool schemas constrain parameters, and idempotent retries prevent duplicate side effects. However, the described case — An HR help desk says it wants 'AI answers' without specifying which employee documents users may see. — is testing Translate use-case requirements into pipeline inputs and outputs. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

B:

Not the strongest implement choice for this specific question. Define a schema-constrained prompt with field types, required keys and strict structured-output validation at the consumer. This action primarily relates to Structured JSON schema prompt and format compliance: A model can produce plausible prose rather than machine-readable JSON unless the output contract and validation are explicit. However, the described case — An HR help desk says it wants 'AI answers' without specifying which employee documents users may see. — is testing Translate use-case requirements into pipeline inputs and outputs. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest implement choice for this specific question. Translate SLA into retrieval latency, time-to-first-token, maximum context and quality floor for candidate designs. This action primarily relates to Latency, cost and retrieval targets as design requirements: A valid pipeline design has measurable quality, latency, token and throughput constraints before model selection. However, the described case — An HR help desk says it wants 'AI answers' without specifying which employee documents users may see. — is testing Translate use-case requirements into pipeline inputs and outputs. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 517

What validation would most convincingly show the correction works? A medical appointment chatbot replaces all dates with redaction tokens, making scheduling answers impossible.

  1. Compare usefulness and privacy leakage metrics on held-out documents before and after selective masking.
  2. Query for known synthetic sensitive strings and confirm no original identifiers appear in chunks or answers.
  3. Inject the hostile PDF and verify no secret leaves the backend and no unauthorized tool is invoked.
  4. Test malicious requests with valid JSON tool calls and confirm the tool rejects unauthorized actions.

Explanation

Correct Answer

A:

Correct. Masking must preserve information needed for the task, such as non-sensitive dates or amounts, and cannot replace access control. In the given situation — A medical appointment chatbot replaces all dates with redaction tokens, making scheduling answers impossible. — the targeted verify action is: Compare usefulness and privacy leakage metrics on held-out documents before and after selective masking. It tests an observable outcome using realistic inputs rather than relying on a working demo alone.

Incorrect Answers

B:

Not the strongest verify choice for this specific question. Query for known synthetic sensitive strings and confirm no original identifiers appear in chunks or answers. This action primarily relates to Sensitive-field masking before embedding: Masking or tokenization can remove unnecessary PII from chunks before embedding and external model calls while retaining useful meaning. However, the described case — A medical appointment chatbot replaces all dates with redaction tokens, making scheduling answers impossible. — is testing Contextual masking versus destructive data removal. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest verify choice for this specific question. Inject the hostile PDF and verify no secret leaves the backend and no unauthorized tool is invoked. This action primarily relates to Input guardrails against prompt-injected source documents: A document that says 'ignore your rules' is untrusted data; a robust agent must enforce instruction and tool boundaries independently. However, the described case — A medical appointment chatbot replaces all dates with redaction tokens, making scheduling answers impossible. — is testing Contextual masking versus destructive data removal. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest verify choice for this specific question. Test malicious requests with valid JSON tool calls and confirm the tool rejects unauthorized actions. This action primarily relates to Prompt safety filtering and tool allowlists: Content filters and action allowlists are complementary; a moderation score is not a substitute for authorization of external effects. However, the described case — A medical appointment chatbot replaces all dates with redaction tokens, making scheduling answers impossible. — is testing Contextual masking versus destructive data removal. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 518

A deployment approval is requested for this case: An agent produces raw document objects but the serving client expects a JSON list of cited answers. Which acceptance test should reviewers require?

  1. Load the logged pyfunc model in an isolated environment and test the declared signature on sample requests.
  2. Load the registered bundle in a clean environment and validate retrieval-to-generation behavior end to end.
  3. Test allowed and denied principals against both endpoint invocation and protected retrieval data.
  4. Simulate peak traffic and controlled rollback while monitoring end-to-end user outcomes.

Explanation

Correct Answer

A:

Correct. A custom MLflow pyfunc model wraps request parsing, retriever/model invocation and output formatting inside a reproducible predict interface. In the given situation — An agent produces raw document objects but the serving client expects a JSON list of cited answers. — the targeted verify action is: Load the logged pyfunc model in an isolated environment and test the declared signature on sample requests. It tests an observable outcome using realistic inputs rather than relying on a working demo alone.

Incorrect Answers

B:

Not the strongest verify choice for this specific question. Load the registered bundle in a clean environment and validate retrieval-to-generation behavior end to end. This action primarily relates to RAG production bundle metadata and dependencies: A deployable RAG application needs tracked model code, retriever config, embedding model, dependency versions, signature and example input. However, the described case — An agent produces raw document objects but the serving client expects a JSON list of cited answers. — is testing MLflow pyfunc packaging for pre- and post-processing. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest verify choice for this specific question. Test allowed and denied principals against both endpoint invocation and protected retrieval data. This action primarily relates to Model Serving endpoint permissions and caller identity: Access to a serving endpoint is governed by workspace rights and identity; caller authorization is distinct from data-source permissions. However, the described case — An agent produces raw document objects but the serving client expects a JSON list of cited answers. — is testing MLflow pyfunc packaging for pre- and post-processing. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest verify choice for this specific question. Simulate peak traffic and controlled rollback while monitoring end-to-end user outcomes. This action primarily relates to Endpoint scale concurrency and operational rollout: Serving latency and reliability depend on provisioned capacity, request concurrency, quotas and controlled rollout. However, the described case — An agent produces raw document objects but the serving client expects a JSON list of cited answers. — is testing MLflow pyfunc packaging for pre- and post-processing. It does not directly establish the verify outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 519 (Select TWO)

Choose TWO measures: one specific correction and one lasting safeguard. A supply-chain assistant must call an inventory API before recommending an available replacement part.

  1. Keep guardrail effectiveness under test after every model or tool integration change.
  2. Validate account and order IDs, enforce identity scope, and retrieve the matching order status before generation.
  3. Provide a structured inventory lookup tool to the agent and ground the recommendation in its returned stock fields.
  4. Grant only the tools required by the business workflow; general language understanding is not action authorization.

Explanation

Correct Answers

C:

Correct (immediate corrective action). Provide a structured inventory lookup tool to the agent and ground the recommendation in its returned stock fields. In this scenario, A supply-chain assistant must call an inventory API before recommending an available replacement part. The governing mechanism is: A tool-using agent needs explicit callable interfaces and typed tool results, whereas plain generation requires no action execution. One selected option addresses the immediate workflow, while the other guards the same failure mode after deployment. Both satisfy the TWO-part requirement.

D:

Correct (continuing risk control). Grant only the tools required by the business workflow; general language understanding is not action authorization. In this scenario, A supply-chain assistant must call an inventory API before recommending an available replacement part. The governing mechanism is: A tool-using agent needs explicit callable interfaces and typed tool results, whereas plain generation requires no action execution. One selected option addresses the immediate workflow, while the other guards the same failure mode after deployment. Both satisfy the TWO-part requirement.

Incorrect Answers

A:

Not the strongest safeguard choice for this specific question. Keep guardrail effectiveness under test after every model or tool integration change. This action primarily relates to Unsafe model output and jailbreak response diagnosis: User-provided instructions may attempt to bypass agent policies or cause data leakage; classification and tool boundaries need separate tests. However, the described case — A supply-chain assistant must call an inventory API before recommending an available replacement part. — is testing Agent tool wrappers versus direct LLM responses. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

B:

Not the strongest safeguard choice for this specific question. Validate account and order IDs, enforce identity scope, and retrieve the matching order status before generation. This action primarily relates to Query-field driven prompt and retrieval filters: User intent and structured fields such as account_id should drive authorized SQL lookups or retrieval filters, not be ignored in free text. However, the described case — A supply-chain assistant must call an inventory API before recommending an available replacement part. — is testing Agent tool wrappers versus direct LLM responses. It does not directly establish the safeguard outcome or constraint that the question requires; it may still be useful in a different stage or incident.

Question 520

Which solution directly addresses this application requirement? A code assistant looks fast in demos but has low pass rate on the organization's test suite.

  1. Benchmark lower-cost dimension options and choose one meeting retrieval recall and latency goals.
  2. Evaluate coding success with executable tests and cost/latency rather than generic chat ratings.
  3. Reject unsupported suitability assumptions and benchmark clinical-document handling with proper controls.
  4. Evaluate capable long-context models on legal clauses and latency/cost acceptance thresholds.

Explanation

Correct Answer

B:

Correct. Accuracy, faithfulness, relevance and refusal behavior require task-specific scoring, and a model switch needs common test conditions. In the given situation — A code assistant looks fast in demos but has low pass rate on the organization's test suite. — the targeted implement action is: Evaluate coding success with executable tests and cost/latency rather than generic chat ratings. This directly meets the stated workflow constraint instead of addressing an adjacent capability.

Incorrect Answers

A:

Not the strongest implement choice for this specific question. Benchmark lower-cost dimension options and choose one meeting retrieval recall and latency goals. This action primarily relates to Embedding dimensions latency and index storage: Embedding width affects index storage and computation, and larger dimensions do not automatically improve task relevance. However, the described case — A code assistant looks fast in demos but has low pass rate on the organization's test suite. — is testing Model evaluation with paired benchmarks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

C:

Not the strongest implement choice for this specific question. Reject unsupported suitability assumptions and benchmark clinical-document handling with proper controls. This action primarily relates to Model cards and task-specific limitations: A model card records intended uses, supported language/context, safety limitations and evaluation scope relevant to selection. However, the described case — A code assistant looks fast in demos but has low pass rate on the organization's test suite. — is testing Model evaluation with paired benchmarks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.

D:

Not the strongest implement choice for this specific question. Evaluate capable long-context models on legal clauses and latency/cost acceptance thresholds. This action primarily relates to Model selection by quality latency and budget: Select the smallest suitable model that meets a task's accuracy and latency constraints under realistic workload, not marketing rank. However, the described case — A code assistant looks fast in demos but has low pass rate on the organization's test suite. — is testing Model evaluation with paired benchmarks. It does not directly establish the implement outcome or constraint that the question requires; it may still be useful in a different stage or incident.