Question 300
A data platform architect must resolve the following design constraint: A bank policy helpdesk must explain updated prose exceptions without converting policy text into database tables. Which observation most directly tests whether the proposed solution worked?
Explanation
Correct Answer
“Grounded answer accuracy on held-out document questions, measured along with passage relevance and source coverage” is supported by this requirement: A bank policy helpdesk must explain updated prose exceptions without converting policy text into database tables. A retrieval layer locates relevant current unstructured evidence before an LLM composes an answer. Verify it using grounded answer accuracy on held-out document questions, measured along with passage relevance and source coverage.
Incorrect Answers
“Successful conversion of natural-language requests into aggregation SQL” would not meet the requirement that a bank policy helpdesk must explain updated prose exceptions without converting policy text into database tables. A retrieval layer locates relevant current unstructured evidence before an LLM composes an answer. The more reliable approach is to use a Cortex Search service to retrieve relevant approved text passages, then ground the generation prompt in those passages.
“Prompt-token consumption without an answer-quality measurement” would not meet the requirement that a bank policy helpdesk must explain updated prose exceptions without converting policy text into database tables. A retrieval layer locates relevant current unstructured evidence before an LLM composes an answer. The more reliable approach is to use a Cortex Search service to retrieve relevant approved text passages, then ground the generation prompt in those passages.
“Total number of documents uploaded regardless of retrieval relevance” would not meet the requirement that a bank policy helpdesk must explain updated prose exceptions without converting policy text into database tables. A retrieval layer locates relevant current unstructured evidence before an LLM composes an answer. The more reliable approach is to use a Cortex Search service to retrieve relevant approved text passages, then ground the generation prompt in those passages.