Question 90
A developer is testing a Snowflake Gen AI workflow with this requirement: An assistant answers contract questions that sometimes require two annexes. Which observation most directly tests whether the proposed solution worked?
Explanation
Correct Answer
“Relevant-passage recall at k, irrelevant-context rate, answer quality and model-token cost” is supported by this requirement: An assistant answers contract questions that sometimes require two annexes. Retrieval-depth decisions trade off answer evidence, noise and context expenditure. Verify it using relevant-passage recall at k, irrelevant-context rate, answer quality and model-token cost.
Incorrect Answers
“Single-passage retrieval count even when evidence spans multiple files” would not meet the requirement that an assistant answers contract questions that sometimes require two annexes. Retrieval-depth decisions trade off answer evidence, noise and context expenditure. The more reliable approach is to tune top-k retrieval and any reranking based on measured recall, precision, cost and final answer quality.
“Maximum number of returned passages with no ranking relevance” would not meet the requirement that an assistant answers contract questions that sometimes require two annexes. Retrieval-depth decisions trade off answer evidence, noise and context expenditure. The more reliable approach is to tune top-k retrieval and any reranking based on measured recall, precision, cost and final answer quality.
“Raw index storage volume alone” would not meet the requirement that an assistant answers contract questions that sometimes require two annexes. Retrieval-depth decisions trade off answer evidence, noise and context expenditure. The more reliable approach is to tune top-k retrieval and any reranking based on measured recall, precision, cost and final answer quality.