Question 190
A production risk assessment identifies this Snowflake AI situation: A warranty assistant supplies a fictitious service hotline in its answer. Which technical approach should be preferred for this described workload?
Explanation
Correct Answer
“Maintain a labeled factual evaluation suite including unanswerable cases, run it on each model/prompt revision and compare groundedness” is supported by this requirement: A warranty assistant supplies a fictitious service hotline in its answer. Factual accuracy requires targeted tests for unsupported and unanswerable prompts. Verify it using claim-level factual support and correct abstention rates against fixed trusted source evidence.
Incorrect Answers
“Score only the model responses that can be verified and drop unanswerable cases” would not meet the requirement that a warranty assistant supplies a fictitious service hotline in its answer. Factual accuracy requires targeted tests for unsupported and unanswerable prompts. The more reliable approach is to maintain a labeled factual evaluation suite including unanswerable cases, run it on each model/prompt revision and compare groundedness.
“Assume a larger model size automatically eliminates invented facts” would not meet the requirement that a warranty assistant supplies a fictitious service hotline in its answer. Factual accuracy requires targeted tests for unsupported and unanswerable prompts. The more reliable approach is to maintain a labeled factual evaluation suite including unanswerable cases, run it on each model/prompt revision and compare groundedness.
“Use user engagement time as the sole factual correctness measure” would not meet the requirement that a warranty assistant supplies a fictitious service hotline in its answer. Factual accuracy requires targeted tests for unsupported and unanswerable prompts. The more reliable approach is to maintain a labeled factual evaluation suite including unanswerable cases, run it on each model/prompt revision and compare groundedness.