Question 480
A new foundation model version improves average accuracy but introduces refusals for a legitimate financial-advice support intent. What release comparison should detect that regression?
Explanation
Correct Answer
Safety includes avoiding unwarranted blocks on permitted requests; per-intent evaluation reveals over-refusal hidden by averages.
Incorrect Answers
Historical results are needed for credible version-to-version comparisons and rollback decisions.
Removing safety tests hides both underblocking and overblocking behavior.
Overblocking legitimate users can violate product utility and must be measured.