Question 220
Before accepting this AI feature, reviewers note: A policy chatbot receives a question about an unpublished reimbursement threshold and confidently invents a number even though the retrieval stage found no matching policy passage. Which underlying AI system principle explains these observations?
Explanation
Correct Answer
The technical basis is: A model can generate plausible statements without evidence, and a prompt that demands a definitive answer can suppress appropriate uncertainty. Resolve this through: Require the response to cite retrieved policy evidence and state that the threshold is unavailable when the authorized source corpus has no support. The expected evidence is: A known missing-policy question returns an explicit evidence gap and an available-policy question cites a genuine source passage.
Incorrect Answers
A prompt with mutually incompatible constraints cannot be made reliable through repetition alone; the task needs a defined priority or an explicit clarification rule. “Resolve or rank the conflicting constraints in a maintained prompt policy and permit a concise conflict notice when the requirements cannot coexist” would remedy that alternative situation, but it would not correct the present issue: a model can generate plausible statements without evidence, and a prompt that demands a definitive answer can suppress appropriate uncertainty.
“Ask the model to state visual uncertainty and request a sharper image or a calibrated instrument reading before using the number operationally” is the supported response when no prompt wording can reliably recover measurement detail absent from the image, so requesting unwarranted precision encourages unsupported guesses. Here the decisive response is “Require the response to cite retrieved policy evidence and state that the threshold is unavailable when the authorized source corpus has no support”, so that different intervention does not meet the stated requirement.
A tool response is lower-trust observation data even if its format resembles a command; treating it as an instruction can create prompt-injection-driven workflow hijacking. “Parse the tool result through an allowlisted data schema and require trusted task authorization before scheduling any additional tool action” applies to that other condition. The reported evidence is a policy chatbot receives a question about an unpublished reimbursement threshold and confidently invents a number even though the retrieval stage found no matching policy passage; this alternative would address the wrong dependency.