Question 150
During a controlled generative AI workflow test, engineers observe: A customer service bot keeps answering with a superseded returns policy even though the company replaced the source PDF yesterday and its file server now hosts the current version. Which observation confirms that the AI workflow now meets its intended constraint?
Explanation
Correct Answer
This specifically verifies rag index stale after approved document revision: A known change in the returns-policy fixture appears in the index and answer, while the prior revoked clause no longer appears in retrieval. The related engineering correction is: Trigger an approved incremental reindex or versioned ingestion job for changed documents and remove obsolete source versions from eligible retrieval results. Evidence from another workflow would not establish the stated outcome.
Incorrect Answers
A repeat build from the approved revision generates consistent test outputs within the expected deterministic limitations and records its artifact digests. “Pin a verified model revision and tokenizer version and introduce a compatibility test before changing the deployed artifact” would remedy that alternative situation, but it would not correct the present issue: a known change in the returns-policy fixture appears in the index and answer, while the prior revoked clause no longer appears in retrieval.
“Bind search filters to a verified tenant identity and restrict both vector retrieval and underlying source document access before assembling prompts” is the supported response when a tenant A test query retrieves only tenant A passages, and an intentionally related tenant B passage never enters the prompt or generated answer. Here the decisive response is “Trigger an approved incremental reindex or versioned ingestion job for changed documents and remove obsolete source versions from eligible retrieval results”, so that different intervention does not meet the stated requirement.
The test index retrieves the required known relevant articles within the stated recall tolerance and response-time budget. “Tune index parameters and candidate retrieval depth using a labeled relevance benchmark with a documented recall and latency target” applies to that other condition. The reported evidence is a customer service bot keeps answering with a superseded returns policy even though the company replaced the source PDF yesterday and its file server now hosts the current version; this alternative would address the wrong dependency.