Question 390
The data governance board is asked to authorize this configuration: A deployed agent shows inconsistent multi-tool sequences after a model upgrade. What measurement should be required at the technical acceptance gate?
Explanation
Correct Answer
“Trace-linked task-quality metrics, retrieval step evidence, error reasons and comparable regression evaluations” is supported by this requirement: A deployed agent shows inconsistent multi-tool sequences after a model upgrade. Operational LLM observability connects end-to-end outcomes with model, prompt, retrieval and tool behavior. Verify it using trace-linked task-quality metrics, retrieval step evidence, error reasons and comparable regression evaluations.
Incorrect Answers
“The number of evaluation rows deleted during failures” would not meet the requirement that a deployed agent shows inconsistent multi-tool sequences after a model upgrade. Operational LLM observability connects end-to-end outcomes with model, prompt, retrieval and tool behavior. The more reliable approach is to instrument the application with supported traces, evaluation datasets and quality metrics to diagnose model and retrieval failures.
“Uptime without faithfulness or retrieval diagnostics” would not meet the requirement that a deployed agent shows inconsistent multi-tool sequences after a model upgrade. Operational LLM observability connects end-to-end outcomes with model, prompt, retrieval and tool behavior. The more reliable approach is to instrument the application with supported traces, evaluation datasets and quality metrics to diagnose model and retrieval failures.
“HTTP 200 rate alone” would not meet the requirement that a deployed agent shows inconsistent multi-tool sequences after a model upgrade. Operational LLM observability connects end-to-end outcomes with model, prompt, retrieval and tool behavior. The more reliable approach is to instrument the application with supported traces, evaluation datasets and quality metrics to diagnose model and retrieval failures.