Question 30
While evaluating AI productivity and governance, staff identify: An enterprise planning service asks a very lightweight model to reconcile conflicting operational constraints and multi-step dependencies, but the returned plan repeatedly violates requirements. What acceptance evidence demonstrates the requirement is satisfied?
Explanation
Correct Answer
This specifically verifies complex reasoning request sent to ultra small latency model: A held-out planning benchmark satisfies all stated dependencies at the agreed error and latency threshold without manual repair. The related engineering correction is: Benchmark approved reasoning-capable candidate models against representative multi-step cases and choose the least costly option that meets measured quality targets. Evidence from another workflow would not establish the stated outcome.
Incorrect Answers
The approved release inventory records the exact model revision and license obligations and excludes any model without adequate commercial rights. “Review the model license, permitted use, dependencies and required notices with appropriate legal approval before packaging the model in the product” would remedy that alternative situation, but it would not correct the present issue: a held-out planning benchmark satisfies all stated dependencies at the agreed error and latency threshold without manual repair.
“Test a smaller approved classification-capable model with fixed label definitions and switch only after comparable accuracy and stability are demonstrated” is the supported response when the lightweight model assigns the approved labels on a representative validation set while materially reducing response time and usage cost. Here the decisive response is “Benchmark approved reasoning-capable candidate models against representative multi-step cases and choose the least costly option that meets measured quality targets”, so that different intervention does not meet the stated requirement.
A controlled invoice scan correctly reports the printed totals and relevant chart labels with source-page provenance rather than guessing from missing text. “Use an approved vision-capable workflow or validated document extraction pipeline and check text, tabular and visual fields together” applies to that other condition. The reported evidence is an enterprise planning service asks a very lightweight model to reconcile conflicting operational constraints and multi-step dependencies, but the returned plan repeatedly violates requirements; this alternative would address the wrong dependency.