Question 400
At an AI model development checkpoint, an AI project leader faces this situation: A modeler tries hundreds of parameter combinations on the final test set and selects the best result. The chosen approach must remain reproducible and satisfy release criteria. Which next project action directly addresses the concern?
Explanation
Correct Answer
In an AI model development checkpoint, the most direct response is: Tune on training and validation partitions, reserving an untouched final holdout. This addresses: Parameter search must be separated from final evaluation to prevent biased model selection. Its acceptance evidence is: The run log identifies chosen hyperparameters and a one-time final holdout result. The project should also remember: Do not publish the highest observed validation score without accounting for search selection.
Incorrect Answers
Define a missingness treatment using an indicator or justified imputation strategy. The appropriate correction for the reported case is: Tune on training and validation partitions, reserving an untouched final holdout.
Test the production-like serving environment under representative peak load and failure conditions. This alternative is not the failing dependency described in the question.
Benchmark compact and larger candidate models under the target device or network conditions. Its verification is distinct from the reported condition: A modeler tries hundreds of parameter combinations on the final test set and selects the best result. The chosen approach must remain reproducible and satisfy release criteria.