Question 310
In a secure development review, the following condition is confirmed: A developer compares two tools using only one contrived coding prompt. What is the best next step under this constraint?
Explanation
Correct Answer
The appropriate action is "Use representative scenarios and agreed evaluation criteria". Reliable comparisons require meaningful workloads and repeated evidence. The decisive condition is A developer compares two tools using only one contrived coding prompt.
Incorrect Answers
The alternative "Measure relevant outcomes before making a causal claim" addresses A team wants to claim Copilot always improves developer productivity. Speed improvements vary by project and task. It does not resolve A developer compares two tools using only one contrived coding prompt; the correct choice is "Use representative scenarios and agreed evaluation criteria". Reliable comparisons require meaningful workloads and repeated evidence.
"Improve context and provide feedback through supported controls" targets A code suggestion is repeatedly irrelevant in one repository. Feedback and contextual adjustments can improve usefulness. That differs from A developer compares two tools using only one contrived coding prompt; the required decision here is "Use representative scenarios and agreed evaluation criteria". Reliable comparisons require meaningful workloads and repeated evidence.
The action "Evaluate outputs using tests and authoritative sources" is useful when An engineer assumes reported model confidence guarantees correctness. Confidence signals are not independently calibrated verification. Under the stated condition A developer compares two tools using only one contrived coding prompt, instead choose "Use representative scenarios and agreed evaluation criteria". Reliable comparisons require meaningful workloads and repeated evidence.