Question 140
A reviewer is challenging several proposals for this case: Two assessors score identical governance evidence differently because the rubric says only 'good practice'. Which proposal should be rejected because it most directly violates the case's control objective?
Explanation
Correct Answer
Clear rating criteria improve inter-rater consistency and assessment credibility. “Allow each assessor to invent level meanings at review time.” would not meet the stated operational condition. Publish behavioral anchors, scoring thresholds and calibration examples for each level. Re-score a blinded sample and measure assessor agreement after calibration.
Incorrect Answers
Publish behavioral anchors, scoring thresholds and calibration examples for each level. Clear rating criteria improve inter-rater consistency and assessment credibility. Re-score a blinded sample and measure assessor agreement after calibration.
Re-score a blinded sample and measure assessor agreement after calibration. Clear rating criteria improve inter-rater consistency and assessment credibility. Publish behavioral anchors, scoring thresholds and calibration examples for each level.
“Document the inconsistent rating rubric decision, its owner, limitations and a review date before closure.” records a controlled change and supports accountable evaluation. Clear rating criteria improve inter-rater consistency and assessment credibility. Re-score a blinded sample and measure assessor agreement after calibration.