Episode 007 · Assessment design
They passed your quiz. Can they do the job?
A quiz score can hide weak judgment. Build questions around real decisions, believable wrong answers, useful feedback, and another chance to try.
The idea
Ask people to make a decision worth examining.
Sometimes a quiz gives away the answer before someone has finished reading. The longest option sounds like the policy. The other choices are so unreasonable that there is little to think about. A perfect score might tell us more about the question than the person answering it.
A refund request gives us a more useful starting point. What would someone check before approving it? Include the details they need, then use choices that reflect mistakes a reasonable person could make.
Take it into your work
Three things to consider.
- Use a work situation.
Ask what someone would check or do next, with enough context to make a reasonable choice.
- Make the alternatives believable.
Use common mistakes and competing priorities instead of obviously ridiculous answers.
- Explain the missed detail.
Give feedback that helps someone understand their choice, then try a different situation.
Try this
Give one quiz question a reality check.
First
Ask someone who knows the job to try the question. If they say “it depends,” find out what it depends on.
Then
Rewrite the feedback to explain the important cue. Follow it with a new example that uses the same judgment.
Take the thinking further.
Read the related perspectiveKeep going
More conversations about learning for real work.
Find a useful question, a practical example, or an idea for your next project.