Item analysis
/recruiting/assessments/item-analysis
Permission: recruiting.assessment_engine.manage (content authors).
A psychometric dashboard for measuring per-question quality from real candidate response data. Use it to identify questions to retire or rewrite.
Filters
- Template — pick which template's questions to analyse
Per-question metrics
| Column | What it measures |
|---|---|
| n | Number of candidate responses to this question |
| Difficulty | % of candidates who got it right (with a healthy-range badge) |
| Discrimination | How well this question separates high vs. low scorers (with low-discrimination flag) |
Healthy ranges and discrimination thresholds are server-provided metadata based on standard psychometric practice — typically you want difficulty around 0.3–0.7 and discrimination above 0.2.
Distractor analysis
Click the expand button on a row to see distractor effectiveness — the selection counts for each option. Two flags surface:
- Correct answer — labelled so you can see where the right answer ranked among picks
- Non-functioning distractor — options no candidate ever picks. These are dead weight; rewrite or remove.
How to use it
A typical content review pass:
- Sort by n descending to focus on questions with enough data.
- Look for discrimination flags — questions that don't distinguish stronger candidates from weaker ones aren't doing their job.
- Look for difficulty at the extremes — too easy (everyone passes) or too hard (everyone fails) means the question carries no signal.
- Expand any flagged row and check distractors — non-functioning distractors are the easiest fix.
Retire or rewrite the worst offenders, then re-publish the bank and let data accumulate again.