A data scientist spends five months on a demand prediction model for a product team. In month five, she shows that the signal is too weak to beat the existing rule of thumb and recommends stopping. The annual review form has one box: features shipped. She has none. Her manager is unsure how to rate five months of careful work that saved the company from a poor launch.
This is the central problem of reviewing research and data roles. The calendar of the review cycle does not match the calendar of the work.
Why the usual review form fails
Most forms count outputs: tickets closed, releases made, revenue touched. Data work has long stretches where the output is understanding. Data arrives late. Labels are wrong. A promising approach fails. If you rate only shipped results, people learn to pick safe, small problems. If you rate only effort, you cannot tell good work from busy work.
There is also the matter of luck. Two scientists may work equally well, yet one finds clean data and a willing business owner while the other meets neither. A review that ignores this context will reward circumstance. Ask in every review what helped and what got in the way, and record it.
Break a long project into checkpoints
The fix is to review what the person can control at each stage and to write the stages down at the start. A rough sequence suits most data and research projects. The stage names matter less than agreeing them early.
- Problem framing: a clear statement of the question, the decision it will support and how success will be measured.
- Data readiness: what data exists, what is missing, what quality checks were done and what the gaps mean.
- Baseline: the simplest approach that works, so that any improvement can be measured against something.
- Experiments: versions tried, recorded with results, including those that failed.
- Handover: documentation, code and monitoring that someone else can run.
- Decision: a recommendation the business can act on, which may be to proceed, adjust or stop.
Review in five steps
- 1
Agree the checkpoints
At the start of the cycle, the manager and the person write the stages for each main project and a rough month for each.
- 2
Hold a monthly check-in
Thirty minutes. What moved, what blocked, what was learned. The manager keeps three lines of notes after each.
- 3
Collect evidence as you go
Keep links to design notes, experiment logs, reviews by peers and messages from the people who used the work.
- 4
Ask peers and users
Before the review, ask two colleagues and one internal or client user what the person made easier or clearer.
- 5
Write the review from the notes
Rate each checkpoint reached, the quality of judgement and the effect on others. Add context where outside events changed the plan.
Reward good judgement, including stopping
A researcher who halts a weak idea at month two has saved the company months. A researcher who keeps polishing a model with no signal has not. Add a line to your review sheet that asks whether the person spotted weak ideas early, said so plainly and proposed an alternative. If your culture punishes unwelcome findings, people will hide them.
Also look at how the person talks about uncertainty. Good researchers say what they know, what they suspect and what they have not tested. Weak ones sound equally sure about everything. A manager can listen for this in monthly check-ins and note examples in the file.
Separate the role types
Engineers, analysts and scientists have different work and need different emphasis, as we explained in our piece on hiring data engineers, analysts and data scientists. Engineers are judged on reliability and cost of pipelines. Analysts are judged on how well their answers changed a decision. Scientists are judged on rigour, honesty about limits and the usefulness of their recommendation.
Role, period and main projects.
Checkpoints agreed at the start and where each project stands now.
Quality of method: problem framing, data checks, baseline, honest reporting of limits.
Judgement: ideas stopped or redirected, and why.
Effect on others: handovers, documentation, reviews given, help to juniors.
Evidence attached: links to notes, logs and feedback.
Context: events outside the person's control that shaped the period.
Agreed focus for the next period and support needed.Handle outcomes you cannot see yet
A model may be delivered and not used for another quarter. Do not wait for the final business effect to rate the work. Rate the quality at handover, and add a follow-up note later when the result is known. If the result changes the picture, record it in the next cycle without rewriting the earlier one. This keeps the process honest and avoids fights about credit.
Link to pay and growth with care
If every rating drives a pay change, people will bargain over ratings. Keep conversations about learning and growth apart from pay decisions where you can, and decide pay from agreed bands and levels, which your company sets. Our performance management process page describes how to build this structure. Teams in product companies, including those described in our SaaS and product companies guide, often mix quarterly goals with longer research cycles, and the same checkpoint method works there.
