The error rate nobody collects
The error rate nobody collects
Every process has one. Almost no process has it written down, and the number is usually higher than anyone in the building would guess.
You need it for one specific reason: without it, your system will be held to a standard of perfection that the thing it replaced never met.
Why this is the number that protects you
Six weeks from now your system will make a mistake. Somebody will notice, and the reaction will be about whether it can be trusted at all.
If you know the human process was wrong 4% of the time and yours is wrong 2%, that is a conversation about an improvement. If you have no baseline, it is a conversation about your system being unreliable, and you have no way to answer it. The absence of a baseline does not mean the old process was perfect — it means nobody can prove it was not.
Collect this before you build. Afterwards it looks like you went looking for excuses.
Define an error before you count one
Write down what counts as wrong, precisely enough that two people would classify the same instance the same way. Then have somebody else classify twenty instances using your definition and check whether you agree.
Disagreement here is common and is itself the finding — a process where nobody can consistently say what wrong means is one where you cannot claim an improvement.
Measuring it without making it personal
You said in Module 0 that you are measuring the process rather than people. This is the lesson where that promise gets tested.
Measure at the aggregate level and report it that way. Do not produce a per-person breakdown, do not let one be derived from what you publish, and say so before you start collecting.
Beyond the ethics, there is a practical reason: the moment people believe error rates are attributable, the error rate you can observe drops and the real one does not. You will have destroyed the instrument.
Baseline error rate, defined and counted
Percentage of instances that meet your written definition of wrong, across the baseline period. Put the definition itself in the note — the number is meaningless without it.
Nobody can tell you what wrong means
You try to write the definition and discover the process has no agreed notion of a correct outcome. Two experienced people classify the same instances differently about a third of the time.
The uncomfortable number
You have measured something the organisation has been operating without for years.