Noise - KeynesYouDigIt/Knowledge GitHub Wiki

Noise: How to overcome the high, hidden cost of inconsistent decision making

Decisions can be:

• Accurate
• Noisy (same input produces different outputs)
• Biased (outputs are consistently skewed one direction)
• Noisy and biased

Managers often know that decisions are noisy and/or biased, but drastically underestimate how much. Professionals have high confidence in their own estimation abilities and also have a high regard for their colleagues' intelligence, which leads to overestimation of how much they agree. Where there is judgment, there is noise, and usually more of it than you think. Noise in judgment scenarios (like chess and driving) is reduced by fast feedback loops.

Noise audits: Even without knowing what the target should be, you can tell whether the results are scattered

Noise index: The average result / Range of results

You reduce noise by building an algorithm (a "Reasoned Rule"), and you don't even need any outcome data up front.

1. Select 6-8 variables that are distinct and obviously related to the outcome.
2. Take your historical cases and compute the mean (μ) and standard deviation (σ) of each variable.
3. For each piece of historical data in a case (x), compute a "standard score" = (x - μ) / σ
4. Compute a "summary score" for each case = average of its variables standard scores
5. Order your cases from best summary score to worst, and determine the appropriate actions for each range