Noise in Judgement

'Noise: A Flaw in Human Judgement' by Daniel Kahneman, Olivier Sibony, and Cass Sunstein is one of my top recommended readings. While I'm still very much a junior behavioural scientist, as I read I found myself connecting the insights to my experiences within my own domains of expertise.

The authors make use of Pythagorean theorem and triangles to explain the relationship between different components of error within human judgement. But what I liked most were the visualisations.

I've recreated the key diagram as an interactive infographic. Clicking on a component will render a description below the diagram:

Mean Squared Error (MSE) Mean Squared Error (MSE) Mean Squared Error (MSE) Bias² Bias² System Noise² System Noise² Level Noise² Level Noise² Pattern Noise² Pattern Noise² Occasion Noise² OccasionNoise² Stable Pattern Noise² StablePattern Noise²

Stable Pattern Noise²

Stable pattern noise is pattern noise produced by long term or permanent factors (stable factors). For instance, a tendency to favour candidates from particular educational institution or rating certain wines higher due to knowledge about where the grapes were grown.

You're specific life experiences and expertise play a role in stable pattern noise. My history as an engineer historically lead me to focus more on the technical considerations when designing and building a new service —something I'm now very concious of—. A user researcher will almost certainly be thinking more about how the user will interact with it. Business and data analysts will be more concerned with the whats and hows of data and measurement. The client is fairy land envisioning customers riding unicorns while management is in pixie land scheduling impossibilities. Our conclusions on the feasibility and value of the project may be poles apart.

Individual variation shows evidence of analysis performed from different perspectives by people with different skills and knowledge. Exploiting the benefits of diverse thinking and training one could say. Too much variation can introduce significant extra cost into decision making. This can take the form of delayed development or poor compromises that undermine the resultant product's ability to compete in the market. The additional unwanted variation is pattern noise we can benefit from reducing.

Sources

[book] Noise: A Flaw in Human Judgement. 2021. D Kahneman, O Sibony, C R Sunstein [Pub] Harper Collins [ISBN] 978-0-00-830903-9.

[book] Knowledge and decisions. 1996. T Sowell [Pub] Basic Books [ISBN] 978-0465037384.