Carragee Consulting

I started re-reading the Alignment Problem and saw that the author Brian Christian references Box too. He gives the cautionary tail of the sorcerer’s apprentice, bewitching brooms to haul water on his behalf, carrying out their prescribed task to a T with an undesirable result, to illustrate the problem of ill-defined objective functions.

In my own work I often wonder will any of my analysis be applied to change decisions or will it result only in slide-ware.

Sometimes I can control these by ensuring my analysis or modeling finds the right audience or planning ahead of time to integrate predictive modeling into a business process — eg Target Price embedded in the pricing/quoting system.

A good question to ask is “What if my model is right and no one cares or implements it?” versus “What if I’m wrong?” and the sorcerer’s apprentice situation occurs?

Models won’t be either 100% right or 100% wrong and so in reality you have to plan for both outcomes — eg allow Sales to override Target Price within pre-established boundaries.

The context of the problems you’re trying to solve, the analytical method you’ve chosen and the implementation of the recommendations are all important factors to consider how to put guardrails around your statistical analysis to maximize the beneficial impact while minimizing the unintended consequences and downside the perfect executive of a flawed model would generate