Carragee Consulting

Working with insufficient data

One of the many ways standardized tests differ from “real life” is that option “E – Not enough information” is usually not a realistic answer.

I was at the Everyday AI conference hosted by Dataiku on Weds and one of the panel speakers, Bruno Aziza, said that the biggest factor in the future of AI is sufficient, high quality data.

While there are quantitative methods to work with sparse data, often we have to triangulate to decisions based an anecdotal or qualitative sources to decide action in the face of uncertain outcomes. We must rely on process or domain understanding to evaluate the likelihood of alternatives with imperfect information.

Annie Duke: Author, Professional Speaker & Decision Strategist presents a framework to get out of the infinite loop of indecision which I’ve labeled in this graphic as “calibrated confidence”.

Knowledge and data acquisition come at their own cost and sometimes the possible benefit of procuring more or better data isn’t worth it; either because it isn’t possible on your decision timeline, or the human or financial resources to acquire it exceed the decision utility it will provide.

Knowing when to stop pursuing more information and just decide is a hugely valuable skill for both your budget and your sanity. Don’t let perfect information be the enemy of good judgement.