Carragee Consulting

Systematic failures in ChatGPT – Don’t throw the baby out with the bath water

This image used from Andrei Lopatenko‘s LinkedIn post

I’ve seen a lot of examples of how ChatGPT gets verbal math problems wrong, basically failing 5th grade algebra.

In the example above, the chat bot misunderstands tasks which inherently need to be preformed simultaneously, with ones which can be done in parallel which is a classic project management failure mode too known as the “pregnancy fallacy“.

What is interesting to me is not that it is wrong, but that it is wrong in some consistent ways — this means that it can be retrained to fix the systemic failures.

I used to have a car with an intermittent fault that caused the engine to shut down seemingly randomly. I paid two separate times to have the car “fixed” only for it to fail again in a dangerous situation so I eventually got rid of the car.

As with any other new tool, folks are casting around for the right use cases where ChatGPT can be effectively and safely applied. Some people will find glaring errors in domains they are knowledgeable about and decide the entire thing is useless, but others will find ways to apply it for time savings and other valuable benefits.

I don’t plan to be an early adopter of ChatGPT because there will be some lessons we learn the hard way, but as it evolves I’m interested to see where it gets embedded.