I think the phrase you are looking for is that LLMs/GPTs are "a solution in search of a problem". They suffer from the exact same that every neural network has

I think the phrase you are looking for is that LLMs/GPTs are "a solution in search of a problem". They suffer from the exact same that every neural network has in that their behaviour is unpredictable in corner cases. That's not something one can trust their business to. There have been enough legal issues created by simple chat bots agreeing to deals with customers that I don't see how one can build trustworthy automation on top of LLMs/GPTs without a human in the loop to catch the failures. Of course, there will be plenty of people that will do that regardless.