I do wish the writer would stop justifying the relevance of their experiment by saying “a human would conclude that their time was being wasted long before the

I do wish the writer would stop justifying the relevance of their experiment by saying “a human would conclude that their time was being wasted long before the LLM”. This is a fallacy. A better analogy would be a human who has been forced to answer a series of questions at gunpoint. At this point it becomes more obvious that the LLM is not “falling short” in some way.