It's definitely been clear for a while that LLMs have human failure modes. Just like people, If you try to trick it by subtly varying a common word problem, it

It's definitely been clear for a while that LLMs have human failure modes. Just like people, If you try to trick it by subtly varying a common word problem, it may get it wrong. But that doesn't actually mean it can't solve the problem. Rewriting the problem so it doesn't bias common priors, being clear there's a twist, or telling it it's making a wrong assumption all seems to have some degree of success, just like people.