Here's where we see the models not having an understanding, but just spitting out madlibs as best they can. The prompter throws in a red herring in front of th

Here's where we see the models not having an understanding, but just spitting out madlibs as best they can. The prompter throws in a red herring in front of the statement which statistically matches a completely different kind of response. The LLM can't backtrack sufficiently far enough to ignore the non-relevant input and reroute to a response tree that just answers the second part. If we resort to a metaphor, however, these things are what, two? A two-year-old responding to the shiny jangling keys and not the dirty pacifier being removed for cleaning seems about on par.