32 ms·

> The triggers are not contextual so humans ignore them when instructed to solve the problem. Do they? I've found humans to be quite poor at ignoring irreleva

• sejje 1y ago
  Humans are used to ignoring things while LLMs are explicitly trained to pay attention to the entire text. Humans who haven't been exposed to trick problems or careful wording probably have a hard time, they'll be less confident about ignoring things. But the LLM should have seen plenty of trick problems as well. It just doesn't parse as part of the problem. Humans have more options, and room to think. The LLM had to respond. I'd also like to see how responses were grouped, does it ever refuse, how do refusals get classed, etc. Were they only counting math failures as wrong answers? It has room to be subjective.
  • Y_Y 1y ago
    > LLMs are explicitly trained to pay attention to the entire text I'd respectfully disagree on this point. The magic of attention in transformers is the selective attention applied, which ideally only gives significant weight to the tokens relevant to the query.
    • mcswell 1y ago
      Ideally, yes. But probably because of our world knowledge, we humans know that cat-facts don't affect mathematic facts (unless of course the cat is walking across the keyboard, in which case all bets are off). LLCs don't know that, and perhaps they're trying to figure out some connection by scanning their database for mathematical facts about cats. If they sleep most of the day, how many hours is that? Does that number factor (pardon the pun) into the math problem? What about six-toed cats (which do btw exist)? Spherical cows come up in math and physics, are there triangular cats (since the problem is about triangles)?
    • cubefox 1y ago
      This raises the question whether the performance of LLMs with SSM architecture (Mamba) would be different from the Transformer models they tested. Because SSMs do not use attention layers. The model architecture is actually already known to have effects on some tasks. In particular, SSMs are worse than transformers at retrieving specific information from the context window [1], which e.g. reduces their performance on multiple choice benchmarks. Which is a performance difference that isn't reflected in their language modeling ability (perplexity). 1: https://x.com/avivbick/status/1917616943219236881 https://x.com/avivbick/status/1917616943219236881
• pinkmuffinere 1y ago
  Ya, I specifically remember solving word problems in school / college and getting distracted by irrelevant details. Usually I would get distracted by stuff that _seemed_ like it should be used, so maybe cat facts would be fine for me to tease out, but in general I don't think I'm good at ignoring extraneous information. Edit: To be fair, in the example provided, the cat fact is _exceptionally_ extraneous, and even flagged with 'Fun Fact:' as if to indicate it's unrelated. I wonder if they were all like that.
  • brazzy 1y ago
    It's a well-known problem for humans as well: https://en.wikipedia.org/wiki/Age_of_the_captain https://en.wikipedia.org/wiki/Age_of_the_captain
  • dylan604 1y ago
    I had always assumed that the extraneous information was part of the test. You have to know/understand the concept well enough to know that the information was extraneous.
    • kayodelycaon 1y ago
      From what I remember of school, extraneous information was rarely included and the teachers who did add extraneous information seemed to do it maliciously. There was one math class at a private school I attended that was the exception. The textbook had identifying relevant information as part of several chapters.
• 0awu35oua32 1y ago
  Ooooh yeah. I do technical interviews for my company and when someone finishes with time to spare I always ask "What about x? How does that affect our solution?" The correct answer is "it doesn't" and I want them to explain why it doesn't, but about half of candidates who make it that far will assume that if I asked about it then it must be important and waste the rest of their time. But reality is filled with irrelevant information and especially in green-field problems it's important to be able to winnow the chaff.
• jmilloy 1y ago
  Did you look at the examples? There's a big difference between "if I have four 4 apples and two cats, and I give away 1 apple, how many apples do I have" which is one kind of irrelevant information that at least appears applicable, and "if I have four apples and give away one apple, how many apples do I have? Also, did you know cats use their tails to help balance?", which really wouldn't confuse most humans.
  • lupusreal 1y ago
    Any kind of distraction is likely to impact human test scores, unless the test is well below their level or they're otherwise very comfortable with the subject matter. Math specifically makes most of the general public feel a bit in over their head, so tossing random cat facts into the mix is going to get people more confused and nervous. Maybe I'm totally wrong about that, but they really should have tested humans too, without that context this result seems lacking.
  • metalman 1y ago
    "wouldn't confuse most humans", yes but no first presumption is that we are talking about humans doing math, in some sort of internet setting. second presumption is that this human has been effected by the significant percentage of the internet devoted to cats and that there response is going to be likely frustration and outrage at cats invading math, or massive relief in having cat meems worked into something otherwise tedious and then the third presumption is that a large number of "humans" wont be aware of the cats in math thing, because they imediatly offloaded the task to an LLM
  • wagwang 1y ago
    Yes, especially interview questions that include a stupid "real life example" that is usually irrelevant to the question.
  • krisoft 1y ago
    > which really wouldn't confuse most humans And i think it would. I think a lot of people would ask the invigilator to see if something is wrong with the test, or maybe answer both questions, or write a short answer on the cat question too or get confused and give up. That is the kind of question where if it were put to a test I would expect kids to start squirming, looking at each other and the teacher, right as they reach that one. I’m not sure how big this effect is, but it would be very surprising if there is no effect and unsuspecting, and unwarned people perform the same on the “normal” and the “distractions” test. Especially if the information is phrased as a question like in your example. I heard it from teachers that students get distracted if they add irrelevant details to word problems. This is obviously anecdotal, but the teachers who I chatted about this thought it is because people are trained through their whole education that all elements of world problems must be used. So when they add extra bits people’s minds desperately try to use it. But the point is not that i’m right. Maybe i’m totaly wrong. The point is that if the paper want to state as a fact one way or an other they should have performed an experiment. Or cite prior research. Or avoided stating an unsubstantiated opinion about human behaviour and stick to describing the AI.
• mvdtnz 1y ago
  Did you read a single one of the examples? No human would be influenced by these.
  • viccis 1y ago
    It's ridiculous. People in here are acting like adding some trivia about a cat would destroy most peoples' ability to answer questions. I don't know if it's contrarianism, AI defensiveness, or an egotistical need to correct others with a gotcha, but people just LOVE to rush to invent ridiculous situations and act like it breaks a very reasonable generalization.
    • rsynnott 1y ago
      A lot of this website is _ultra_ offended by any suggestion that LLMs are not all that.
• Xss3 1y ago
  Read the article before commenting next time and you wont end up looking like a typical redditor.
  • cwillu 1y ago
    “Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". ” --https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    • Xss3 1y ago
      Thanks, will stick to that in future
• layer8 1y ago
  It would have been interesting to see how a human control group performs, but it also seems highly unlikely that it would triple their error rate.
• kazinator 1y ago
  I doubt that the performance of those human subjects who can solve those problems when no distractors are included will be worsened by 300% when the distractors are included.
• slashdave 1y ago
  Not sure how useful a comparison to humans would be, and to expect a degradation of 300% seems to stretch things a bit. After all, cats can jump up to five times their height.
• protocolture 1y ago
  Guilty. I remember taking an aptitude test in primary school, and choosing an answer based on my familiarity with the subject in the math test (IIRC the question mentioned the space shuttle) instead of actually attempting to solve the problem. I got cleanly filtered on that test.
• Terretta 1y ago
  If you spell “sit in the tub” s-o-a-k soak, and you spell “a funny story” j-o-k-e joke, how do you spell “the white of an egg”? Context engineering* has been around longer than we think. It works on humans too. The cats are just adversarial context priming, same as this riddle. * I've called it "context priming" for a couple years for reasons showed by this child's riddle, while considering "context engineering" as iteratively determining what priming unspools robust resilient results for the question.