4 ms·

Numerical Precision Affects Mathematical Reasoning Capabilities of LLMs

• hggigg 2y ago
  Isn't reasoning a little strong word? I mean finding evidence of any reasoning is the first point and I haven't seen anything that points to that. (note that I mean structured reasoning to solve new problems here, not just the correct answer and steps for existing problems).
  • ben_w 2y ago
    Depends on which definition of "reasoning" you prefer. If you need reasoning to be a "conscious" process, then we've got another problem because there's about 40* different meanings of that, too. * Wikipedia cites: Vimal RL, Sansthana DA (2010). "On the Quest of Defining Consciousness" (PDF). Mind and Matter. 8 (1): 93–121 but then gives a dead link, so I couldn't check it
  • ziofill 2y ago
    I agree. Reasoning should be robust to perturbations of the way in which the same problem is cast, but LLMs have problems with that.
    • krisoft 2y ago
      Does human reasoning survive that test? My reasoning for example is roboust to some extent but you can easily confuse me with appropriately (or innapriately) worded questions. Not even talking about if you grab me by the ankle and swing me around with 10g acceleration i won’t even be able to answer the simplest questions. So clearly my reasoning is highly sensitive to certain perturbations.
      • d4mi3n 2y ago
        Not disagreeing with you, but a point to consider for reasoning is that we consider it a key cornerstone of sapience and sentience. I think one could argue that a LLM could be considered sapient (able to solve problems) in a general sense, but probably not sentient (able to sustain undifferentiated consciousness, continue to absorb and apply information, etc). Part of the difficulty of these conversations, though, is many of these definitions were made before our math was advanced enough to approximate any of these criteria. We humans have also been notorious for excluding other things that could be considered sapience (e.g. elephants, dolphins, corvids, etc). In all cases, I think this is going to be a difficult topic to find consensus on until we better understand how our biology results in the emergent properties of our own sapience.
      • ziofill 2y ago
        I see what you are saying. I guess it all depends on how complex one makes that test. But arguably vanilla LLMs (0-shot, no CoT, no rewrite memory) are much worse than humans at this.
  • swatcoder 2y ago
    The field of AI likes to borrow pop psychology/philosophy language and use it for its own narrow jargon. It's been doing that for almost a century now. That's how it builds enthusiasm and raises funds, and also how it shoots itself in the foot and invites the AI winters that hollow it out again. The field named itself that very way to in response to a early 20th century boom of interest (money) in "intelligence" as something quantifiable (and therefore potentially computable). Tons of money was flying around based on the idea that people could be quantitatively measured for work-suitability, and researchers in computation saw opportunity to get research funds to explore that in their own budding endeavor, so they adopted the language and made a very long bet that they might eventually figure it out. At this point, there is largely consensus within the field that current models perform a process that the the field has collectively decided to call "reasoning" as part of their tradition of repurposing psychology and philosophy language as jargon. There's no putting that cat back in the bag. It's a term of art within the field now. Frustratingly, it's easier to just accept that this is just a novel sense of the word than to try to reconcile it with senses that we might be familiar with from lay usage or from other fields.
    • d4mi3n 2y ago
      I find it apropos that this topic (disagreement about if LLMs can "reason") boils down to being an issue of one of the hard problems in computer science: naming things.
      • ben_w 2y ago
        And the linked topic, off by one errors. One could also the hallucinated citations are null pointers. At least cache invalidation is easier: the users can backtrack up a conversation as far as they need to in order to un-cache things from the context window.
    • astrange 2y ago
      > The field of AI likes to borrow pop psychology/philosophy language and use it for its own narrow jargon. They also like to invent metaphors for how the brain works, and then forget that it's a metaphor and decide that humans either literally have one, that they should program one into AIs, or that AIs must have invented one on their own. This is why you keep seeing them talk about "world models", which probably don't actually exist and which people don't actually need in order to live their life.
  • krisoft 2y ago
    > finding evidence of any reasoning is the first point Happy to oblige. Oxford Languages dictionary defines “reasoning” as “the action of thinking about something in a logical, sensible way.” This was my prompt to chatgpt: “Hey chatgpt! On hacker news user hggigg said this: Isn't reasoning a little strong word? I mean finding evidence of any reasoning is the first point and I haven't seen anything that points to that. Based on this comment what do you think is the likely attitude of hggigg about large language models? (Answer with a single sentence)” This was the response: “hggigg likely expresses skepticism about large language models' ability to genuinely reason.” That sounds like reasoning to me. It was an action of thinking about something in a logical, sensible way. The something (a snippet of your comment) was novel. I don’t know what you mean by “structured”. Does it provide you evidence of reasoning for you? (If not, what would?)
    • swatcoder 2y ago
      > It was an action of thinking about something in a logical, sensible way If only it was so simple! Empirically, you streamed a short text into a black box system and received a short text out. The output text was related and coherent, and -- after the fact -- you assessed it to be logically, sensibly applicable given the input. On your way to using that to exemplify "reasoning" you took for granted that there was "thinking", and that the "thinking" was both "logical" and "sensible". Any number of systems might deliver the empirical result you measured, and many of them would involve none of that. LLM's are sophisticated and highly capable, but the discussion about what language to use for what they do isn't nearly so simple as you suggest.
      • wbl 2y ago
        And how do I know you think?
        • deleted 2y ago
          [deleted]
        • swatcoder 2y ago
          Are you sure you do? You're just responding to a block of text on an internet forum with trivial signup flow, known to be populated by bots of greater or less sophistication, as everywhere else on the internet.
• thrance 2y ago
  By "Mathematical Reasoning Capabilities" they meant "basic arithmetic" and by "Numerical Precision" they meant "quantization".
  • Vetch 2y ago
    That's not quite right. By numerical precision they mean numerical precision, of which quantization is one method to arrive at reduced precision. They also perform experiments where they train from scratch for float32 and float16. Reasoning, they should have replaced with: iterative computations with accumulating state. This paper, on the impact of quantization, is actually a lot more significant than it appears at first and I think the authors could have done a better job of discussing the broader implications. The paper's core (and unsurprising) argument is that low-precision arithmetic significantly limits the representational capacity of individual neurons. This forces the model to encode numerical values across multiple neurons to avoid overflow, particularly when storing intermediate computational results. This distributed representation, in turn, increases complexity and makes the model vulnerable to accumulating errors, particularly during iterative operations. It's unclear from my initial reading whether low-precision training (quantization-aware training) is necessary for the model to effectively learn and utilize these distributed representations, or if this capacity is inherent. Regardless, while QAT likely offers benefits, especially with larger numbers, the fundamental limitations of low precision computation persist. Why not just use a calculator? For some of the same reason humans shouldn't be completely dependent on calculators. It's not just the ability to perform fermi estimates that's constrained but internal computations that require physical "intuition" or modeling the trajectories of physical systems, the ability to work with growing algebraic representations, relative numeric comparisons of large magnitudes (where the model does not internally switch to a favorable format--notice this is easier to do in-context), and representing iterative computation on complex logical chains and hierarchical structures. Why do we not see this in practice? I contend that we do. There is a small but quite vocal contingent in every LLM forum who insist that quantization, even 8-bits, results in severe degradation in quality despite what most benchmarks say. It's quite likely that common tasks and most tests do not require iterative computations where accumulating state representations must be accurately tracked and these individuals are encountering some of the exceptions.
• alexvitkov 2y ago
  I wonder if we'll get better performance on arithmetic tasks if we let LLMs generate digits backwards, e.g. 12345+11111=<rev>65432</rev> where the rev tags are special tokens. The reason being that there's less context needed to do it this way, for addition at every step there's only 3 digits that need to be considered and they're already in the token stream.
  • astrange 2y ago
    Since tokens are numbers, I'd like to see the numbers simply tokenized as themselves, or maybe an exponential Golomb code.
  • thesz 2y ago
    Most probably, something like 65432 is just one token.
    • alexvitkov 2y ago
      It's not, it wouldn't make sense to have 100,000 tokens just for the first 100,000 numbers. There's a playground [1] where you can see how LLMs tokenize a string. 12345678987654321 is tokenized on various models like so: GPT4 123-456-789-876-543-21 GPT3 123-45-678-98-765-43-21 Llama-2, Mistral 1-2-3-4-5-6-7-8-9-8-7-6-5-4-3-2-1 [1] https://huggingface.co/spaces/Xenova/the-tokenizer-playground https://huggingface.co/spaces/Xenova/the-tokenizer-playgroun...
      • Jerrrrrrry 2y ago
        Looks like number string parsing may be of enough importance to warrant token look-ahead recursive sub-parsing, then use the most "promising" token-ization; the one that generates the highest yielding probability tree following that number string.
  • Vetch 2y ago
    It does and the paper mentions some papers that investigate similar strategies in its appendix, but the fundamental precision issues do not go away.
• magicalhippo 2y ago
  Just been dabbling with local models, and while the several models I've tried generates decent sentences while quantized, they suffered heavily in following instructions and picking up details. So a larger model but fairly aggressively quantized could perform worse than a smaller variant of the model with just light quantization, even though the larger still used more memory in total. I guess some of this is due to the models not being trained to the quantization levels I used. In any case, I say don't get blended by parameter count alone, compare performances.
• Animats 2y ago
  The empirical results seem legit. Not sure about all those "theorems". Those are claims of function existence, not a demonstration that transformers actually work that way. Comments?
• amelius 2y ago
  One thing I was wondering about regarding transformers, which perhaps someone more knowledgeable can explain: as far as I understand, the attention heads are essentially two-dimensional structures where values related to tokens are compared to each other in a matrix. Has anyone tried to generalize this and make the dimension of the attention-heads higher than two?
  • deleted 2y ago
    [deleted]
  • svachalek 2y ago
    I'm not really in this space but I like to read the papers and as I understand it, the typical dimensionality is far higher than 2. For example in the original "All you need is attention" paper, the example they give has 64 dimensions. They're vectors so even though they might be drawn as a matrix, each value represents a distance in a different dimension.
    • amelius 2y ago
      I'm talking about the matrices W_Q, W_K and W_V. My question is why these are matrices (tensors of dimension 2) and not tensors of a higher dimension than 2. My thinking goes like: a matrix can represent a graph (each entry may correspond to an edge between two nodes), but e.g. a 3-dimensional tensor may correspond to a hypergraph where each entry is a 3-hyperedge, so you can not just talk about the relation between two tokens, but also about the relation between three tokens (in language this could be e.g. subject, object and indirect-object/dative).
• AIFounder 2y ago
  [dead]