6 ms·

The main limitation of tokenization is actually logical operations, including arithmetic. IIRC most of the poor performance of LLMs for math problems can be att

• calibas 1y ago
  It's a non-deterministic language model, shouldn't we expect mediocre performance in math? It seems like the wrong tool for the job...
  • drdeca 1y ago
    Deterministic is a special case of not-necessarily-deterministic.
  • CamperBob2 1y ago
    We passed 'mediocre' a long time ago, but yes, it would be surprising if the same vocabulary representation is optimal for both verbal language and mathematical reasoning and computing. To the extent we've already found that to be the case, it's perhaps the weirdest part of this whole "paradigm shift."
  • rictic 1y ago
    Models are deterministic, they're a mathematical function from sequences of tokens to probability distributions over the next token. Then a system samples from that distribution, typically with randomness, and there are some optimizations in running them that introduce randomness, but it's important to understand that the models themselves are not random.
    • mgraczyk 1y ago
      This is only ideally true. From the perspective of the user of a large closed LLM, this isn't quite right because of non-associativity, experiments, unversioned changes, etc. It's best to assume that the relationship between input and output of an LLM is not deterministic, similar to something like using a Google search API.
      • ijk 1y ago
        And even on open LLMs, GPU instability can cause non-determinism. For performance reasons, determinism is seldom guaranteed in LLMs in general.
        • rar00 1y ago
          yep, even with greedy sampling and fixed system state, numerical instability is sufficient to make output sequences diverge when processing the same exact input
    • geysersam 1y ago
      The LLMs are deterministic but they only return a probability distribution over following tokens. The tokens the user sees in the response are selected by some typically stochastic sampling procedure.
      • danielmarkbruce 1y ago
        Assuming decent data, it won't be stochastic sampling for many math operations/input combinations. When people suggest LLMs with tokenization could learn math, they aren't suggesting a small undertrained model trained on crappy data.
        • anonymoushn 1y ago
          I mean, this depends on your sampler. With temp=1 and sampling from the raw output distribution, setting aside numerics issues, these models output nonzero probability of every token at each position
          • danielmarkbruce 1y ago
            A large model well trained on good data will have logits so negative for something like "1+1=" -> 3 that they won't come up in practice unless you sample in a way to deliberately misuse the model.
  • currymj 1y ago
    thanks to training data + this being a popular benchmark, they're pretty good at grinding through symbolic mathematical derivations, which is often useful if you want an explanation of a mathematical concept. there's not really a better tool for this job, except for "a textbook which answers the exact question you have". but from time to time, doing this does require doing arithmetic correctly (to correctly add two exponents or whatever). so it would be nice to be able to trust that. i imagine there are other uses for basic arithmetic too, QA applications over data that quotes statistics and such.
    • agarren 1y ago
      > but from time to time, doing this does require doing arithmetic correctly (to correctly add two exponents or whatever). so it would be nice to be able to trust that. It sounds weird, but try writing your problem in LaTeX - I don’t know why, I’ve found a couple models to be incredibly capable at solving mathematical problems if you write them in LaTeX.
• cschmidt 1y ago
  This paper has a good solution: https://arxiv.org/abs/2402.14903 https://arxiv.org/abs/2402.14903 You right to left tokenize in groups of 3, so 1234567 becomes 1 234 567 rather than the default 123 456 7. And if you ensure all 1-3 digits groups are in the vocab, it does much better. Both https://arxiv.org/abs/2503.13423 https://arxiv.org/abs/2503.13423 and https://arxiv.org/abs/2504.00178 https://arxiv.org/abs/2504.00178 (co-author) both independently noted that you can do this with just by modifying the pre-tokenization regex, without having to explicitly add commas.
  • jvanderbot 1y ago
    Ok great! This is precisely how I chunk numbers for comparison. And not to diminish a solid result or the usefulness of it or the baseline tech: its clear that it we keep having to create situation - specific inputs or processes, we're not at AGI with this baseline tech
    • chmod775 1y ago
      > [..] we're not at AGI with this baseline tech DAG architectures fundamentally cannot be AGI and you cannot even use them as a building block for a hypothetical AGI if they're immutable at runtime. Any time I hear the goal being "AGI" in the context of these LLMs, I feel like listening to a bunch of 18th-century aristocrats trying to get to the moon by growing trees. Try to create useful approximations using what you have or look for new approaches, but don't waste time on the impossible. There's no iterative improvements here that will get you to AGI.
      • kristjansson 1y ago
        > "So... what does the thinking?" > "You're not understanding, are you? The brain does the thinking. The meat." > "Thinking meat! You're asking me to believe in thinking meat!" https://www.mit.edu/people/dpolicar/writing/prose/text/thinkingMeat.html https://www.mit.edu/people/dpolicar/writing/prose/text/think...
      • mgraczyk 1y ago
• search_facility 1y ago
  regarding “math with tokens”: There was paper with tokenization that has specific tokens for int numbers, where token value = number. model learned to work with numbers as numbers and with tokens for everything else... it was good at math. can’t find a link, was on hugginface papers
  • samus 1y ago
    Shouldn't production models already do this? They already tend to use tokenizers with complex rules to deal with a lot of input that would otherwise be tokenized in a suboptimal way. I recall a bug in an inference engine (maybe llama.cpp?) because of an implementation difference in their regex engine compared to the model trainer. Which means that the tokenizer used regex-based rules to chop up the input.
    • search_facility 1y ago
      turns out - no, by intuition they should do this for sure - but no. UPD: Found the paper: - https://huggingface.co/papers/2502.09741 https://huggingface.co/papers/2502.09741 - https://fouriernumber.github.io/ https://fouriernumber.github.io/ in paper mentioned “number” is a single sort-of “token” with numeric value, so network dealing with numbers like real numbers, separately from char representation. All the math happens directly on “number value”. In majority of current models numbers are handled like sequences of chars
• js8 1y ago
  It's not strange at all. I am playing with lambda calculus and combinatory logic now, as a base for mathematics (my interest is to understand rigorous thinking). You can express any computation using just S and K combinators, however, there is a price to that - the computations will be rather slow. So to make the computation faster, we can use additional combinators and rules to speed things up (good example is clapp() function in https://github.com/tromp/AIT/blob/master/uni.c https://github.com/tromp/AIT/blob/master/uni.c). Of course, the extra rules have to be logically consistent with the base S and K combinators, otherwise you will get wrong result. But if the inconsistent rule is complicated enough to be used only infrequently, you will still get correct result most of the time. Which brings me to LLMs and transformers. I posit that transformers are essentially learned systems of rules that are applied to somewhat fuzzily known set of combinators (programs), each represented by a token (the term being represented by the embedding vector). However, the rules learned are not necessarily consistent (as it happens in the source data), so you get an occasional logical error (I don't want to call it hallucination because it's a different phenomenon from nondeterminism and extrapolation of LLMs). This explains the collapse from the famous paper: https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf https://ml-site.cdn-apple.com/papers/the-illusion-of-thinkin... One infrequent but inconsistent rule is enough to poison the well due to logical principle of explosion. It also clearly cannot be completely fixed with more training data. (There is also analogy to Terry Tao's stages of mathematical thinking: https://terrytao.wordpress.com/career-advice/theres-more-to-mathematics-than-rigour-and-proofs/ https://terrytao.wordpress.com/career-advice/theres-more-to-... Pre-rigorous corresponds to soomewhat random set of likely inconsistent logical rules, rigorous to small set of obviously consistent rules, like only S and K, and post-rigorous to a large set of rules that have been vetted for consistency.) What is the "solution" to this? Well, I think during training you somehow need to make sure that the transformer rules learned by the LLM are logically consistent for the strictly logical fragment of the human language that is relevant to logical and programming problems. Which is admittedly not an easy task (I doubt it's even possible within NN framework).
• vendiddy 1y ago
  Do LLMs need to be good at math with the same approach? To draw an a analogy, we've got our human brain specialized. Why not implement a part of the AI brain that's not neural nets, but instead circuitry specialized to math? Maybe a dumb question since I'm a layperson!
• williamdclt 1y ago
  Even if LLMs get better at arithmetic, they don't seem like the right tool for the job. LLMs might never be able to crunch numbers reliably, however I expect they should be very good at identifying the right formula and the inputs for a problem ("i need the answer to x*y, where x=12938762.3 and y=902832.2332"). Then they can call a math engine (calculator or wolfram alpha or whatever) to do the actual computation. That's what humans do anyway!