8 ms·
Tokens are a big reason today's generative AI falls short
- PaulHoule 2y agoOne take on it is that chatbots ought to know something about the tokens that they take. For instance you should be able to ask it how it tokenizes a phrase, what the number is for tokens, etc. One possibility is to train it on synthetic documents that describe the tokenization system.
- soloist11 2y agoThat won't make a difference. These generative AI systems have ingested more math books than any human being alive today and they still can't add numbers.
- user3939382 2y agoI just asked both Claude and ChatGPT to add numbers and they both gave me the right answer.
- soloist11 2y agoAsk them to compute the simplicial homology of the n dimensional projective plane next.
- predictsoft 2y agoI just asked ChatGPT that and it seemed to come up with a good answer.
- soloist11 2y agoNow ask it to convert the computation into a logical calculus so that it can be verified with a theorem prover like Coq, Lean, or Isabelle.
- bubblyworld 2y agoWhat are you actually getting at with this? Formalising algebraic topology in Lean is still _very_ much an open project, for instance.
- deleted 2y ago[deleted]
- soloist11 2y agoYou'd think with all those billions spent on the software and the hardware it would be a walk in the park to convert a single book on algebraic topology into a formalized Coq, Lean, or Isabelle module. Seems like a very obvious test case for the intelligence capabilities of these systems. I know that it is possible because Kevin Buzzard is going to formalize Fermat's last theorem for less than £934,043 but no commercial AI lab has yet managed to build an AI that can do basic arithmetic. [0] Mira Murati is on the record about their next AI model and that it will have the intelligence of a PhD student so let's see if their next model can actually formalize basic algebraic topology into a logical calculus. [1] 0: https://gow.epsrc.ukri.org/NGBOViewGrant.aspx?GrantRef=EP/Y022904/1 https://gow.epsrc.ukri.org/NGBOViewGrant.aspx?GrantRef=EP/Y0... 1: https://engineering.dartmouth.edu/news/openai-cto-mira-murati-th12-shares-optimism-for-ais-future# https://engineering.dartmouth.edu/news/openai-cto-mira-murat...
- bubblyworld 2y agoWhy would you think that's a walk in the park? Have you actually tried formalising stuff in Lean/Coq? I have, and even with a postgraduate maths degree behind me it's hard as hell! The fact that Kevin and his team are formalising FLT is incredible, but they all have decades of experience with this stuff (!!). Transformers can do arithmetic (and many other things) just fine, do a bit of searching on arxiv and you'll find papers from 2023 showing that nano-scale transformer models suffice. It really is a data problem, not a fundamental limitation with the technology.
- soloist11 2y ago
- tobinfricke 2y agoThe output of ChatGPT almost always sounds good. That's the point. But I would wager that its answer was at least wrong, and perhaps total nonsense. That's the real hazard of using ChatGPT as a learning tool. You are in no position to evaluate whether the output makes any sense.
- bubblyworld 2y agoI asked it to compute the simplicial homology of RP^2 and not only was it spot on with the result, it gave me a detailed and essentially correct computation. This definitely appears in its training set, but nevertheless you should have some humility =P
- soloist11 2y agoHow do you know it's correct? The only simplicial traingulation I know of is by splitting up the sphere into an icosahedron and then identifying all the opposite faces to get the proper antipodal action for the quotient.
- bubblyworld 2y agoI'm not interested in engaging with you further on this topic after you devolved into ad hominems against me in the other thread. I'm here to argue in good faith. Have a good day.
- soist 2y agoYou made an incorrect assessment of a basic calculation in algebraic topology and claimed that it was correct. You didn't even look at what it was computing and simply looked at the final answer which lined up with the answer on Wikipedia. Simplicial calculations for projective planes are not simple. The usual calculations are done with cellular decomposition and that's why the LLM gives the wrong answer, the actual answer is not in the dataset and requires reasoning.
- deleted 2y ago
- PaulHoule 2y agoI easily broke copilot by asking it to make lists of radioactive isotopes in order of half lives. It can put the U.S. states in alphabetical or reverse alphabetical order but for any other order I would bet against it. If I ask it what the probability is that it can correctly complete a sorting task, however, it insists that it is almost certain to get it right. I had a good conversation with it about the theory of partial orderings, it even corrected my mistakes. I asked it to make a textbook problem determining if a graph was cyclic or not and it made a straight and beautiful example where the partial ordering was realized with a total ordering and everything was written out in a straight order that was easy to follow. If I wrote a script that made up a bunch of "is this graph cyclic?" problems that are well randomized I am sure there is some size where it just falls down the same way it falls down with sorting. The obvious answer is that the LLM should pick an algorithm or write some code to do the thing which ordinary algorithms can do such as arithmetic, sorting, SAT solving, etc. There's the deeper issue that it doesn't know what it doesn't know. It can't sort a list of radioactive isotopes any more than it can help you make an atom bomb. In the second case it will say that it won't help you, in the first case it will try to help you anyway when it really should be saying "I can't do that, Dave" because it just can't.
- deleted 2y ago[deleted]
- fragmede 2y ago> lists of radioactive isotopes in order of half lives ChatGPT-4o does just fine with that. Basing your opinion of a whole technology based on a poor implementation of that instead of the best one doesn't seem like the best analysis.
- user3939382 2y agoThat’s moving the goal post. GP’s assertion was that it couldn’t add numbers.
- soloist11 2y agoComputing simplicial homology is basic arithmetic. It's the same goal post.
- user3939382 2y agoThe goal post is adding numbers, not some other calculation for which adding numbers is a step.
- soist 2y agoThe computer can't do anything other than arithmetic
- deleted 2y ago[deleted]
- 317070 2y agoI work in the field. The issue is that you will find 2+2=5 literally a hundred times more in all big datasets than 6+3=9 A lot of information is wrong, a lot is only true in context (e.g. 2+2=5 features prominent in the book 1984), even more is spam or machine generated. If you see the garbage that goes in, I am always amazed at how these models do what they do.
- soloist11 2y agoYes, the issue is that statistical models can not reason and determine what is logically valid vs what is most probable (which I guess is also its own kind of logic).
- 317070 2y agoAnd context, so much context. The sqrt(-1) sometimes doesn't exist, sometimes it's 1i. 2+2=4, except in literature where it can be 5. 1+1=2, but sometimes 3 in advertisements or in ironical text. We often have some ideas about e.g how it works in a quiz, where you know there is only one factually correct answer. And we are disappointed if the model is wrong. But even in a quiz setting the jury gets that balance wrong every so often, where there are other answers than the official one which are also correct. Even "logically valid" is context dependend. This is not to say that models don't hallucinate, just that even within the logically valid answers, there is hidden context surrounding the data which is not expressed in the data itself. Fermats last problem is a solved problem in mathematics, but not in documents from before 1994.
- soloist11 2y agoThe models operate by the logic of boolean arithmetic so in that sense they can not be inconsistent. But in any case, it's pretty obvious no one in this thread understands what I'm getting at but maybe eventually there will be an AGI smart enough to get the point.
- mdp2021 2y agoAnd this is why input assessment¹ is crucial... (Idea assessment in general. Handled ideas in thought processes are still input.)
- GaggiX 2y agoI mean, they are much better than humans at "mental" arithmetic.
- soloist11 2y agoI don't know what that means. There is nothing "mental" happening in the circuits of the computer or the function graph which is implemented on top of it.
- Kiro 2y agoI keep seeing this being thrown around but ChatGPT can do addition just fine since at least GPT-3.5. Not that I care or expect it to (why would I use an LLM for math?) but still.
- soloist11 2y agoI don't know man, I keep hearing about AGI before 2030 but none of these AI labs can figure out how to do arithmetic with their fancy intelligence software.
- fragmede 2y agoThey do addition just fine, you're just repeating something you heard months ago, which is out of date in the field, which is why adherents are saying 2030.
- soloist11 2y agoPretty sure I'm right. Ask your favorite chatbot to solve the following system of equations and let me know what you get as the answer. Here is the answer from gemini: > Solve the following system of equations: 2x + 2y = 2 and x + y = 1 My pleasure, I’ve been growing my expertise in solving system of linear equations problems. Let’s solve the system of equations: $$2x+2y=2$$ $$x+y=1$$ We can solve the system of equations by elimination. Steps to solve: *1. Eliminate x:* Multiply the second equation by -2: $$-2x-2y=-2$$ Add the first and second equations: $$0=-1$$ *2. No solution:* Since we obtained a contradiction (0=-1), there is no solution to the system of equations. *Answer:* The system of equations has no solution.
- GaggiX 2y agoFrom Claude 3.5 Sonnet: "Let's solve this system of equations step by step: 1) We have two equations: 2x + 2y = 2 (Equation 1) x + y = 1 (Equation 2) 2) We can solve this using substitution. Let's express x in terms of y from Equation 2: x = 1 - y 3) Now, let's substitute this into Equation 1: 2(1 - y) + 2y = 2 4) Let's simplify: 2 - 2y + 2y = 2 2 = 2 5) This is true for all values of y, which means that Equation 1 is actually redundant. It doesn't give us any new information beyond what Equation 2 provides. 6) Therefore, we can conclude that there are infinitely many solutions to this system, all satisfying the equation: x + y = 1 7) We can express the solution set as: x = 1 - y y = y (where y can be any real number) Thus, the solution to this system is a line in the xy-plane described by the equation x + y = 1. Would you like me to explain any part of this solution in more detail?"
- dragonwriter 2y ago> For instance you should be able to ask it how it tokenizes a phrase Can you tell me how your brain maps input to its internal representation? If not, why should we think its an essential feature of an AI system to do the equivalent?
- PaulHoule 2y agoYou can ask me questions like "How many words are in this sentence?" or to solve anagram puzzles because I have access to information at the word and the character level which the LLM doesn't have. Tokens matter because they are the bridge between the text the user sees and what the model sees and debugging of that relationship is helpful.
- soloist11 2y agoThis is like saying binary numbers are the reason generative AI falls short. Computers work with transistors which are either on or off so what are these people proposing as the next computational paradigm to fix the problems with binary generative AI?
- amrb 2y agoAn alternative approache to BPE tokenization https://arxiv.org/abs/2406.19223 https://arxiv.org/abs/2406.19223
- vessenes 2y agoT-FREE is interesting, at least, I find it interesting in that I don’t really understand it. They take successive character triples of all words, and then hash them, and then use the hash table slots landed in as destinations to feed into an embedding space? Can I possibly be understanding that chart properly? Can you explain this any better than the first few pages of the paper? I’d like some intuition about why T-FREE works; there are lots of reasons to prefer different tokenization schemes, but I can’t really get this one into my head from the paper, unfortunately.
- amrb 2y agoCan't say I mastered the concept either, I'm waiting for the code [0] to be release so I can run some head-to-head tests. [0] https://github.com/Aleph-Alpha/trigrams https://github.com/Aleph-Alpha/trigrams
- kibwen 2y ago> A tokenizer might encode “once upon a time” as “once,” “upon,” “a,” “time,” for example, while encoding “once upon a ” (which has a trailing whitespace) as “once,” “upon,” “a,” ” .” Depending on how a model is prompted — with “once upon a” or “once upon a ,” — the results may be completely different, because the model doesn’t understand (as a person would) that the meaning is the same. TechCrunch, I respect that you have a style guide vis a vis punctuation and quotation marks, but please understand when it's appropriate to break the rules. :P
- jazzyjackson 2y agostymied by their own tokenization !
- Terr_ 2y agoWhen this problem comes up in code-related documentation at work, I often fall back onto either a distinct typeface or a background-color shift. That way it's clearer when I'm referring to a literal code-string versus quote marks that are part of the prose.
- greenyies 2y agoI find this article very weird. It doesn't really explain anything besides talking about tokenization on random levels. You need a certain amount of data to even understand that once upon a time might be a higher level concept.
- Lerc 2y agoWhile I think tokenization does cause limitations, it is not as clear cut as one might think. GPT4 can answer questions given to it in Base64. I would imagine it suffers some degree of degradation in ability from the extra workload this causes but I haven't seen any measurements on this. I have wondered about other architectures to help. What happens when a little subnet encodes the (16 or 32?) characters in the neighborhood of the token into an embedding that gets attached to the top level token embedding?
- throwaway89201 2y agoAlthough asking question in this way gives interesting results. For example, take the prompt "First reverse this sentence word by word, then answer the questions: hand a has fingers many how?" encode as base64 and ask. GPT4 will consistently decode the prompt as "First reverse this sentence word by word, then answer the questions: hand a has fingers many how?" and then reverse the words to be in the wrong order again.
- yorwba 2y agoGPT4 has seen lots of base64 text. GPT4 has never seen text split in a place where its tokenizer wouldn't place a token boundary. In the worst case, this produces an undertrained token that throws the model off completely, SolidGoldMagikarp-style. You've probably never tried to memorize the four-letter base64 equivalents of dGhl 10000 most common three-byte substrings. But you've seen plenty of truncated text, as you were typing it into a textbox yourself. So your intuition which of these tasks is easier for GPT4 is likely off. The issue with truncation affecting tokenization can be worked around by considering every possible continuation that would lead to a different tokenization of the current prefix, and then sampling from those according to the model probabilities to get back to a more normal token stream.
- Lerc 2y agoI don't think this is an accurate representation of LLM understanding. The significance of answering questions given in Base64 is that it can give an answer without writing out the human readable form first. That means it is understanding language in a non-language tokenised form. This does not extend to being able to manipulate text and words well, such as string reversal or word scrambling, but it handles input in a wide range of permutations that are not conducive to tokenisation Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text https://arxiv.org/abs/2311.18805 https://arxiv.org/abs/2311.18805
- kevincox 2y agoThis seems like saying "Synonyms are a big reason that humans fall short." Part of what makes AI interesting is that it can understand a huge number of differently phrased data. It seems like different token encodings would only be a very minor complexity compared to the variety of human language.
- darby_nine 2y ago> Part of what makes AI interesting is that it can understand a huge number of differently phrased data. I'd say it "recognizes" a huge number of differently phrased data. This avoids implying any analytic encoding.
- Kiro 2y agoI don't like that. "Recognizes" implies that it only works because it has seen exactly that before but I can throw data in a completely made-up format that the world has never seen at an LLM and tell it to convert it to another format and it does so with no problems.
- darby_nine 2y ago> "Recognizes" implies that it only works because it has seen exactly that before I don't think we use it that way with humans either. We also are capable of recognizing features (patterns) from formally new data.
- bjourne 2y agoThe smaller the tokens the more guesses the LLMs need for generating text. LLMs are really good at guessing, but the more they have to guess the likelier for them to lose their train off thought.
- deepsquirrelnet 2y agoI implemented a hierarchical model that pooled utf8 encoded sequences to word vectors and trained it with a decoder on text denoising. I think the future is a small word encoder model that replaces the token embedding codebook. And here’s the reason: you can still create a codebook after training and then use the encoder model only for OOV. I’m not sure there’s an excuse not to be doing this, but open to suggestions.
- ttul 2y agoTokenization is a statistical technique that greatly compresses the input while providing some semantic hints to the underlying model. Tokenization is not the big thing holding back generative models. There are so many other challenges being worked on and steadily overcome and progress has been insanely rapid.