4 ms·
Can anyone give a layman's rundown on "it guesses the next token" in situations where it's seemingly able to apply what looks like logic to certain prompt reque
by sakopov 3y ago
Can anyone give a layman's rundown on "it guesses the next token" in situations where it's seemingly able to apply what looks like logic to certain prompt requests?
- tomohelix 3y agoThe way I understand it, our language is an abstraction to our logics. We think using it and we communicate our logic with words. Literally, how do you reason within your head? By forming sentences and argue with yourself using words? At least that is how I do it. I can't make any deep reasoning without first translating it to sentences. So an AI after learning so much text and forming an extremely dense network of connections between different words and phrases, it can mimic something similar to reasoning. At its core, it is still just predicting the next word. But the scale of this "prediction" is so large that it begins to mirror our own "reasoning" process. Because in the end, when we apply logic, we are doing it through our own network of concept connections which is reflected in our language.
- sakopov 3y agoI guess the author did make a complete explanation when they mentioned arrays of floating point numbers representing connections. I just thought there was more to it that was omitted. This seems like a relatively simple solution which simulates an fascinatingly complicated process when performed at scale. Thanks for a great explanation!
- visarga 3y agoIt is applying logic very often, but in an implicit way. How would it solve unseen math and code problems otherwise.
- mcluck 3y agoDoes it solve unseen math? I can consistently get it to trip over even simple math
- stevenhuang 3y agoHere's how I understand it. By nature of its architecture, it can only predict the next token. However, the architecture may be flexible enough that it may be able to uncover the underlying patterns within its training data in order to help it predict the next token. https://en.m.wikipedia.org/wiki/Universal_approximation_theorem#:~:text=Universal%20approximation%20theorems%20imply%20that,such%20a%20construction%20is%20possible https://en.m.wikipedia.org/wiki/Universal_approximation_theo.... Then suppose there exists a well defined algorithmic process for human level intelligence, of which some approximation to it appreciably resides in the training data. Then perhaps the LLM or networks like it, have started to learn aspects of this algorithmic process, then leverage it in order to successfully predict the next token. In other words, by forcing it to predict the next token, but giving it the flexibility to do so in any way it deems helpful, LLMs have learned higher order concepts such as abstract reasoning and "general intelligence".
- merlincorey 3y ago> In other words, by forcing it to predict the next token, but giving it the flexibility to do so in any way it deems helpful, LLMs have learned higher order concepts such as abstract reasoning and "general intelligence". To my understanding there is not any evidence of Abstract Reasoning or "General Intelligence" in Large Language Models such as ChatGPT. For example, I found a website that defines and has abstract reasoning tests it says are currently "AI Resistant"[0]: > Resistance to AI tools (like ChatGPT) is checked each week. > Based on the last check, it's unlikely that AI tools will be able to help answer this question The test is a "Combining Figures" test where one sees a series of three shapes where the first two overlayed on top of each other create the final shape and a final series where one of the three shapes is missing and you have to figure out which of a set of possible shapes should go there. It's pretty straightforward to show that you and I can pass this test, but ChatGPT 3.5[1] has low chances: --- Please solve the following abstract reasoning puzzle. Given a numbered set of 1 by 3 tiles represented by uppercase O's for empty spaces and uppercase X's for filled spaces such as: 1. OXO 2. XOO 3. XXO And given a different numbered set of 1 by 3 tiles such as: 1. OOX 2. XOX 3. XOX Now given just the first two 1 by 3 tiles of a numbered set of three: 1. XXX 2. OXO Which of the following choices is the correct third tile following the previous sequence? a. OXO b. XOX c. OOO d. XXX Please explain your reasoning for how you arrived at this choice. --- In order to solve this abstract reasoning puzzle, we need to identify the pattern or rule that governs the transformation between the first two sets of 1 by 3 tiles. Let’s analyze the given sets: Set 1: 1. OXO 2. XOO 3. XXO Set 2: 1. OOX 2. XOX 3. XOX Comparing both sets, we can see that the transformation occurs horizontally across the three tiles. Let’s look at each column independently: 1st column: In both sets, the first tile is “O”, so no filling pattern can be determined. 2nd column: In both sets, the second tile alternates between “X” and “O”. 3rd column: In both sets, the third tile repeats the same filling pattern as the second tile. Now, let’s analyze the third and final set consisting of the tiles: 1. XXX 2. OXO Based on the pattern observed in sets 1 and 2, we can deduce that the third tile must repeat the same filling pattern as the second tile. Looking at the second tile of the third set, it is “OXO”. Therefore, the correct third tile to follow the pattern would be “OXO”. Therefore, the correct choice is: a. OXO --- [0] https://www.testdome.com/tests/abstract-reasoning-test/142 https://www.testdome.com/tests/abstract-reasoning-test/142 [1] https://huggingface.co/spaces/yuntian-deng/ChatGPT https://huggingface.co/spaces/yuntian-deng/ChatGPT or similar free ChatGPT 3.5 interface
- lucubratory 3y agoThe training process forces it to optimise predicting the next token, across an extremely large and diverse set of tokens. To do this, it has access to a very large and layered neural network, "the weights", after the particular property of this network that changes during training. We largely do not know what that neural network does in order to successfully predict the next token. We know how it learned in an obvious sense, it adjusts weights randomly on a pass and then evaluates if the loss is lower, preserving or rejecting weight adjustments as necessary using gradient descent. We know in a general sense "the neural net takes information from previous tokens and transforms them into a prediction for the next token". But we largely don't know how. This is where the controversy over whether it engages in reasoning or logic comes from; it clearly does, we can see it in how it responds to certain questions, but it is not obvious where in this process anything like a logic engine could fit in. Some convincing research has suggested that the neural network generates a world model ("models the world" might be easier to understand grammatically) based on the information in the input data (the internet), which it then uses during inference to evaluate questions, with different parts of the model activating for different tasks, like recalling a memorised fact ("When was the Grand Canyon named?"), or answering a logical puzzle like the ones with Alex and Bob. A hypothesis is that the process of taking a large neural net and forcing it to learn how to predict the next token on extremely large datasets with lots of compute gives it the ability to create "things" within itself, subnetworks or little logic engines or coarse emulations of various intelligent entities. After fine-tuning and all the other housekeeping, if you ask it a question it will figure out which of its "things" can answer that type of question, hand it to that "thing", and the "thing" uses both memorised information and other "things" to come up with an answer. If this hypothesis is true, there is no contradiction between "it predicts the next token" and "it reasons logically and relies on its memory to answer the questions it is asked". The full sentence would be "It reasons logically and relies on its memory in order to answer the questions it is asked, by predicting one token at a time." I have not seen another hypothesis that adequately explains to me how it is possible for the current crop of LLMs to correctly answer novel logic questions. A ~good challenge to this hypothesis is that there are still problems that LLMs can't solve, like correctly formatted novel Winograd schemas, and it's not clear why this hypothesis should work for the things it can do but not for them. My answer to that challenge would probably be: build a bigger system and see if that can solve those other problems too. Scale seems to be the overwhelmingly dominant factor in performance for these things.
- astrange 3y ago"Guessing the next token" is an interface, not an implementation. A transformer is a universal function approximator and is capable of learning the function "give the right answer" through this interface. (Also, if you do beam search, it's not "the next token" but "the next tokens".)