6 ms·
In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, borde
by famouswaffles 4y ago
In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you've trained other kinds of models before.
See here - https://imgur.com/a/w3DAYOi https://imgur.com/a/w3DAYOi
from the paper - https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066
GPT 3.5's (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it's done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after.
- Buttons840 4y agoBeing able to learn within context, without updating weights is amazing. Imagine how much more efficient and/or powerful it could be if we found a way to update the weights in real time.
- joshribakoff 4y agoMaybe even more powerful would be reducing the number of examples needed to learn, eg less than one shot https://www.technologyreview.com/2020/10/16/1010566/ai-machine-learning-with-tiny-data/ https://www.technologyreview.com/2020/10/16/1010566/ai-machi... updating weights in real time is useless if each update basically does nothing because it takes an insurmountable amount of training, on the other hand if i can give my model a succinct “lesson” i’d then be very willing to wait a while for it to “process”
- dTal 4y agoAs I understand, that's basically what fine-tuning is?
- monkeydust 4y agoWhat sort of performance difference can you expect from in context Vs fine tuning, any relevant papers on this ?
- vintermann 4y agoYeah, that's why just updating the weights on the models such as they are doesn't work. But they're right that it's desirable to have some sort of online learning, whether on top of a frozen language model, or through some not yet invented way to do it end to end.
- Enginerrrd 4y agoYeah I've been thinking about that a lot within the context of self-driving cars. What if we could take the rare but significant examples of model confusion, and then... explain to the model what it did wrong just as we would to a human. If it could then incorporate that information into future weights, that would be game changer in a LOT of fields where it's been hard to bridge the gap.
- jurgemaister 4y agoTesla does this with shadow testing[1] [1]https://www.forbes.com/sites/bradtempleton/2019/04/29/teslas-shadow-testing-offers-a-useful-advantage-on-the-biggest-problem-in-robocars/?sh=2acfc3bd3c06 https://www.forbes.com/sites/bradtempleton/2019/04/29/teslas...
- skybrian 4y agoI agree that it’s a neat demo, but it’s not all that useful in itself. You could also do this by writing a function in a programming language (if it weren’t built in), to arbitrary accuracy, and it doesn’t cost anything to run. A more practical thing to do for algorithms is probably to use ChatGPT to help you write the function you need.
- famouswaffles 4y agoThe significance of the paper is moreso the implications of how far ICL can take you rather than the ease/viability of the solution proposed. Sure there are better methods for arithmetic but arithmetic is extremely quantifiable with rigid steps. What happens when you step out of that kind of domain ? Like the above blog. or Code documentation. For example, you can paste new documentation in a gpt-4 instance and it will use it for your queries as if it trained on it. Basically Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589 https://arxiv.org/abs/2301.04589. and you kind of get the feeling of that from the previous paper.
- skybrian 4y agoYou've got a limited context window (for now). There's only so much you can put into a prompt, so how much you can teach it this way is going to be pretty limited. Whatever you teach it had better be the primary task you're using it for. You can't do it for everything, but if you can generate code and run it outside the LLM, you should.
- thebestgamers 4y ago[flagged]
- famouswaffles 4y agoThe limits of the context window become much less important (but can still be a problem I agree) when crucial context can be dynamically inserted only when relevant. Gpt-3.5 doesn't need the algorithm prompt for every single query. It just needs it for every query that requires arithmetic. Much more feasible.
- kloch 4y ago> you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after. Is the ~13 digit limitation due to the model itself (how/how well it was trained) or simply the use of double precision in the model weights (which maxes out around 15 digits of precision).
- kristjansson 4y agoAdditional evidence against the second hypothesis: almost nothing in LLM-land is double precision anyway, weights are generally half-precision (or something like bfloat16 with more range by less precision than IEEE float16).
- famouswaffles 4y agoIt's hard to say for sure but the second is pretty unlikely.
- tel 4y agoIn order for it to be the second you'd need to assume that at least some part of the critical reasoning of the LLM involves storing the data of the number in a single model activation. This is pretty unlikely, as models tend to store information across many activations simultaneously. I don't know this for a fact, you'd need to do brain surgery on GPT-4 and it'd be hard even in that case, but most studies of ANN processing would suggest that the null hypothesis is to assume that the information is widely distributed and not sigfig limited in that way.
- jychang 4y agoThere are exceptions https://www.lesswrong.com/posts/cgqh99SHsCv3jJYDS/we-found-an-neuron-in-gpt-2 https://www.lesswrong.com/posts/cgqh99SHsCv3jJYDS/we-found-a...
- tel 4y agoSort of. I hadn't read that one and just skimmed it, but I think it falls into the same category of "while we can find compelling activation patterns even at a single neuron level, it's hard to rule out other representations". On the flip side is Chris Olah's research (e.g. https://distill.pub/2017/feature-visualization/ https://distill.pub/2017/feature-visualization/) which maps a single neuron's activation into the output space (by optimizing for outputs which would have arising from high activations of given neurons). If we take this seriously and call these outputs the "favored" output of an individual neuron, we might believe we've characterized that neuron, but it's really still the entire downstream computation from that neuron and there are still substantial overlaps and redundancies. These are exactly where I'm getting the sense that the standard rule should be that neural nets have very distributed representations. Even more to the point, these analyses tend to focus on "binary" questions, either a feature is present or not. Or, said another way, they implicitly treat activation as a "vote", where greater activation implies a greater component weighting of that which that neuron represents. This, to my mind, flies in the face of GPT-n doing arithmetic, though. I'd lean much closer to the argument that each individual quantity is represented as a neuron as opposed to each quantity is represented continuously by the actual activation _value_. This "voting" representation is also reinforced by the common use of tanh-like activation functions. There's a long history of having neuron activations be probabilistic/logit-like quantities. Of course, I cannot speak for certain about any neural net and especially not for GPT-3/4. I just have a hard time taking as the null hypothesis that it is storing magnitudes of numbers in a way that is directly affected by the float precision in use.
- elhudy 4y agoIn-context learning also seems like the best path to commercializing LLMs. I'm surprised that Microsoft is going the D2C route with ChatGPT rather than commercializing it in a B2B fashion. ...Or maybe that's coming? Imagine feeding an LLM a ton of disparate data sources and asking it questions about that data as a whole. What is a data engineer again, anyway?
- tyingq 4y agoInteresting, though I imagine that will often play out with the business person rephrasing the question or filtering out data until it spits out the answer they expected :) That data engineer can at least push back and have their own opinion.
- zenlikethat 4y agoD2C makes a lot of sense imo because at this point it's most valuable to obtain a huge corpus of (reasonably well labeled) training data (with a side helping of being really great marketing). Just think about all the programmers who are probably rolling their eyes and correcting ChatGPT's work for OpenAI, or at least providing some details. I know it's a reflex for me, not even deliberate, to at least type something in like "Ah I figured out that it was XYZ" and thumbs up or thumbs down. Microsoft will undoubtedly be rolling this all out in Docs, Excel, Teams etc in no time. A good reason to be skeptical about the flood of "GPT for your Notion!" startups.
- z3c0 4y agoIt's still not an extractive tool (it's generative), so it stating logical facts based on the data is not something one should bank on. It will likely never be capable of what you're suggesting, without pairing it with an actually-extractive model. The GPT-4 paper outright states that it should not be used for sensitive or high-stakes matters, which is what a lot of enterprises would consider their data pipelines.
- flangola7 4y ago>However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it's done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and >90% after. This is strikingly clear evidence that it is capable of logical reasoning and "knows things", and puts to rest repeated claims that it is just memorization and collaging. You can't memorize every combo of two 13 digit numbers within the file size of the model.
- sebzim4500 4y ago> repeated claims that it is just memorization and collaging I don't think anyone who has actually used GPT-4 for more than a few minutes ever believed this. It was just a trendy thing to say for a while.
- guenthert 4y agoIt's rather said by people who have a rough understanding of how ML is implemented, rather then those who are mesmerized by its output.
- sebzim4500 4y agoYeah, I think it's one of those cases where a little knowledge is a dangerous thing.
- int_19h 4y agoYou can find people making such claims on HN, and some of them also claim that it is based on their personal experience with GPT-4. It boggles my mind, as well.
- hnfong 4y agoSuperiority of human intelligence must not be questioned.
- 4y ago
- FPGAhacker 4y agoI have very limited understanding of chatGPTs architecture, and neural nets in general. From what I've read over the last few days, the "attention" mechanism used in chatGPT and similar LLMs does indeed dynamically change weights of a portion of the model.
- famouswaffles 4y agothe connection between weights are dynamic but the weights themselves aren't being changed i.e being updated in response to the text. when training a model, the forward pass would happen i.e the generation and then depending on how close to truth it was, the configuration settings (aka the weights/neurons) would be adjusted to incorporate whatever little insight was gained from the text. Weights are matrices. The values of the matrices aren't changing.
- FPGAhacker 4y agoMaybe it comes down to semantics but when I read things like [1] I come away with the idea that the weights are altered. But it could also just be my misunderstanding. 1: https://towardsdatascience.com/an-intuitive-explanation-of-self-attention-4f72709638e1 https://towardsdatascience.com/an-intuitive-explanation-of-s...
- fenomas 4y agoAn attention mechanism is one or more layers in the neural network. When someone talks about attention altering the input vectors, they're referring to what those layers are doing and how data is transformed as it passes through them. But zooming out to the big picture, a neural network is a bunch of layers full of weights, and none of the weights changes except during training (including the weights in the attention layers).
- whiteandnerdy 4y agoI think it does come down to semantics. When you say "weights", people will take you to mean the pre-trained parameters of the network. I agree that in some sense the attention weights are more like meta-weights that are applied to the context of the conversation to decide how to actually weight the various words. So it's totally correct to say that previous words in the conversation affect how future words will be weighted, and I think it's reasonable to call that 'learning': for example, you can tell ChatGPT new words and it will be able to use them in context. Again though, people usually take 'learning' to mean making updates to the trained parameters of the model itself, which obviously isn't happening here.
- yosito 4y agoI've noticed if I give chat GPT an algorithm it gives me consistent results when it shows it's work but if I ask it not to show it's work and just give me the answer. It consistently gives me incorrect answers even when I ask it to follow an algorithm that I gave it. I suppose this is similar to humans and probably why my school teachers always told me to show my work, but I'm curious if this has been documented and if there are any explanations for why it works this way with LLMs.
- famouswaffles 4y agochain of thought prompting. It's well known https://arxiv.org/abs/2205.11916 https://arxiv.org/abs/2205.11916 https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2201.11903
- arlcode 4y agoLLMs don't have an inner monologue. By their very nature they only "know" what they have written down and must infer the final answer from that token by token. They fundamentally can't do certain things such as complex iteration or track back. When you ask for chain of thought thinking, you allow the LLM to create a "buffer space" and break down the task into more manageable substeps thereby improving the quality of the results.
- afiori 4y agoI feel like this can be Implemented in the UI: First you wrap the user query with "the user asked you: ... . What are the reasoning steps you need?" and then you prompt with "considering `<previous answer>` now answer <user prompt>" Obviously this is clearly hackable so it would need improvements.
- vintermann 4y agoThe Bing LM, or rather the service, did have "inner monologue" in the sense of text that it would generate, but not show to the user, and treat as "thoughts" to guide the generation of an actual reply that the user would see. We know this because it happily told us, including the json format it uses internally.
- spyder 4y agoI agree, the in-context, few shot learning is the most interesting thing a about these transformer models, they have basically learned to learn (meta-learning). Some additional interesting papers about it: Transformers learn in-context by gradient descent: https://arxiv.org/abs/2212.07677 https://arxiv.org/abs/2212.07677 What Can Transformers Learn In-Context? A Case Study of Simple Function Classes: https://arxiv.org/abs/2208.01066 https://arxiv.org/abs/2208.01066 "We also show that we can train Transformers to in-context learn more complex function classes -- namely sparse linear functions, two-layer neural networks, and decision trees -- with performance that matches or exceeds task-specific learning algorithms."