8 ms·
The author identifies a real problem and poses a simple solution. It passes all my crank tests (why did no one come up with this before? Because the author is i
by fwlr 3y ago
The author identifies a real problem and poses a simple solution. It passes all my crank tests (why did no one come up with this before? Because the author is intimately familiar with the softmax function from work outside of ML, and plausibly nobody who’s investigating these issues is remotely as familiar, so despite researchers narrowing the issue down to “something to do with softmax”, they don’t have a deep enough understanding of softmax to see what’s wrong).
If the author is reading any of these comments, though, I would urge them to expand on their claim that “I’m 99.44% sure that it will resolve the outlier feedback loop”. As it stands, that’s the only explanation we get of how the outliers might be related to softmax!
- Majromax 3y ago> why did no one come up with this before? And because the effects of the problem are subtle. Supposing the diagnosis is correct, full-precision LLMs still avoid the issue through large attention weights given to meaningless tokens to give harmless attention outputs. The problem only matters when quantizing weights, and quantized performance isn't really the goal of recent cutting-edge LLM development.
- Legend2440 3y agoYeah, but it lacks the most important test: results. He hasn't actually tried it, he just thinks it will work. For such a simple change to the softmax it wouldn't take long to verify. It's really embarrassing to not do that before publishing.
- refulgentis 3y agoIt's not embarrassing at all. I think there might be some curse of the auto-didact here, hinging on the meaning of publish: it would be embarrassing if he was capital-P publishing, as in a scientific paper. The blog goes to great lengths to point out it is _not_ capital-P publishing.
- joebiden2 3y agoYou seem to really disregard the positions of this author. They seem to have invested substantial efforts in that specific area of research. To validate the idea the author has, it would be required to train a LLM from zero. If the author is right, you would get similar results to the current generation of LLMs, but with (a lot) less space required for the intermediate layers. The time to achieve that is still measured in kilo- to mega-dollars, why is it wrong to put that idea in the open to substantially criticize or adopt?
- Legend2440 3y agoYou don't need to train a ChatGPT-sized LLM, a toy nanoGPT would have been enough. You can train those on a consumer GPU in an afternoon. And yes I do disregard his research effort. There are hundreds of well-justified and well-researched "clever tricks" for improving Transformers, and almost all of them don't work. I'll believe it when I see the results.
- knewter 3y agoGoogle used it in flaxformers since 2021 apparently
- renewiltord 3y agoDo you know of handy testing steps? I suppose I could ask ChatGPT, but if someone has a validated "here, this is how you do it" I have a 3090 that I can do it on, but I'm not keen to debug anything here.
- nl 3y agoTesting steps (based on thinking about this for 30 seconds - so probably can be improved): Train a Transformer based model with and without the modified Softmax (Suggestions: GPT-2 or nanoGPT) Measure performance - I'd probably start with Perplexity and see if there is any difference (we'd expect little difference). Quantize both models with different quantization strategies. Measure the perplexity of the quantized models of different sizes. We'd expect the performance to drop off quicker for the non-modified model than the modified one if this is working.
- furyofantares 3y agoIt's a blog post. And it includes a call for help in testing the idea.
- tel 3y ago> why did no one come up with this before? Because the author is intimately familiar with the softmax function from work outside of ML, and plausibly nobody who’s investigating these issues is remotely as familiar I doubt that is true. Softmax is extremely well understood within the ML community. It's a very common trick, these properties are well-known as well. It feels very unlikely that nobody has thought of this before. That said, it's also plausible that the current softmax convention was chosen by accident and the author is right to identify this drawback.
- sedael 3y ago>why did no one come up with this before So it turns out someone did. Specifically google did. This exact same idea has been in flaxformers since at least November 2021. https://github.com/google/flaxformer/blame/ee62754ebe5a5eeb111493622de5537133822e3e/flaxformer/components/attention/dense_attention.py#L50 https://github.com/google/flaxformer/blame/ee62754ebe5a5eeb1... Specifically to save people a click it says: > """Softmax function with an additional virtual logit equal to zero. For compatibility with some previously trained models. This is equivalent to adding one to the denominator. In the context of attention, it allows you to attend to nothing. And creates the exact same modified softmax as this essay. I suppose only time will tell why it was ignored publicly before, maybe it doesn't do much, maybe it just fell through the cracks, maybe google just didnt push it, who knows
- littlestymaar 3y ago> I suppose only time will tell why it was ignored publicly before, maybe it doesn't do much, maybe it just fell through the cracks, maybe google just didnt push it, who knows Maybe quantization wasn't as hot back then than it is now?
- jablongo 3y agoYea the benefit is not going to come in terms of performance for a given model, but in terms of ability to be efficiently quantized.
- toxik 3y agoOr maybe it doesn’t really do anything to improve performance.
- deleted 3y ago[deleted]
- visarga 3y agoI interpreted it as cracking a joke about miscalibrated probs in softmax, it tends to be 99.9% sure, or 0.1%, but little in-between.