4 ms·

In all seriousness: why has it been years now and it feels like there is no incremental engineering-level progress on these issues? Like, it seems like doing so

by saurik 2y ago

In all seriousness: why has it been years now and it feels like there is no incremental engineering-level progress on these issues? Like, it seems like doing some manual intervention to the tokenization to at least remove exceptional tokens and add some semantics to how they break up numbers seem like quick wins.

falcor84 2y ago
>In all seriousness: why has it been years now and it feels like there is no incremental engineering-level progress on these issues? From where I'm standing, LLMs appear to be the fastest moving technological field in history.
- PaulHoule 2y ago
  A field can seem to be going quickly and going nowhere at the same time. Or rather a new technique can be invented and then exhausted in the time it takes somebody to get a PhD. (See https://en.wikipedia.org/wiki/Renormalization_group https://en.wikipedia.org/wiki/Renormalization_group applied to phase transitions, which turned up just in time for the physics job crisis of 1970) I didn't ever believe that there was going to be a GPT-5 trained with exponentially more text and resources. Not only is there not enough text, but that's the path to ruin. Why? Cycle time. Two years ago we had little idea of how those models work so I knew there was a huge room in improving performance. It gets the cost down, it lets you put the models on your device, and it speeds up development. If I can train 10 models in the time it takes you to train 1 model I can make much faster progress. However even a GPT-15 trained with a Dyson sphere is going to struggle to sort things. (Structurally a pure LLM can't do that!) My #1 beef with Microsoft's Copilot is that you can ask it if it can sort a certain list of items (either a list you are discussing with it or say "states of the United States ordered by percent water area") it will say yes and if you ask it what it thinks the probability is that it will get it in the right order it will say "very high" but when you try it the list comes out totally wrong. It is equally unable to "help me make an atom bomb" except in the bomb case it will say that it can't but in the sorting case it says it can. The obvious answer is that it should use tools to sort. That's right but the problem of "knowing what you can really do with your tools" is philosophically challenged. (With problems so intractable it leads people like Roger Penrose to conclude "I couldn't do math if I wasn't a thetan")
  - JustAndy 2y ago
    I'm not really sure I understand your sorting example, maybe try it out in gpt and post the link to show exactly what you mean. The refusal of the model is something trained into the model by the process of rlhf, and it can also be untrained, by the process of abliteration [1]. Also, LLMs are capable of using tools in this very moment [2]. [1]: https://huggingface.co/blog/mlabonne/abliteration https://huggingface.co/blog/mlabonne/abliteration [2]: https://www.anthropic.com/news/analysis-tool https://www.anthropic.com/news/analysis-tool
    - PaulHoule 2y ago
      I'm deliberately blurring refusal with having an accurate picture of its own abilities and, past that, having an accurate picture of of what it can do given tools. Both are tested by "Can you X?" With refusal you find just how shallow it is because it really will answer all sorts of questions that are "helpful" in making a nuclear bomb but when you ask it directly it shuts up. In another sense nothing it does is "helpful" because it's not going to hunt down some people in central asia who have 50kg of U235 burning a hole in their pocket for you, which is what would actually "help". I use tool using LLMs frequently, but I find they frequently need help using their tools, it is a lot of fun to talk to Windsurf about the struggles it has with its tools and it feels strangely satisfying to help it out.
- saurik 2y ago
  You totally ignored "on these issues" and are essentially saying there is no need to work on that as they worked on something else, which is extremely strange for a thing which feels like a really trivial win, and should be shocking. Whether you like it or not, it is entirely fair to look at an entire ecosystem and ask why some trivial thing that everyone talks about all the time hasn't seen any attention even if the entire ecosystem is getting widespread advancement. Like, I think it would also be fair to complain about how bad the hinge on AirPods are, causing the case to explode when dropped and your earbuds to fly everywhere (potentially getting very dirty) as well as wear out and cause spurious activation (leading to audio routing issues and rapid battery drain). To then point out that this is one of the most successful consumer devices in recent years and was a remarkable improvement to what came before as well as a continuing achievement of engineering as they do in fact get better in amazing ways every couple years is more than just a non sequitur: it is frankly just annoying.
entilzha 2y ago
(Author Here) There is at least some work on character based modeling, but it hasn’t scaled well before. The challenge I think with something more adhoc for exceptional tokens is that it’s hard to see gains since they are by definition, infrequent. If the text is rare enough, BPE should produce many single byte tokens, so current models actually expend more compute on these rare sequences. BLT scales well because it expends less compute (by patching) on more predictable (low entropy) byte sequences. Current models only to some degree get this benefit, if it’s a larger BPE token, but that only goes so far. So it’s really two related, but different motivations.

by saurik 2y ago

falcor84 2y ago

>In all seriousness: why has it been years now and it feels like there is no incremental engineering-level progress on these issues? From where I'm standing, LLMs appear to be the fastest moving technological field in history.

PaulHoule 2y ago
A field can seem to be going quickly and going nowhere at the same time. Or rather a new technique can be invented and then exhausted in the time it takes somebody to get a PhD. (See https://en.wikipedia.org/wiki/Renormalization_group https://en.wikipedia.org/wiki/Renormalization_group applied to phase transitions, which turned up just in time for the physics job crisis of 1970) I didn't ever believe that there was going to be a GPT-5 trained with exponentially more text and resources. Not only is there not enough text, but that's the path to ruin. Why? Cycle time. Two years ago we had little idea of how those models work so I knew there was a huge room in improving performance. It gets the cost down, it lets you put the models on your device, and it speeds up development. If I can train 10 models in the time it takes you to train 1 model I can make much faster progress. However even a GPT-15 trained with a Dyson sphere is going to struggle to sort things. (Structurally a pure LLM can't do that!) My #1 beef with Microsoft's Copilot is that you can ask it if it can sort a certain list of items (either a list you are discussing with it or say "states of the United States ordered by percent water area") it will say yes and if you ask it what it thinks the probability is that it will get it in the right order it will say "very high" but when you try it the list comes out totally wrong. It is equally unable to "help me make an atom bomb" except in the bomb case it will say that it can't but in the sorting case it says it can. The obvious answer is that it should use tools to sort. That's right but the problem of "knowing what you can really do with your tools" is philosophically challenged. (With problems so intractable it leads people like Roger Penrose to conclude "I couldn't do math if I wasn't a thetan")
- JustAndy 2y ago
  I'm not really sure I understand your sorting example, maybe try it out in gpt and post the link to show exactly what you mean. The refusal of the model is something trained into the model by the process of rlhf, and it can also be untrained, by the process of abliteration [1]. Also, LLMs are capable of using tools in this very moment [2]. [1]: https://huggingface.co/blog/mlabonne/abliteration https://huggingface.co/blog/mlabonne/abliteration [2]: https://www.anthropic.com/news/analysis-tool https://www.anthropic.com/news/analysis-tool
  - PaulHoule 2y ago
    I'm deliberately blurring refusal with having an accurate picture of its own abilities and, past that, having an accurate picture of of what it can do given tools. Both are tested by "Can you X?" With refusal you find just how shallow it is because it really will answer all sorts of questions that are "helpful" in making a nuclear bomb but when you ask it directly it shuts up. In another sense nothing it does is "helpful" because it's not going to hunt down some people in central asia who have 50kg of U235 burning a hole in their pocket for you, which is what would actually "help". I use tool using LLMs frequently, but I find they frequently need help using their tools, it is a lot of fun to talk to Windsurf about the struggles it has with its tools and it feels strangely satisfying to help it out.
saurik 2y ago
You totally ignored "on these issues" and are essentially saying there is no need to work on that as they worked on something else, which is extremely strange for a thing which feels like a really trivial win, and should be shocking. Whether you like it or not, it is entirely fair to look at an entire ecosystem and ask why some trivial thing that everyone talks about all the time hasn't seen any attention even if the entire ecosystem is getting widespread advancement. Like, I think it would also be fair to complain about how bad the hinge on AirPods are, causing the case to explode when dropped and your earbuds to fly everywhere (potentially getting very dirty) as well as wear out and cause spurious activation (leading to audio routing issues and rapid battery drain). To then point out that this is one of the most successful consumer devices in recent years and was a remarkable improvement to what came before as well as a continuing achievement of engineering as they do in fact get better in amazing ways every couple years is more than just a non sequitur: it is frankly just annoying.

entilzha 2y ago

(Author Here) There is at least some work on character based modeling, but it hasn’t scaled well before. The challenge I think with something more adhoc for exceptional tokens is that it’s hard to see gains since they are by definition, infrequent. If the text is rare enough, BPE should produce many single byte tokens, so current models actually expend more compute on these rare sequences. BLT scales well because it expends less compute (by patching) on more predictable (low entropy) byte sequences. Current models only to some degree get this benefit, if it’s a larger BPE token, but that only goes so far. So it’s really two related, but different motivations.