3 ms·

I think the base64 decoding is interesting: in a sense, model training set likely had lots of base64-encoded data (imagine MIME data in emails, JSON, HTML...),

I think the base64 decoding is interesting: in a sense, model training set likely had lots of base64-encoded data (imagine MIME data in emails, JSON, HTML...), but for it to decode successfully, it had to learn decode sequences for every 4 base64 characters (which turn into 3 bytes). This could have been generated as a training set data easily, and I only wonder if each and every one was them was found enough times to end up in the weights?

curioussquirrel 1y ago
Even GPT 3.5 is okay (but far from great) at Base64, especially shorter sequences of English or JSON data. Newer models might be post-trained on Base64-specific data, but I don't believe it was the case for 3.5. My guess is that as you say, given the abundance of examples on the internet, it became one of the emergent capabilities, in spite of its design.
- ACCount37 1y ago
  No one does RL for better base64 performance. LLMs are just superhuman at base64, as a natural capability. If an LLM wants a message to be read only by another LLM? Base64 is occasionally chosen as an obfuscation method of choice. Which is weird for a number of reasons.
  - necovek 1y ago
    Why are you so confident about this? I am honestly interested if you were part of any one LLM training data collection teams because that's the only way to be so certain. It's trivial to generate a full mapping of all base64 4-byte sequences which map to all 3-byte 8-bit sequences (there is only 8^3 of different "tokens", or 2048), and especially to any sequences coming out as ASCII (obviously even fewer). If I was building a training set, I would include the mapping in multiple shapes and formats, because why not? If it's an emergent "property", have you tried asking an LLM to do a base48 for instance? Or maybe even something crazier like base55 (keeping it a subset of base64 set).
    - necovek 1y ago
      For kicks, I've tried this out with ChatGPT5: it nicely explained how it will use A-Za-z0123 as the alphabet for base55, and then duly went and produced a string with a 4 in it. It's not even base64, so it's all sorts of messy :)
    - ACCount37 1y ago
      The conventional wisdom is that real world text is the most valuable pre-training data. There is some experimentation on using algorithmically generated synthetic data in pre-training, as well as some intentional inclusions of "weird" data - like CSV logs of weather readings. But generally, it's seen as computationally inefficient - compared to "normal" pre-training done on natural data. In a world where compute is much cheaper and getting new data is much more expensive, I would expect this kind of thing to be pursued more. We're heading for that world. But we aren't there yet. I haven't experimented with baseN encodings myself, no. But if I were to down the expectations in advance: 1. Base64 is by far the best-known baseN encoding in LLMs. 2. This is driven mainly by how well represented meaningful base64 strings are in the natural "scraped web" datasets. LLMs learn base64 the way they learn languages. 3. Every LLM pre-trained on "scraped web" data will be somewhat capable of reading and writing base64. 4. Base64-encoded text is easier to read for an LLM than encoded non-text binary data. 5. The existence of a strict, learnable "4 characters -> 3 bytes" map is quite beneficial, but not vital.
    - Timwi 1y ago
      > It's trivial to generate a full mapping of all base64 4-byte sequences which map to all 3-byte 8-bit sequences (there is only 8^3 of different "tokens", or 2048) How did you get this number? The correct number is 64^4 = 256^3 = 16777216.
      - necovek 1y ago
        My bad, just a total brainfart on my part: you are completely right! :)
flowerthoughts 1y ago
I would much rather that the model learns natural languages and knows how to run base64(1) to do decoding. Parameters should be considered precious so we can get the model sizes down from these absurd levels. I'm a big fan of the Mixture of Experts approach, and having agents attached to some of those experts would be a great step forward. Say an expert that knows how to run common shell scripts, and its parameters are only used when it realizes a shell script would solve the problem.

curioussquirrel 1y ago

Even GPT 3.5 is okay (but far from great) at Base64, especially shorter sequences of English or JSON data. Newer models might be post-trained on Base64-specific data, but I don't believe it was the case for 3.5. My guess is that as you say, given the abundance of examples on the internet, it became one of the emergent capabilities, in spite of its design.

ACCount37 1y ago
No one does RL for better base64 performance. LLMs are just superhuman at base64, as a natural capability. If an LLM wants a message to be read only by another LLM? Base64 is occasionally chosen as an obfuscation method of choice. Which is weird for a number of reasons.
- necovek 1y ago
  Why are you so confident about this? I am honestly interested if you were part of any one LLM training data collection teams because that's the only way to be so certain. It's trivial to generate a full mapping of all base64 4-byte sequences which map to all 3-byte 8-bit sequences (there is only 8^3 of different "tokens", or 2048), and especially to any sequences coming out as ASCII (obviously even fewer). If I was building a training set, I would include the mapping in multiple shapes and formats, because why not? If it's an emergent "property", have you tried asking an LLM to do a base48 for instance? Or maybe even something crazier like base55 (keeping it a subset of base64 set).
  - necovek 1y ago
    For kicks, I've tried this out with ChatGPT5: it nicely explained how it will use A-Za-z0123 as the alphabet for base55, and then duly went and produced a string with a 4 in it. It's not even base64, so it's all sorts of messy :)
  - ACCount37 1y ago
    The conventional wisdom is that real world text is the most valuable pre-training data. There is some experimentation on using algorithmically generated synthetic data in pre-training, as well as some intentional inclusions of "weird" data - like CSV logs of weather readings. But generally, it's seen as computationally inefficient - compared to "normal" pre-training done on natural data. In a world where compute is much cheaper and getting new data is much more expensive, I would expect this kind of thing to be pursued more. We're heading for that world. But we aren't there yet. I haven't experimented with baseN encodings myself, no. But if I were to down the expectations in advance: 1. Base64 is by far the best-known baseN encoding in LLMs. 2. This is driven mainly by how well represented meaningful base64 strings are in the natural "scraped web" datasets. LLMs learn base64 the way they learn languages. 3. Every LLM pre-trained on "scraped web" data will be somewhat capable of reading and writing base64. 4. Base64-encoded text is easier to read for an LLM than encoded non-text binary data. 5. The existence of a strict, learnable "4 characters -> 3 bytes" map is quite beneficial, but not vital.
  - Timwi 1y ago
    > It's trivial to generate a full mapping of all base64 4-byte sequences which map to all 3-byte 8-bit sequences (there is only 8^3 of different "tokens", or 2048) How did you get this number? The correct number is 64^4 = 256^3 = 16777216.
    - necovek 1y ago
      My bad, just a total brainfart on my part: you are completely right! :)

flowerthoughts 1y ago

I would much rather that the model learns natural languages and knows how to run base64(1) to do decoding. Parameters should be considered precious so we can get the model sizes down from these absurd levels. I'm a big fan of the Mixture of Experts approach, and having agents attached to some of those experts would be a great step forward. Say an expert that knows how to run common shell scripts, and its parameters are only used when it realizes a shell script would solve the problem.