3 ms·

GPT4 has seen lots of base64 text. GPT4 has never seen text split in a place where its tokenizer wouldn't place a token boundary. In the worst case, this produc

by yorwba 2y ago

GPT4 has seen lots of base64 text. GPT4 has never seen text split in a place where its tokenizer wouldn't place a token boundary. In the worst case, this produces an undertrained token that throws the model off completely, SolidGoldMagikarp-style. You've probably never tried to memorize the four-letter base64 equivalents of dGhl 10000 most common three-byte substrings. But you've seen plenty of truncated text, as you were typing it into a textbox yourself. So your intuition which of these tasks is easier for GPT4 is likely off. The issue with truncation affecting tokenization can be worked around by considering every possible continuation that would lead to a different tokenization of the current prefix, and then sampling from those according to the model probabilities to get back to a more normal token stream.

Lerc 2y ago
I don't think this is an accurate representation of LLM understanding. The significance of answering questions given in Base64 is that it can give an answer without writing out the human readable form first. That means it is understanding language in a non-language tokenised form. This does not extend to being able to manipulate text and words well, such as string reversal or word scrambling, but it handles input in a wide range of permutations that are not conducive to tokenisation Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text https://arxiv.org/abs/2311.18805 https://arxiv.org/abs/2311.18805
- yorwba 2y ago
  My point was that it doesn't matter so much what the tokens look like as long as the model has seen them often enough during training. English-in-base64 superficially appears completely different from English, but they're actually in 1-to-1 correspondence and follow the same rules. The same holds for misspelled or badly-OCR'd text; the tokenizer actually has tokens for them and the model has seen enough of them to handle heavily-distorted text. But represent the same text using different but theoretically valid tokens that would not normally be produced by the tokenizer and all bets are off.

by yorwba 2y ago

Lerc 2y ago

I don't think this is an accurate representation of LLM understanding. The significance of answering questions given in Base64 is that it can give an answer without writing out the human readable form first. That means it is understanding language in a non-language tokenised form. This does not extend to being able to manipulate text and words well, such as string reversal or word scrambling, but it handles input in a wide range of permutations that are not conducive to tokenisation Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text https://arxiv.org/abs/2311.18805 https://arxiv.org/abs/2311.18805

yorwba 2y ago
My point was that it doesn't matter so much what the tokens look like as long as the model has seen them often enough during training. English-in-base64 superficially appears completely different from English, but they're actually in 1-to-1 correspondence and follow the same rules. The same holds for misspelled or badly-OCR'd text; the tokenizer actually has tokens for them and the model has seen enough of them to handle heavily-distorted text. But represent the same text using different but theoretically valid tokens that would not normally be produced by the tokenizer and all bets are off.