9 ms·
This reminds me of how GPT-2/3/J came across https://reddit.com/r/counting https://reddit.com/r/counting, wherein redditors repeatedly post incremental numbers
by btown 3y ago
This reminds me of how GPT-2/3/J came across https://reddit.com/r/counting https://reddit.com/r/counting, wherein redditors repeatedly post incremental numbers to count to infinity. It considered their usernames, like SolidGoldMagikarp, such common strings on the Internet that, during tokenization, it treated them as top-level tokens of their own.
https://www.alignmentforum.org/posts/8viQEp8KBg2QSW4Yc/solidgoldmagikarp-iii-glitch-token-archaeology https://www.alignmentforum.org/posts/8viQEp8KBg2QSW4Yc/solid...
https://www.lesswrong.com/posts/LAxAmooK4uDfWmbep/anomalous-tokens-reveal-the-original-identities-of-instruct https://www.lesswrong.com/posts/LAxAmooK4uDfWmbep/anomalous-...
Vocabulary isn't infinite, and GPT-3 reportedly had only 50,257 distinct tokens in its vocabulary. It does make me wonder - it's certainly not a linear relationship, but given the number of inferences run every day on GPT-3 while it was the flagship model, the incremental electricity cost of these Redditors' niche hobby, vs. having allocated those slots in the vocabulary to actually common substrings in real-world text and thus reducing average input token count, might have been measurable.
It would be hilarious if the subtitle on OP's site, "IECC ChurnWare 0.3," became a token in GPT-5 :)
- mort96 3y agoDuring tokenization, the usernames became tokens... but before training the actual model, they removed stuff like that from the training data, so it was never trained on text which contains those tokens. As such, it ended up with tokens which weren't associated with anything; glitch tokens.
- zelphirkalt 3y agoSo it becomes a game of getting things into the training data, past the training data cleanup step.
- btown 3y agoIt's interesting: perhaps the stability (from a change management perspective) of the tokenization algorithm, being able to hold that constant, between old and new training runs was deemed more important than trying to clean up the data at an earlier phase of the pipeline. And the eventuality of glitch tokens was deemed an acceptable consequence.
- aidenn0 3y agoI wonder how much the source content is the cause of hallucinations rather than anything inherent to LLMs. I mean if someone posts a question on an internet forum that I don't know the answer to, I'm certainly not going to post "I don't know" since that wouldn't be useful. In fact, in general, in any non one-on-one conversation the answer "I don't know" is not useful because if you don't know in a group, your silence indicates that.
- abracadaniel 3y agoThat’s a good observation. If LLMs had taken off 15 years ago, maybe they would answer every question with “this has already been asked before. Please use the search function”
- exe34 3y agoMarked as duplicate.
- btown 3y agoYour question may be a better fit for a different StackExchange site.
- amadeuspagel 3y agoWe prefer questions that can be answered, not merely discussed.
- exe34 3y agoFeel like we're going to get dang on our case soon...
- TeMPOraL 3y agoTopic locked as not constructive.
- taco-hands 3y ago
- Karellen 3y agoMore glitch token discussion over at Computerphile: https://www.youtube.com/watch?v=WO2X3oZEJOA https://www.youtube.com/watch?v=WO2X3oZEJOA
- qeternity 3y ago> GPT-3 reportedly had only 50,257 The most common vocabulary size today is 32k.