4 ms·
I found this tool recently when it was linked from this Computerphile video[1] about "glitch tokens". tldw: Certain junk data was thrown out post-tokenization
by non- 4y ago
I found this tool recently when it was linked from this Computerphile video[1] about "glitch tokens".
tldw:
Certain junk data was thrown out post-tokenization e.g. the /r/counting[2] community data and debug logs from Rocket League
some tokens specific to those contexts stuck around, however, and are now like "a color you've never seen before" as far as GPT-X models are concerned
giving the model one of these "glitch" tokens causes it to kind of freak out and return gibberish or some completely random response because it has not encountered them during training, because they were removed when the data was cleaned.
[1] https://www.youtube.com/watch?v=WO2X3oZEJOA https://www.youtube.com/watch?v=WO2X3oZEJOA
[2] https://reddit.com/r/counting https://reddit.com/r/counting
- non- 4y agoAnother interesting tweet[1] I saw today shows how you can ask ChatGPT to compress text and it invents it's own (effective!) shorthand. I bet it's related somehow to glitch tokens and the way GPT is grouping tokens internally. [1] https://mobile.twitter.com/VictorTaelin/status/1642664054912155648 https://mobile.twitter.com/VictorTaelin/status/1642664054912...
- artdigital 4y agoI experimented with something similar previously and it doesn’t really work. It usually can’t decompress it properly
- kleton 4y agoThe example given is obviously gpt's own generation. It might not be expected to roundtrip arbitrary text.
- Genbox 4y agoHad not seen the Computerphile video yet. Laughing my socks off right now. Thanks for that.