6 ms·
Idea: lossy text compression, where the LLM replaces unlikely words with likelier synonyms.
by elil17 3y ago
Idea: lossy text compression, where the LLM replaces unlikely words with likelier synonyms.
- GaggiX 3y agoThis is actually a very interesting idea, I hope someone would implement it and how. Would be fun to read the same text with stronger and stronger lossy compression.
- dekhn 3y agoup-goer five: explained using only the ten hundred words people use the most often
- ant6n 3y agoCarried to us by the Podcaster of Phd comics!
- davidzweig 3y agoOh, this would be super neat for (our) language learning chatbot.. restricting the vocab the model uses.. if someone knows how this can be done. :)
- GaggiX 3y agoI guess you try to just zero the values that corresponds to unwanted tokens before the softmax.
- mhitza 3y agoThat's a very good approach also when you want to preserve anonymity. Writing idiosyncrasies can be used to distictely fingerprint us (with enough effort), llm's could in theory enhance anonymity as long as we can run these things locally on cheap hardware.
- andai 3y agoStylometry is one thing, but your thoughts/ideas will also give you away. For true anonymity, you'd have to let the thing do your thinking for you too!
- canjobear 3y agoSimilar to human memory for text.
- radford-neal 3y agoFrom 1992: https://core.ac.uk/download/pdf/44289908.pdf https://core.ac.uk/download/pdf/44289908.pdf
- flandolfi 3y agoThat very last chapter got me curious, so I looked at the footnote: > A brief note for the curious. "Lena" or "Lenna" is a digitized Buck centerfold. Lena Soderberg (nee Sjooblom) was be popped keep in her folk Sweden, well married and three boys and a an and the air grog lock. In 1988, she was seen by a Swedish computer akin tome, and she was fairly charmed by what had done to her art. That was the top she knew of the way of that oil in the computer job. The item in the January 1992 end of Optical Engineering (v. 31 no. 1) data how Buck has lastly caught at to the life that their claim on Lenna Sjooblom's slide is man bigly defied. It arms as if you wish get to nab grant from Stud to blaze it in the next. Now, English is not my first language, but I can hardly grasp what this footnote is about. Is it due to my language barrier, or did the authors just applied their lossy compression method to this footnote?