5 ms·
I guess they've worked out to always get exactly the same text back? Hopefully this won't be a repeat of the photocopiers that used a similar compression techn
by jandrese 3y ago
I guess they've worked out to always get exactly the same text back? Hopefully this won't be a repeat of the photocopiers that used a similar compression technique and occasionally substituted letters in the copy.
https://www.bbc.com/news/technology-23588202 https://www.bbc.com/news/technology-23588202
- wmf 3y agoYeah, using a probabilistic predictor to do lossless compression has been known for a few decades.
- stevefan1999 3y agohttps://en.m.wikipedia.org/wiki/Arithmetic_coding https://en.m.wikipedia.org/wiki/Arithmetic_coding
- cbhl 3y agoThe author, Fabrice Bellard, is well-known for his work on QEMU (among other things), but I'd treat the rest of the posts as "idea sharing" at an early stage. For example, while BPG (Better Portable Graphics) didn't take off, the core idea (image formats based on video codecs) can be found in WebP (VP8, VP9) and AVIF (AV1) and HEIF (HEVC/H.265). For this, I'd look at comparable work in the audio codec space, including Lyra (https://arxiv.org/abs/2102.09660 https://arxiv.org/abs/2102.09660) and SoundStream (https://arxiv.org/abs/2107.03312 https://arxiv.org/abs/2107.03312). For text compression, I think it would be novel if you could get a lossy but semantically equivalent decompression that was resilient to the exact hardware used for inference. I don't think that's what happened here, though, given the requirement for "exact same GPU and model".
- wood_spirit 3y agoFabrice is also well known for creating Ffmpeg and discovering an algorithm for computing digits of PI https://en.m.wikipedia.org/wiki/Fabrice_Bellard https://en.m.wikipedia.org/wiki/Fabrice_Bellard He has also held the ‘world record’ for data compression since 2019 with nncp http://mattmahoney.net/dc/text.html#1085 http://mattmahoney.net/dc/text.html#1085
- foooorsyth 3y agoHe is simply one of the most important programmers alive today. He should win a Turing award in his lifetime.
- stevefan1999 3y agoI am afraid not. Turing awards are all theorists game, not engineers. He could get in the IEEE hall of fame though, I recommends.
- foooorsyth 3y ago>Turing awards are all theorists game, not engineers Robert Metcalfe just won the award for Ethernet. Go scan through the list of Turing winners again. It is not just awarded to pure theorists. There’s a huge slant to academics who also found (or caused) massive commercial success, especially in recent years. Bellard could get hit by a bus tomorrow and have >the most important video tool ever >the most important emulation tool ever >the best text compression tool ever …on his resume. And with those tools he’s not just wrapping algorithms cooked up by others, but also innovating on fundamental theory. VirtualBox and the Android emulator? That’s QEMU. Every video platform in existence? Thin wrappers to FFmpeg. Seems deserving to me, and certainly seems as or more impactful that many names on that winners list.
- rcme 3y agoDon’t forget about QuickJS!
- IggleSniggle 3y ago"Semantically equivalent decompression" is a very interesting idea, but I take issue with the idea that different words can share semantic equivalence. But I've also been reading a lot of poetry today, so subtle word choice feels important, even though I can recognize it may not be that important...a LLM could easily produce "better" or "worse" text than a human (while being lossy in either case), but then I have questions about the reason for using words in the first place.
- crazygringo 3y agoWow. Lossy text compression is a... horrifying concept.
- throwaway14356 3y agolossy text horror
- Solvency 3y agoWw. Lssy txt cmprsn is hrffyng cncpt. Or is it actly ok.
- crazygringo 3y agoJeez. Text squeezing in the manner of loss is a scary idea.
- midoridensha 3y agoLossy text comparison is a horrifying concept?
- supertrope 3y agoY ur account b lanc is $36 2.17
- fsiefken 3y agoCheck keyscript https://www.reddit.com/r/FastWriting/comments/w7a4k1/keyscript_shorthand/ https://www.reddit.com/r/FastWriting/comments/w7a4k1/keyscri...
- bambax 3y agoBut why? A summary is a form of lossy text compression, and it's extremely useful. That's not what this is but if it can actually produce "semantically equivalent" decompressed text it's fascinating.
- deleted 3y ago[deleted]
- ks2048 3y agoIf you can predict text very well, it can still be a good lossless compressor - you just have to encode the errors you make. Fewer errors, less data to store
- mistercow 3y agoMore specifically, you can use entropy coding to spend fewer bits the more likely you rated the token and more bits the less likely you rated the token. So "errors" aren't a black and white thing. And since LLMs output log probabilities, they're basically perfect for this.
- OGWhales 3y agoJBIG2 is what the NSO group exploited as part of their zero-click iMessage exploit. Google zero has a good write up if anyone hasn’t heard about it yet—it’s super cool: https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-into-nso-zero-click.html https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i...
- nocoiner 3y agoThis is so very much worth a read. It is almost literally mind blowing once you realize how deep the exploit goes.
- Uptrenda 3y ago>Practical circuits >JBIG2 doesn't have scripting capabilities, but when combined with a vulnerability, it does have the ability to emulate circuits of arbitrary logic gates operating on arbitrary memory. So why not just use that to build your own computer architecture and script that!? That's exactly what this exploit does. Using over 70,000 segment commands defining logical bit operations, they define a small computer architecture with features such as registers and a full 64-bit adder and comparator which they use to search memory and perform arithmetic operations. It's not as fast as Javascript, but it's fundamentally computationally equivalent. >The bootstrapping operations for the sandbox escape exploit are written to run on this logic circuit and the whole thing runs in this weird, emulated environment created out of a single decompression pass through a JBIG2 stream. It's pretty incredible, and at the same time, pretty terrifying. Wow... that's really out of the box. They built their own VM from a JBIG compression stream vuln and wrote an exploit to run on it... Not bad.
- CobrastanJorji 3y agoEven if you haven't, passing along a checksum of the correct original would still allow it to be used safely. Decompression would either be correct or fail to decompress (outside of considerations of intentional hash collision attacks).
- frud 3y agoFrom the link: "The same exact GPU model and program versions must be used for compression and decompression". So it seems to me that you need a standardized and deterministic LLM execution model before this becomes useful.