3 ms·

> I'm not even sure that a memory safe language could have prevented this; this is a rare case where you actively want to avoid the overflow check. This compon

> I'm not even sure that a memory safe language could have prevented this; this is a rare case where you actively want to avoid the overflow check. This component should be written in WUFFS. If you were correct, that the bounds check isn't needed that's fine, WUFFS doesn't emit runtime bounds checks. If, as was the case here, the software is wrong because it has a bounds miss, that won't compile in WUFFS. You might be thinking, "That's impossible" and if WUFFS was a general purpose programming language you'd be correct. Rice's Theorem, non-trivial semantic properties are Undecidable. Fortunately WUFFS isn't a general purpose language. The vast majority of software can't be written with WUFFS. But you can write image codecs.

lifthrasiir 3y ago
I agree that Wuffs [1] would have been a very good alternative! ...if it can be made more general. AFAIK Wuffs is still very limited, in particular it never allows dynamic allocation. Many formats, including those supported by Wuffs the library, need dynamic allocation, so Wuffs code has to be glued with unverified non-Wuffs code [2]. This approach only works with simpler formats. [1] https://github.com/google/wuffs/blob/main/doc/wuffs-the-language.md https://github.com/google/wuffs/blob/main/doc/wuffs-the-lang... [2] https://github.com/google/wuffs/blob/main/doc/note/memory-safety.md#allocation-free-apis https://github.com/google/wuffs/blob/main/doc/note/memory-sa...
- tialaramex 3y ago
  To me it looks like more generality is neither necessary nor desirable. It's better that the codec is 100% safe but there's a tiny amount of unsafe glue code, than make the codec unsafe to add generality. Google has smart people, finding the bug in 15 lines of glue code before it goes out the door is definitely easier than finding this C bug. This is the same thing that powers Rust successfully, but taking it further.
  - lifthrasiir 3y ago
    > It's better that the codec is 100% safe but there's a tiny amount of unsafe glue code, [...] I meant that many codec cannot be easily made in such way because memory allocation can occur here and there.
    - tialaramex 3y ago
      > many codec cannot be easily made in such way because memory allocation can occur here and there. This is an extremely vague complaint and thus suspicious. Of course we could imagine an image format which decides to allow you to declaratively construct cloud servers which transmit XML and so it needs DRM to protect your cloud service credentials - but I claim (and I feel like most people will agree) the fact WUFFS can't do that is a good thing and we should not use this hypothetical "image" format aka massive security hole. Try specifics. This is a WebP bug. For a WebP codec, where does it need memory allocation? My guess is it does allocations only during a table creation step, and after it figures out how big the final image is. So, twice, in specific parts of the code, like JPEG.
      - michaelt 3y ago
        If you want to open an image file of unknown resolution, you might want the library to allocate and return a byte array matching the resolution of the image. Oh, you could design a more complex API to avoid that - but I'm a programmer of very average ability, ask me to juggle too many buffers and struts and I'll probably introduce a buffer overflow by mistake.
      - lifthrasiir 3y ago
        WebP Lossless is a comparably simple codec; it is basically a PNG with more transformation methods and custom entropy coding. So I believe it is probably more doable to rewrite in Wuffs than other codecs. The entirety of WebP has a lot more allocations though, a brief skim through libwebp shows various calls to malloc/calloc for, e.g. I/O buffering (src/dec/io_dec.c among others), alpha plane (src/dec/alpha_dec.c), color map and transformation (src/dec/vp8l_dec.c), gradient smoothing (src/utils/quant_levels_dec_utils.c) and so on.
dvhh 3y ago
Looks pretty interesting, I wonder it the result produced by WUFFS transpiled code could be used as drop-in replacement Of course the issue with transpiled code ( same for generated code ), is that is making debugging a lot more difficult.
- tialaramex 3y ago
  Chrome's GIF rendering is already WUFFS, but no, this would not be a "drop-in" replacement because it needs glue. I assume the GIF decoding has glue but you can't "just" reuse it.
  - lifthrasiir 3y ago
    Wait, really? (Searches) Oh indeed [1]. I should note that the glue code is more than 1000 lines of C++ though. Still glad to see that it got into the production. [1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/skia/src/codec/SkWuffsCodec.cpp https://source.chromium.org/chromium/chromium/src/+/main:thi...
    - nigeltao 3y ago
      Wuffs and SkWuffsCodec.cpp author here. Wuffs' GIF decoder has shipped in Google Chrome since July 2021 (milestone M93). A fair chunk out of that 1000 lines of glue code is adapting Wuffs' API to Skia's API. (Skia is the 2-D graphics library used by Chromium and many other projects). Specifically, GIFs can be animated, Wuffs' animation API is designed for sequential access and its state needs O(1) memory but Skia's SkCodec animation API allows random access and needs O(N) memory, where N is the number of animation frames. Random access means that, after decoding frame 100, the SkCodec can rewind and produce frame 70 (by scanning backwards through its O(N) state to find the most recent I-Frame equivalent that's <= 70 and replaying forward from there). Random access seems a bit of a weird feature to me, but Chrome/Skia's old GIF codec could do it, for whatever historical reasons, so the new Wuffs-backed one does too (even though it needed a chunk of glue code).
  - dvhh 3y ago
    As pointed out by another commenter, WUFFS would not itself allocate memory, which is fine, but would need some additional help to perform that task

lifthrasiir 3y ago

I agree that Wuffs [1] would have been a very good alternative! ...if it can be made more general. AFAIK Wuffs is still very limited, in particular it never allows dynamic allocation. Many formats, including those supported by Wuffs the library, need dynamic allocation, so Wuffs code has to be glued with unverified non-Wuffs code [2]. This approach only works with simpler formats. [1] https://github.com/google/wuffs/blob/main/doc/wuffs-the-language.md https://github.com/google/wuffs/blob/main/doc/wuffs-the-lang... [2] https://github.com/google/wuffs/blob/main/doc/note/memory-safety.md#allocation-free-apis https://github.com/google/wuffs/blob/main/doc/note/memory-sa...

tialaramex 3y ago
To me it looks like more generality is neither necessary nor desirable. It's better that the codec is 100% safe but there's a tiny amount of unsafe glue code, than make the codec unsafe to add generality. Google has smart people, finding the bug in 15 lines of glue code before it goes out the door is definitely easier than finding this C bug. This is the same thing that powers Rust successfully, but taking it further.
- lifthrasiir 3y ago
  > It's better that the codec is 100% safe but there's a tiny amount of unsafe glue code, [...] I meant that many codec cannot be easily made in such way because memory allocation can occur here and there.
  - tialaramex 3y ago
    > many codec cannot be easily made in such way because memory allocation can occur here and there. This is an extremely vague complaint and thus suspicious. Of course we could imagine an image format which decides to allow you to declaratively construct cloud servers which transmit XML and so it needs DRM to protect your cloud service credentials - but I claim (and I feel like most people will agree) the fact WUFFS can't do that is a good thing and we should not use this hypothetical "image" format aka massive security hole. Try specifics. This is a WebP bug. For a WebP codec, where does it need memory allocation? My guess is it does allocations only during a table creation step, and after it figures out how big the final image is. So, twice, in specific parts of the code, like JPEG.
    - michaelt 3y ago
      If you want to open an image file of unknown resolution, you might want the library to allocate and return a byte array matching the resolution of the image. Oh, you could design a more complex API to avoid that - but I'm a programmer of very average ability, ask me to juggle too many buffers and struts and I'll probably introduce a buffer overflow by mistake.
    - lifthrasiir 3y ago
      WebP Lossless is a comparably simple codec; it is basically a PNG with more transformation methods and custom entropy coding. So I believe it is probably more doable to rewrite in Wuffs than other codecs. The entirety of WebP has a lot more allocations though, a brief skim through libwebp shows various calls to malloc/calloc for, e.g. I/O buffering (src/dec/io_dec.c among others), alpha plane (src/dec/alpha_dec.c), color map and transformation (src/dec/vp8l_dec.c), gradient smoothing (src/utils/quant_levels_dec_utils.c) and so on.

dvhh 3y ago

Looks pretty interesting, I wonder it the result produced by WUFFS transpiled code could be used as drop-in replacement Of course the issue with transpiled code ( same for generated code ), is that is making debugging a lot more difficult.

tialaramex 3y ago
Chrome's GIF rendering is already WUFFS, but no, this would not be a "drop-in" replacement because it needs glue. I assume the GIF decoding has glue but you can't "just" reuse it.
- lifthrasiir 3y ago
  Wait, really? (Searches) Oh indeed [1]. I should note that the glue code is more than 1000 lines of C++ though. Still glad to see that it got into the production. [1] https://source.chromium.org/chromium/chromium/src/+/main:third_party/skia/src/codec/SkWuffsCodec.cpp https://source.chromium.org/chromium/chromium/src/+/main:thi...
  - nigeltao 3y ago
    Wuffs and SkWuffsCodec.cpp author here. Wuffs' GIF decoder has shipped in Google Chrome since July 2021 (milestone M93). A fair chunk out of that 1000 lines of glue code is adapting Wuffs' API to Skia's API. (Skia is the 2-D graphics library used by Chromium and many other projects). Specifically, GIFs can be animated, Wuffs' animation API is designed for sequential access and its state needs O(1) memory but Skia's SkCodec animation API allows random access and needs O(N) memory, where N is the number of animation frames. Random access means that, after decoding frame 100, the SkCodec can rewind and produce frame 70 (by scanning backwards through its O(N) state to find the most recent I-Frame equivalent that's <= 70 and replaying forward from there). Random access seems a bit of a weird feature to me, but Chrome/Skia's old GIF codec could do it, for whatever historical reasons, so the new Wuffs-backed one does too (even though it needed a chunk of glue code).
- dvhh 3y ago
  As pointed out by another commenter, WUFFS would not itself allocate memory, which is fine, but would need some additional help to perform that task