9 ms·

I rewrote Firefox's BMP decoder

148 points by nnethercote 11y ago

885895 11y ago
Interesting write-up. Still, why even support BMP?
- Natanael_L 11y ago
  Backwards compatibility. There's too many of them around
- chkuendig 11y ago
  https://en.wikipedia.org/wiki/ICO_%28file_format%29 https://en.wikipedia.org/wiki/ICO_%28file_format%29 is used for favicons and can contain BMP.
- q3k 11y ago
  Why not? It's an old, well-supported, relatively-well-understood, simple to emit file format that Just Works. It's the lowest common denominator for 24-bit graphics interchange. The only real alternative to it (== simple uncompressed bitmaps) that I can think of are .tga files, but these haven't penetrated the Internet as well as BMP has.
  - IshKebab 11y ago
    Why would you ever need uncompressed bitmaps? Also - all code exposes security risks. Especially image decoders written in C/C++.
    - to3m 11y ago
      Some libraries output them, because the code for writing them is very simple, and you don't need any separate area of memory for managing the compression or whatever.
    - chriswarbo 11y ago
      > Why would you ever need uncompressed bitmaps? Because they're incredibly easy to generate programatically. For example, here's a bunch of experiments I did to test the code and image embedding process for my site: http://chriswarbo.net/essays/procedural http://chriswarbo.net/essays/procedural The snippets of Haskell code on those pages are executed when the page's Markdown is rendered, and the images are the results. To make this work, each function maps x and y pixel coordinates to either a Bool (for b/w), an Int (for greyscale) or an (Int, Int, Int) tuple (for RGB). These are trivially converted into strings of PPM image data (via [1]), rendered to PNG (via [2]) and embedded into the page as a data URI (via [3]). The "view source" links show all the code, although it's a bit convoluted ;) Whilst this example is just for experimentation, I could image someone generating raw bitmaps in a monitoring situation, for example. [1] http://chriswarbo.net/git/chriswarbo-net/branches/master/static/procedural/Pic.hs.raw.html http://chriswarbo.net/git/chriswarbo-net/branches/master/sta... [2] http://chriswarbo.net/git/chriswarbo-net/branches/master/static/procedural/includePic.raw.html http://chriswarbo.net/git/chriswarbo-net/branches/master/sta... [3] http://chriswarbo.net/git/chriswarbo-net/branches/master/static/file2img.sh.raw.html http://chriswarbo.net/git/chriswarbo-net/branches/master/sta...
      - eru 11y ago
        Off-topic: I produced some similar-ish visualisations (also with Haskell). http://paquari.com/qsort2000.png http://paquari.com/qsort2000.png Quicksort of 2000 random elements, (x,y) is black iff the algorithm compares x and y. You can clearly see how the pivots get compared to all the other elements. http://paquari.com/msortOrig2000.png http://paquari.com/msortOrig2000.png A similar picture for merge-sort, but here (x,y) is black iff the algorithm compares the elements at original position x and y. http://paquari.com/msort2000.png http://paquari.com/msort2000.png Mergesort, but colouring like in the Quicksort case.
    - okasaki 11y ago
      It's much faster to write. So it's useful for e.g. game screenshots.
- chriswarbo 11y ago
  I think a more pertinent question would be: why have the browser decoding images at all, rather than loading a dedicated library? The points mentioned in the article about streaming data and untrustworthy images make sense, but would be useful even outside the context of a browser rendering engine (e.g. many local image files will have come from the Web, so are just as untrustworthy).
  - i229718 11y ago
    Less dependencies usually means less bloat.
  - amelius 11y ago
    > why have the browser decoding images at all, rather than loading a dedicated library? Because a dedicated library could change its API in version 2.0, and at the same time fix important security flaws. At that point your only options are: switch to a different library, write your own library, put the old API back in the original library, or ask the maintainers of the library. All of those options are difficult or unreliable, especially given the small timeframe in which it all should happen.
    - chriswarbo 11y ago
      > Because a dedicated library could change its API in version 2.0, and at the same time fix important security flaws. The correct thing to do is back-port fixes to the 1.x branch, or come up with an alternative fix if the 1.x/2.x transition changes too much (in the latter case, the 1.x and 2.x versions would essentially be different libraries which just-so-happen to share the same name). Anyone can (attempt to) do this patch, including the library authors, the browser authors (who may be the same people), or any other users of the library. If upstream don't accept such patches, and repeatedly indulge in such uncooperative behaviour, there is always the option to fork (and, in the process, perhaps strip out the parts which the browser doesn't need to make maintenance easier). As an aside, the situation you describe sounds a lot like the Firefox/Iceweasel drama in Debian!
      - 11y ago
- phoboslab 11y ago
  .ico files (as used for favicons) typically contain BMPs.
- dspillett 11y ago
  From the article: > On the web it’s mostly used on the web for favicons though it can be used for normal images. Basically the .ico format often used for favicons comes from Windows icons (the first favicon implementation originated in Internet Explorer and MS supported .ico so people could reuse existing application icon files, though IIRC .gif files and other formats supported in web pages could be used from the start too) which are a variant of the .bmp format which has a few extra features like containing different images for different sizes (so a single file can contain separately optimised 16x16 and 32x32 bitmaps). I assume .ico and .bmp files are fairly rare now, presumably completely unheard of for new sites/apps, but you can't drop support because there are enough of them out there on legacy sites that it would break things people care about.
  - deleted 11y ago
    [deleted]
  - gdwatson 11y ago
    IE only supported .ico favicons for a long time.
    - dspillett 11y ago
      I've just looked it up (assuming https://en.wikipedia.org/wiki/Favicon#File_format_support https://en.wikipedia.org/wiki/Favicon#File_format_support is an accurate reference here) and was surprised that it seems IE11 was the first to support other formats. I have kept our icons in .ico format but I thought that was to support proper legacy IE (5 and below). Where I have used other formats I presumably didn't notice the icon not working in IE8/9/10 because I generally only care about Firefox & Chrome and worry about IE10/11 breaking when someone reports a problem [if someone reports a problem in IE prior to 10 outside of my day job (where I have to support IE8 for some rather backward clients) I consider that their problem because of their browser choice, and even 10 is rapidly dropping off my care radar].
- noselasd 11y ago
  I'd like to see a breakdown of file types at e.g. imgur.com - I'm guessing more than 0% are bmp images, and there isn't a compelling reason for a browser to not be able to view those.
- mhw 11y ago
  Good question - it's obviously still widely enough used that the need to decode the format lives on. Many years ago I wrote the code to support PBM/PGM/PPM images for Mozilla (https://bugzilla.mozilla.org/show_bug.cgi?id=117983 https://bugzilla.mozilla.org/show_bug.cgi?id=117983). They never got that much traction on the web though, and so support for that image format was dropped: <https://bugzilla.mozilla.org/show_bug.cgi?id=197530> https://bugzilla.mozilla.org/show_bug.cgi?id=197530>.
- blt 11y ago
  Because some old websites serve BMP images?
heinrich5991 11y ago
I kind of hoped that this would be written in Rust. :)
- nnethercote 11y ago
  There's still a bunch of build system issues to be solved before any Rust code can be included in Firefox. You can follow along at https://bugzilla.mozilla.org/show_bug.cgi?id=1135640 https://bugzilla.mozilla.org/show_bug.cgi?id=1135640. Note especially the two bugs listed in the "Blocks" field, which are the first two candidates for components to be rewritten once the build system issues are sorted out. (The second of these would replace the notoriously insecure libstagefright with a Rust alternative.) Piston has a BMP decoder, and my understanding is that it is the one used by Servo. I looked at it briefly and I think its support for the dark corners of the BMP format would be even worse than Firefox's would be before my rewrite. See https://github.com/PistonDevelopers/image/tree/master/src/bmp https://github.com/PistonDevelopers/image/tree/master/src/bm... for the code.
  - steveklabnik 11y ago
    This is true, and we're actively working on those build system issues.
  - pcwalton 11y ago
    > There's still a bunch of build system issues to be solved before any Rust code can be included in Firefox. You can follow along at https://bugzilla.mozilla.org/show_bug.cgi?id=1135640 https://bugzilla.mozilla.org/show_bug.cgi?id=1135640. It seems unfortunate to be maintaining two BMP decoders because of transient build system issues. Especially if we're going to have to rewrite Piston's, making us maintain three for a time. I understand if there was some pressing need to ship it now, of course, but other than that I think I would have preferred this be written in Rust as well. I guess it's not a big deal for 1,000 lines of code, though. We could probably do a line for line translation without much effort.
    - nnethercote 11y ago
raverbashing 11y ago
BMP was one of the last formats that would be possible to "decode manually" (that is, read the actual data and paint the image from it). PCX was one of the other (using RLE) Sure, easier to use a library, but sometimes you have some constraints
- mnw21cam 11y ago
  Ever heard of the Netpbm image format? https://en.wikipedia.org/wiki/Netpbm_format https://en.wikipedia.org/wiki/Netpbm_format
  - raverbashing 11y ago
    Yes, but they're not very commonplace in 'real usages'
    - dr_zoidberg 11y ago
      It depends. A friend of mine works with PAM[1] with satellite images. That counts as real enough for me. [1] https://en.wikipedia.org/wiki/Netpbm#PAM_graphics_format https://en.wikipedia.org/wiki/Netpbm#PAM_graphics_format
    - pmelendez 11y ago
      They are actually very common depending on what you are doing. For instance, if you are looking for public image datasets for face recognition you would find that the file format is being used a lot.
- woofyman 11y ago
  TARGA files are also easy to read and write. BMPs are annoying since each scan line needs to be long word aligned.
  - angersock 11y ago
    Seconding this. TGA also has the nicest file spec I've ever read: http://www.dca.fee.unicamp.br/~martino/disciplinas/ea978/tgaffs.pdf http://www.dca.fee.unicamp.br/~martino/disciplinas/ea978/tga...
fazkan 11y ago
How much of the mozillas code is written in C/C++, and for what purpose...
- deleted 11y ago
  [deleted]

148 points by nnethercote 11y ago

885895 11y ago

Interesting write-up. Still, why even support BMP?

Natanael_L 11y ago
Backwards compatibility. There's too many of them around
chkuendig 11y ago
https://en.wikipedia.org/wiki/ICO_%28file_format%29 https://en.wikipedia.org/wiki/ICO_%28file_format%29 is used for favicons and can contain BMP.
q3k 11y ago
Why not? It's an old, well-supported, relatively-well-understood, simple to emit file format that Just Works. It's the lowest common denominator for 24-bit graphics interchange. The only real alternative to it (== simple uncompressed bitmaps) that I can think of are .tga files, but these haven't penetrated the Internet as well as BMP has.
- IshKebab 11y ago
  Why would you ever need uncompressed bitmaps? Also - all code exposes security risks. Especially image decoders written in C/C++.
  - to3m 11y ago
    Some libraries output them, because the code for writing them is very simple, and you don't need any separate area of memory for managing the compression or whatever.
  - chriswarbo 11y ago
    > Why would you ever need uncompressed bitmaps? Because they're incredibly easy to generate programatically. For example, here's a bunch of experiments I did to test the code and image embedding process for my site: http://chriswarbo.net/essays/procedural http://chriswarbo.net/essays/procedural The snippets of Haskell code on those pages are executed when the page's Markdown is rendered, and the images are the results. To make this work, each function maps x and y pixel coordinates to either a Bool (for b/w), an Int (for greyscale) or an (Int, Int, Int) tuple (for RGB). These are trivially converted into strings of PPM image data (via [1]), rendered to PNG (via [2]) and embedded into the page as a data URI (via [3]). The "view source" links show all the code, although it's a bit convoluted ;) Whilst this example is just for experimentation, I could image someone generating raw bitmaps in a monitoring situation, for example. [1] http://chriswarbo.net/git/chriswarbo-net/branches/master/static/procedural/Pic.hs.raw.html http://chriswarbo.net/git/chriswarbo-net/branches/master/sta... [2] http://chriswarbo.net/git/chriswarbo-net/branches/master/static/procedural/includePic.raw.html http://chriswarbo.net/git/chriswarbo-net/branches/master/sta... [3] http://chriswarbo.net/git/chriswarbo-net/branches/master/static/file2img.sh.raw.html http://chriswarbo.net/git/chriswarbo-net/branches/master/sta...
    - eru 11y ago
      Off-topic: I produced some similar-ish visualisations (also with Haskell). http://paquari.com/qsort2000.png http://paquari.com/qsort2000.png Quicksort of 2000 random elements, (x,y) is black iff the algorithm compares x and y. You can clearly see how the pivots get compared to all the other elements. http://paquari.com/msortOrig2000.png http://paquari.com/msortOrig2000.png A similar picture for merge-sort, but here (x,y) is black iff the algorithm compares the elements at original position x and y. http://paquari.com/msort2000.png http://paquari.com/msort2000.png Mergesort, but colouring like in the Quicksort case.
  - okasaki 11y ago
    It's much faster to write. So it's useful for e.g. game screenshots.
chriswarbo 11y ago
I think a more pertinent question would be: why have the browser decoding images at all, rather than loading a dedicated library? The points mentioned in the article about streaming data and untrustworthy images make sense, but would be useful even outside the context of a browser rendering engine (e.g. many local image files will have come from the Web, so are just as untrustworthy).
- i229718 11y ago
  Less dependencies usually means less bloat.
- amelius 11y ago
  > why have the browser decoding images at all, rather than loading a dedicated library? Because a dedicated library could change its API in version 2.0, and at the same time fix important security flaws. At that point your only options are: switch to a different library, write your own library, put the old API back in the original library, or ask the maintainers of the library. All of those options are difficult or unreliable, especially given the small timeframe in which it all should happen.
  - chriswarbo 11y ago
    > Because a dedicated library could change its API in version 2.0, and at the same time fix important security flaws. The correct thing to do is back-port fixes to the 1.x branch, or come up with an alternative fix if the 1.x/2.x transition changes too much (in the latter case, the 1.x and 2.x versions would essentially be different libraries which just-so-happen to share the same name). Anyone can (attempt to) do this patch, including the library authors, the browser authors (who may be the same people), or any other users of the library. If upstream don't accept such patches, and repeatedly indulge in such uncooperative behaviour, there is always the option to fork (and, in the process, perhaps strip out the parts which the browser doesn't need to make maintenance easier). As an aside, the situation you describe sounds a lot like the Firefox/Iceweasel drama in Debian!
    - 11y ago
phoboslab 11y ago
.ico files (as used for favicons) typically contain BMPs.
dspillett 11y ago
From the article: > On the web it’s mostly used on the web for favicons though it can be used for normal images. Basically the .ico format often used for favicons comes from Windows icons (the first favicon implementation originated in Internet Explorer and MS supported .ico so people could reuse existing application icon files, though IIRC .gif files and other formats supported in web pages could be used from the start too) which are a variant of the .bmp format which has a few extra features like containing different images for different sizes (so a single file can contain separately optimised 16x16 and 32x32 bitmaps). I assume .ico and .bmp files are fairly rare now, presumably completely unheard of for new sites/apps, but you can't drop support because there are enough of them out there on legacy sites that it would break things people care about.
- deleted 11y ago
  [deleted]
- gdwatson 11y ago
  IE only supported .ico favicons for a long time.
  - dspillett 11y ago
    I've just looked it up (assuming https://en.wikipedia.org/wiki/Favicon#File_format_support https://en.wikipedia.org/wiki/Favicon#File_format_support is an accurate reference here) and was surprised that it seems IE11 was the first to support other formats. I have kept our icons in .ico format but I thought that was to support proper legacy IE (5 and below). Where I have used other formats I presumably didn't notice the icon not working in IE8/9/10 because I generally only care about Firefox & Chrome and worry about IE10/11 breaking when someone reports a problem [if someone reports a problem in IE prior to 10 outside of my day job (where I have to support IE8 for some rather backward clients) I consider that their problem because of their browser choice, and even 10 is rapidly dropping off my care radar].
noselasd 11y ago
I'd like to see a breakdown of file types at e.g. imgur.com - I'm guessing more than 0% are bmp images, and there isn't a compelling reason for a browser to not be able to view those.
mhw 11y ago
Good question - it's obviously still widely enough used that the need to decode the format lives on. Many years ago I wrote the code to support PBM/PGM/PPM images for Mozilla (https://bugzilla.mozilla.org/show_bug.cgi?id=117983 https://bugzilla.mozilla.org/show_bug.cgi?id=117983). They never got that much traction on the web though, and so support for that image format was dropped: <https://bugzilla.mozilla.org/show_bug.cgi?id=197530> https://bugzilla.mozilla.org/show_bug.cgi?id=197530>.
blt 11y ago
Because some old websites serve BMP images?

heinrich5991 11y ago

I kind of hoped that this would be written in Rust. :)

nnethercote 11y ago
There's still a bunch of build system issues to be solved before any Rust code can be included in Firefox. You can follow along at https://bugzilla.mozilla.org/show_bug.cgi?id=1135640 https://bugzilla.mozilla.org/show_bug.cgi?id=1135640. Note especially the two bugs listed in the "Blocks" field, which are the first two candidates for components to be rewritten once the build system issues are sorted out. (The second of these would replace the notoriously insecure libstagefright with a Rust alternative.) Piston has a BMP decoder, and my understanding is that it is the one used by Servo. I looked at it briefly and I think its support for the dark corners of the BMP format would be even worse than Firefox's would be before my rewrite. See https://github.com/PistonDevelopers/image/tree/master/src/bmp https://github.com/PistonDevelopers/image/tree/master/src/bm... for the code.
- steveklabnik 11y ago
  This is true, and we're actively working on those build system issues.
- pcwalton 11y ago
  > There's still a bunch of build system issues to be solved before any Rust code can be included in Firefox. You can follow along at https://bugzilla.mozilla.org/show_bug.cgi?id=1135640 https://bugzilla.mozilla.org/show_bug.cgi?id=1135640. It seems unfortunate to be maintaining two BMP decoders because of transient build system issues. Especially if we're going to have to rewrite Piston's, making us maintain three for a time. I understand if there was some pressing need to ship it now, of course, but other than that I think I would have preferred this be written in Rust as well. I guess it's not a big deal for 1,000 lines of code, though. We could probably do a line for line translation without much effort.
  - nnethercote 11y ago

raverbashing 11y ago

BMP was one of the last formats that would be possible to "decode manually" (that is, read the actual data and paint the image from it). PCX was one of the other (using RLE) Sure, easier to use a library, but sometimes you have some constraints

mnw21cam 11y ago
Ever heard of the Netpbm image format? https://en.wikipedia.org/wiki/Netpbm_format https://en.wikipedia.org/wiki/Netpbm_format
- raverbashing 11y ago
  Yes, but they're not very commonplace in 'real usages'
  - dr_zoidberg 11y ago
    It depends. A friend of mine works with PAM[1] with satellite images. That counts as real enough for me. [1] https://en.wikipedia.org/wiki/Netpbm#PAM_graphics_format https://en.wikipedia.org/wiki/Netpbm#PAM_graphics_format
  - pmelendez 11y ago
    They are actually very common depending on what you are doing. For instance, if you are looking for public image datasets for face recognition you would find that the file format is being used a lot.
woofyman 11y ago
TARGA files are also easy to read and write. BMPs are annoying since each scan line needs to be long word aligned.
- angersock 11y ago
  Seconding this. TGA also has the nicest file spec I've ever read: http://www.dca.fee.unicamp.br/~martino/disciplinas/ea978/tgaffs.pdf http://www.dca.fee.unicamp.br/~martino/disciplinas/ea978/tga...

fazkan 11y ago

How much of the mozillas code is written in C/C++, and for what purpose...

deleted 11y ago
[deleted]