7 ms·
Firefox will consider a Rust implementation of JPEG-XL
- hencoappel 2y ago> 100k lines of multithreaded C++ Just for a JPEG library. Damn!
- thrdbndndn 2y agoI'm also surprised by this number. Granted it's not just old JPEG standard, still 100k sounds like a lot for a decoder.
- archerx 2y agoThere must be a lot of edge cases that were discovered over the years.
- pixelesque 2y agoIt's the encoder as well, plus converter code for things like JPEG reading as well, and JPEG XL supports animation...
- thrdbndndn 2y agoI guess the original comment in TFA is a little bit misleading, then. > of the reference decoder (currently behind a pref in Firefox Nightly), which weighs in at more than 100,000 lines of multithreaded C++
- lonjil 2y agoThat number includes all the different extra tools in the repo, and all tests. And as mentioned, the encoder.
- bdemirkir 2y agoI don't know where that line count comes from. It looks like libjxl has 38314 lines (https://app.codecov.io/gh/libjxl/libjxl/tree/main/lib%2Fjxl https://app.codecov.io/gh/libjxl/libjxl/tree/main/lib%2Fjxl) at the moment.
- JyrkiAlakuijala 2y agoand that 38 kloc includes the encoder, psychovisual model, etc. etc. that are not needed for the decoder current decoder is around 20 kloc
- lifthrasiir 2y agoI think Highway is a critical component for libjxl and has to be counted towards the line count. (CMS can be ignored here, and any Rust implementation should ideally use qcms [1] which is already in use in Firefox anyway.) [1] https://github.com/FirefoxGraphics/qcms https://github.com/FirefoxGraphics/qcms
- janwas 2y agoIndeed, though Highway is approaching a system library at this point. It is also a thin wrapper over the compiler's intrinsics headers: e.g. ~16KLOC for arm_neon.h, times the number of targets. That's a lot of code, but it is not comparable with actual codec logic lines.
- JyrkiAlakuijala 2y agoPersonally, I'm on the edge on this one. I think both viewpoints are valid. One way to think of Highway is that it is portable multi-platform SIMD intrinsics for C++. While we developed it originally as a part of JPEG XL, it has long ago graduated into a general-purpose library that has various uses, including the recent Gemma.cpp ML launch. Other modern highway uses include: Audio, Browsers, Computational biology, Computer graphics, Cryptography, Grok JPEG 2000, JPEGenc, Jpegli, OpenHTJ2K, Image processing, Image viewers, Information retrieval, Machine learning, Numpy, and Robotics... (copied from: https://github.com/google/highway https://github.com/google/highway) I derived the name from the CityHash, FarmHash, and then HighwayHash series, and considered that Highway would link this library to its roots in the HighwayHash (of course much is also based on Jan's previous work with SIMD). Notably, I resisted using the -li naming here :-D
- justmarc 2y agoWhy just consider? Hell yes. Anything that has remotely something to do with user-supplied data should be done in Rust, especially for such a high profile piece of software.
- amelius 2y agoRust adds a lot of complexity (a new language) and does not even offer a general approach to automated correctness verification (not saying that C++ is better in this regard). Maybe it is just better to wait for Rust++.
- SahAssar 2y agoRust is already in the firefox codebase (hell, it was basically invented to be used in firefox) so it doesn't add any complexity.
- dwattttt 2y agoRust is not a new language for Firefox, some of Firefox is already written in Rust.
- pornel 2y agoRust is much better suited to static analysis and formal verification than C++, primarily thanks to stricter pointer aliasing, immutability without loopholes of const, and thread safety annotations in the type system. Most things that are advanced correctness verification in C++ are regular compiler errors in Rust.
- deleted 2y ago[deleted]
- Omin 2y agoThey are not on the fence about Rust. They are on the fence about JPEG-XL when it entails a major increase in attack surface due to the current implementation being in C++.
- 2y ago
- kookamamie 2y ago> mozilla locked as too heated and limited conversation to collaborators. I wonder which one was objected to - the implementation considered to be in Rust or that a rewrite might not actually make sense for the dying browser (2.75% share) at this stage?
- lost_womble 2y agoFirefox is around 8% of market share in the US on non-mobile devices -- up from around 4.5% this time last year. It's downright wrong to see 80% increase in users over 12 months as 'dying' -- and in general 1 in 12 users is significant enough that you should be caring about it.
- Y_Y 2y agoCommenters seen to love declaring things to be dying, and marketshare trends are only tangentially related. (see e.g. "Netcraft confirms it" meme) I personally feel like softeare is dead once it is no longer useful, and there is no likelihood of that changing in the foreseeable future. That's obviously not the case woth Firefox. On the other hand I've seen exaggerated reports of the demise of a project that hasn't had a git commit in the last week, or isn't commercially relevant for whatever the commenter is interested in. Even this doesn't seem to me to be true for Firefox, but I don't think the accusations will stop, or the signal-to-noise will increase.
- illiac786 2y agoThis is basic trolling, you’re making Mozilla’s point here…
- Timber-6539 2y agoWhat does Google get out of this?
- greener_grass 2y agoThey have a general interest in the quality of the web. More web use is good for Google Ads. This sometimes comes into tension with Google's attempts to control and monetize the web.
- arp242 2y agoAnd at "Google scale" shaving off a few bytes on every request means saving real money.
- rapsey 2y agoDoes bandwidth actually cost Google anything?
- throw0101c 2y ago> Does bandwidth actually cost Google anything? Latency can cost monetization. Fewer bits at X b/s is fewer seconds.
- Hendrikto 2y agoAt the datacenter scale, bandwidth is expensive, yes.
- arp242 2y agoIt's not free, no. And servers aren't free either. At "Google scale" 1% or 2% savings can add up to hundreds of thousands or millions of dollars per year. 1% of 100,000 servers is 1,000 servers.
- JyrkiAlakuijala 2y agoMy take on this is that junior engineers optimize for the cpu as it is easy to measure, senior engineers for the user experienced latency as it matters more. Users know that their time is valuable. If your website is fast, they will explore it, and came back soon.
- BoingBoomTschak 2y agoReminder that https://github.com/google/wuffs https://github.com/google/wuffs exists too.
- Vecr 2y agoMight be too much effort if the original is 100 kilolines.
- fuck_google 2y ago[dead]
- lifthrasiir 2y agoIt will be fairly challenging to rewrite libjxl or any other independent JPEG XL decoder into wuffs, due to its inherent (and for now necessary) limitations. Even its PNG decoder is not very traditional, and JPEG XL has way more bookkeeping in its format.
- BoingBoomTschak 2y agoYeah, probably, just thought it worth mentioning since it was literally made by Google for use within Chromium.
- timeon 2y agoI'm already covered by rust, but thanks for the link it is interesting. I was not aware of wuff.
- user36188 2y agoRust is the savior of Firefox.
- _Amrinder 2y agoif only it had no telemetry. Try opening wireshark with firefox running and you will see how much of a noisyboi it is
- pshirshov 2y agoTelemetry can be disabled by a couple of config entries.
- hu3 2y agoYou're making parent commenter's point stronger. So not only it is on by default, but it also requires a couple of config entries to disable.
- pshirshov 2y agoStill, if you are unhappy with it, you have a way to opt out. Unlike in many other cases.
- illiac786 2y agoDon’t see how. If there is too much telemetry, there is only two options: it can or it cannot be disabled via config. Very unlikely someone complaining about telemetry opted into telemetry on purpose… hence, the fact it can be disabled is “good news in the bad news”.
- gardaani 2y ago> the team at Google has agreed to apply their subject matter expertise to build a safe, performant, compact, and compatible JPEG-XL decoder in Rust There's already a JPEG-XL decoder written in Rust: https://crates.io/crates/jxl-oxide https://crates.io/crates/jxl-oxide It would be nice to hear why it's not good enough.
- tux3 2y agoThis is the same Google that wrote the libgav1 decoder instead of using dav1d, they don't have a strong history of starting from code that was invented elsewhere. (In the case of AV1 I think they eventually gave up and started offering dav1d on Android, but not before shipping their in-house implementation that was much less efficient and no safer than state of the art, perplexing everyone in the process.)
- lifthrasiir 2y agoIt should be noted that all known independent JPEG XL decoders (especially my J40 [1] and jxl-oxide) were primarily written by a single individual for each library. So the request is that some library, either the future version of jxl-oxide or Rust rewrite of libjxl or something else, should be written in a provably memory-safe manner, actively maintained by JPEG XL developers, and successfully reproducing the original libjxl's decoding performance. I think it is indeed a fair deal to both Mozilla and JPEG XL developers. [1] The main blocker for J40 was that I don't really want to keep it less safe than I hope to achieve, but I also want to keep it in C for practical reasons. This and my daily job prevented any significant improvements so far.
- pchangr 2y agoWell, blink did come out of WebKit ..
- Timshel 2y agoThey are relatively neutral about the format, so I'm guessing they want a commitment from Google to maintain it. As good as Oxide might be it looks like a one developer project.
- 2y ago
- pornel 2y agoThis is a great news for JPEG XL. Web standards need two independent implementations of every feature – this proves that the spec is actually possible to implement, and makes it possible to verify that the implementations are interoperable. It makes uses in the wild much less likely to depend on bugs in a particular implementation, which makes it possible to upgrade or replace implementations without creating bug-compatibility problems.
- joshlk 2y agoI’m not sure this holds as true if it’s the same team creating both implementations.
- pornel 2y agoIt's sufficient. Another implementation is still unlikely to have the exact same bugs. Especially rewrite in Rust will force the code to be structured differently (Rust is very opinionated about that). The spec is big enough that the team won't be able to just write the exact same implementation from memory.
- zcorpan 2y agoI don't disagree that it's sufficient, but also, ideally different people would implement the spec. If you have a particular mental model or understanding of a part of the spec that doesn't match what the spec actually says, that is likely to translate identically when writing a second implementation.
- JyrkiAlakuijala 2y agoI believe that there are already several decoder implementations that pass the ISO conformance test for JPEG XL. I don't remember which one's exactly but I believe at least both jxl-oxide and libjxl do.
- leduyquang753 2y agoThat sentiment is just so unnecessary but oh well Rust is literally their brain child so.
- rapsey 2y agoLibpng and libjpeg have been huge sources oh high severity security vulnerabilities. It is not unnecessary at all.
- sharpshadow 2y ago[flagged]
- deleted 2y ago[deleted]
- pogue 2y agoCan someone summarize why JPEG-XL is so controversial to browser devs? Why not just allow as many image standards as possible?
- rowbin 2y agoBecause of the increased attack surface and maintenance burden. Which is totally understandable. But JPEG-XL should really be supported...