14 ms·

Eliminating Data Races in Firefox

470 points by caution 5y ago

est31 5y ago
> Overall Rust appears to be fulfilling one of its original design goals: allowing us to write more concurrent code safely. Both WebRender and Stylo are very large and pervasively multi-threaded, but have had minimal threading issues. What issues we did find were mistakes in the implementations of low-level and explicitly unsafe multithreading abstractions — and those mistakes were simple to fix. > This is in contrast to many of our C++ races, which often involved things being randomly accessed on different threads with unclear semantics, necessitating non-trivial refactorings of the code.
- chowells 5y ago
  Yeah, that was the part that stood out to me as well. I thought it was pretty intuitively obvious that having your data races confined to small segments is better than allowing them to be spread over the entire codebase, but a lot of people act like it negates the advantage of working in a safer language entirely. It's nice to have empirical reports that back up the "obvious" though. Sometimes "obvious" things turn out to be wrong. It's always good to check them.
  - setr 5y ago
    > but a lot of people act like it negates the advantage of working in a safer language entirely. I think the underlying assumption is more along the lines of: the easy cases are made easier, and the hard cases are made impossible — or rather the concern being that rust hasn’t made the problem easier, it just moved it around. Which is often the case with claims of solving long-lasting problems
    - kibwen 5y ago
      > the concern being that rust hasn’t made the problem easier, it just moved it around. This is largely the point of Rust, though. Rust takes problems that used to manifest at runtime and causes them to manifest at compile-time, shortening the feedback loop drastically. It also takes problems that used to manifest all over the codebase and roots their causes in the leaf nodes of the program that are making use of `unsafe` blocks. Moving around the problem turns out to be a valuable way of addressing the problem, without ignoring any of the essential complexity or making anything fundamentally impossible.
      - wtetzner 5y ago
        > shortening the feedback loop drastically More importantly, forcing you to catch them. Problems at runtime don't get caught until that condition actually occurs during an actual run of the software.
    - pdpi 5y ago
      Not all forms of moving things around are equivalent. Some just keep the problem as diffuse as it was before, others restrict the problem to a smaller area. Rust does not, by any means, solve this problem. What it does do is help you prove that the problem is contained to a smaller subset of your codebase that you can audit/reason about much more easily.
    - chriswarbo 5y ago
      > Which is often the case with claims of solving long-lasting problems This is what I really like about programming language theory: problems which are literally undecidable in existing languages, can be made almost trivial by designing a language with that problem in mind. The trick is to have the language keep track of the information needed for the solution, so we're not dealing with "black boxes". A classic example is annotating every variable with a type, and propagating that information through compound expressions; lets us spot problems like trying to use a number as a string, without worrying about the halting problem (we do this by forbidding some technically-valid programs, like `1 + (true? 10 : "hello")`). Of course, this leaves the much harder problems of (a) keeping the burden imposed by this extra information down to a reasonable level, and (b) trying to make the language compelling enough for people to actually use. Rust is one of the few examples of going all-in on a new ecosystem, and it actually being embraced. Often it's easier to "embed" a new language inside an existing language, to piggy-back on the existing tooling (e.g. the variety of auto-differentiating languages built in Python)
      - Quekid5 5y ago
        Yes, people often forget that TMs really are "pure runtime". Sure, the program itself is static, but nothing else is. There's no imposed semantics on anything beyond the pure mechanism of the interpreter itself. This lessens the 'problem' part of the Halting Problem drastically if you just impose some constraints on the programs, e.g. static types, maybe a borrow-checker, etc.
    - stouset 5y ago
      > I think the underlying assumption is more along the lines of: the easy cases are made easier, and the hard cases are made impossible — or rather the concern being that rust hasn’t made the problem easier, it just moved it around. Except in this case the article linked demonstrates that they've made things dramatically easier overall. Data races went from being endemic in a C++ codebase to being in a few low-level and easy-to-locate areas. The easy stuff stayed easy, the bulk of the rest was folded into the easy case, and even the hard cases appear to have been made easier to find, identify, and fix.
      - Quekid5 5y ago
        Once you pick away enough at the straws, the needle becomes a lost easier to find.
- kibwen 5y ago
  And when it comes to finding bugs in your unsafe concurrency primitives in Rust programs, there is a tool called Loom that helps with that, as a powerful complement to what ThreadSanitizer provides: https://github.com/tokio-rs/loom https://github.com/tokio-rs/loom . You can think of Loom sort of like QuickCheck, except the permutations are all the possible thread states of your program based on the memory orderings that you have specified. Here's a video that briefly talks about what Loom can do https://youtu.be/rMGWeSjctlY?t=7519 https://youtu.be/rMGWeSjctlY?t=7519 (it's part of a larger video about memory orderings, and is a great introduction to the topic).
  - gamegoblin 5y ago
    Loom is really awesome, though it is focused on exhaustive testing, so not suitable for code that has a lot of possible interleavings (e.g. due to a ton of threads, or a large body of code). There is a new project out of AWS called Shuttle [1] which is like Loom, but it does random exploration instead of exhaustive exploration, which enables massively distributed testing of really complicated stuff. [1] https://github.com/awslabs/shuttle https://github.com/awslabs/shuttle
    - kibwen 5y ago
      Excellent, thanks for the tip! I have to ask though, given a tool that does exhaustive testing (like Loom), shouldn't it be trivial to adapt it to do random testing merely by randomizing rather than enumerating the test inputs? Is there something more going on that I'm not seeing?
    - wyldfire 5y ago
      Is there anything like Loom or Shuttle for C/C++? Or are these only implemented in rust and they'd still work with C/C++?
      - kibwen 5y ago
        I vaguely recall that Loom is an implementation of a research paper that purported to implement this for C/C++, but I don't know if that implementation is available/production-ready. In the worst case it should be relatively straightforward to back-port Loom to C++, since Rust also uses the C memory model.
- evmar 5y ago
  It's interesting, I read this and had the opposite conclusion -- people are still writing racy code using Rust. I mean, it's great that you can write threaded code with more confidence, but it seems the conclusion here is that Rust gives you more confidence but not absolute confidence and you still need the normal stable of runtime detection tooling. The second fix[1] in particular resembles the bad pattern of "sprinkle atomics it until it works" you see in non-Rust languages. (If this comment raises your hackles, please read the "no true Scotsman[2]" fallacy before responding.) [1]: https://hg.mozilla.org/integration/autoland/rev/044f8b594c71 https://hg.mozilla.org/integration/autoland/rev/044f8b594c71 [2]: https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman
  - brundolf 5y ago
    > What issues we did find were mistakes in the implementations of low-level and explicitly unsafe multithreading abstractions (emphasis mine) The important thing is that the races only happened in unsafe { } blocks. It's well-established that these blocks should basically be treated like C++ in terms of scrutiny, but (I believe) you can still have roughly "absolute confidence" in non-unsafe code. It's true and interesting that unsafe { } blocks deserve sanitization analysis like C++ code does - and maybe this hasn't been discussed enough - but I don't think it's fair to suggest that the enormous language benefits are outweighed by some false sense of confidence. The proportions in that idea just seem way out of whack.
    - benschulz 5y ago
      AFAIU one can create races in safe code using atomics with insufficient ordering. (That notwithstanding, I agree that Rust is a vast improvement over C/++.)
      - steveklabnik 5y ago
        "races" is too general. Safe Rust cannot have data races. It can have race conditions.
- WalterBright 5y ago
  D takes a different approach. D has a type constructor `shared`. Only types marked as `shared` (or `immutable`) can be accessed by multiple threads. If you've got concurrency problems, the scope of the problems are immediately reduced to the subset of the code typed as `shared`. In addition, `shared` types cannot be accessed directly. They can only be accessed via library routines, such as D's atomic library. Having an `immutable` data type also helps, as immutable data can be accessed by multiple threads without need for synchronization.
  - eximius 5y ago
    It's not that different. Rust has Sync + Send + &refs and &mut refs.
  - secondcoming 5y ago
    Does this approach have to deal with potential recursive locking?
    - WalterBright 5y ago
      Not by itself. But D's prototype ownership-borrowing system can help with that.
  - d110af5ccf 5y ago
    Did D ever get around to specifying a formal memory model? Because when I tried to adopt D a couple years ago I felt there was a lot of ambiguity surrounding the semantics of `shared`, particularly when interfacing with C and C++ code. I ended up just casting to and from `shared`, and that seemed to work, but it was pretty verbose for highly parallel code and I was never quite sure under what circumstances doing that might violate the compiler's invariants. Also such casting appeared to eliminate most of the benefits, since what appeared to be local data might have been cast and had a pointer sent across to another thread elsewhere. In the end `shared`, @nogc antics (the language internals weren't fully annotated), closures requiring the GC (compare to C++), and the friction of interfacing with highly templated C++ code such as the Eigen library caused me to abandon the attempt. I sure learned a lot in the process though!
- deleted 5y ago
  [deleted]
ChiefOBrien 5y ago
Who knew using already existing data race analysis tools is much more feasible than shoehoning a huge project into a brand new language. Way to go mozilla!
- Jweb_Guru 5y ago
  If you believe that this tool has found all the data races in the C++ code, or even a majority of them, I have a bridge to sell you.
  - ChiefOBrien 5y ago
    Oh come on now. If it was really important for them then why did they chose C++ a long time ago after deciding to rewrite Netscape from scratch? Their history is full of dumb choices and now suddenly going full Rust is the most important thing because... the codebase is lousy? It's not gonna bring back any new/old users, and right now the web needs a healthy marketshare.
    - steveklabnik 5y ago
      > now suddenly going full Rust is the most important thing As said above, Mozilla does not seem to have ever suggested, in the past or now, that "going full Rust" is a goal, let alone "the most important thing."
      - ChiefOBrien 5y ago
        Yeah and Webrender, Stylo, and Servo just happen to appear out of thin air.
        dralley 5y ago
        Firefox has something like 8 million lines of C++. Stylo replaced about 150,000 lines of that, and WebRender is more of a fast path than a true replacement of anything. Both of those rewrites came with massive speedups due to the amount of parallelism they were able to use, which was impractical in C++. Other parts of Firefox are not as critical or parallelizable, hence why they aren't being replaced any time soon. You seem to be quite aggressively misinformed, if you are not deliberately trolling.
wyldfire 5y ago
> What is ThreadSanitizer? ThreadSanitizer (TSan) is compile-time instrumentation to detect data races according to the C/C++ memory model on Linux. It is important to note that these data races are considered undefined behavior within the C/C++ specification. TSan, ASan, UBSan - if you are writing code and your toolchain supports these, you should use them. Over the past 6-7+ years I've used them, I have never seen a false positive (I'm sure some have occasionally existed but they seem rare in practice). > However, developers claimed on various occasions that a particular report must be a false positive. In all of these cases, it turned out that TSan was indeed right and the problem was just very subtle and hard to understand. Yes, I have seen this phenomenon!
- inetknght 5y ago
  Address Sanitizer -- never seen a false positive. Ever. UBSan -- again never had a false positive. Thread Sanitizer does have false positives in some circumstances. They're hard to verify to be falsy and in many ways it's better to refactor to eliminate the complaint because you'll end up with better cleaner code. For example, Qt uses those atomic fences that the article describes [0] [1]. [0]: TSan does not produce false positive data race reports when properly deployed, which includes instrumenting all code that is loaded into the process and avoiding primitives that TSan doesn’t understand (such as atomic fences). [1]: https://lists.qt-project.org/pipermail/interest/2014-March/011481.html https://lists.qt-project.org/pipermail/interest/2014-March/0...
  - nyanpasu64 5y ago
    I've gotten false positives in ASan when trying to instrument a Win32 application code compiled in MSVC using MSVC's then-experimental ASan support, using the wrong combination of the maze of ASan libraries (static library combined with dynamic MFC). And I also get startup crashes when running MSVC ASan binaries in Wine. But it's obscure beta use-cases and some may be fixed by now.
  - derf_ 5y ago
    I have seen false positives in ASAN. In Firefox, media threads use a reduced stack size (128 kB), due to, historically, 32-bit address-space concerns with lots of media elements on a page. Opus, in its default configuration, requires a fair amount of working stack memory (a few dozen kB in some cases). "A few dozen" is much smaller than 128 kB, but apparently enabling ASAN increases the stack usage, quite a bit. So we got crashes under ASAN when it overflowed the stack that would not have been possible without ASAN enabled. https://bugzilla.mozilla.org/show_bug.cgi?id=750231 https://bugzilla.mozilla.org/show_bug.cgi?id=750231
BlueTemplar 5y ago
Pff, I always suspected that they were a bunch of racists...
- dang 5y ago
  Please stop posting unsubstantive and/or flamebait comments to HN. It looks like you've been doing it a lot, and it's not what this site is for. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - BlueTemplar 5y ago
    Well, it was supposed to be a joke, but I now realize that it was in bad taste, so I apologize.
xvilka 5y ago
Do they have a roadmap for complete Firefox conversion to Rust?
- steveklabnik 5y ago
  It's been a while, but I don't believe that is a goal, so I don't believe there's a roadmap for it.
- jdashg 5y ago
  The current roadmap for that is "never". :) There just isn't enough ROI for rewriting all old code.
  - villasv 5y ago
    I'd go further and say that the ROI is not only "insufficient", it is negligible. Unless your legacy code is written in a language/framework where available experts are disappearing (which is not the case here), there is practically zero value on rewriting code that works as intended.
    - zozbot234 5y ago
      Plenty of legacy code was reimplemented from C++ to Java (a memory-safe language, not unlike Rust) when the latter was released. The ROI is definitely there. But given the scale of that task, it makes sense to pick the lowest-hanging fruit first. AIUI, most of the effort wrt. "oxidation" is now around WebRender, specifically making it the default render backend.
  - jhgb 5y ago
    Surely at one point the return for getting rid of the rest of the C++ code would be getting rid of the C++ build process?
- stjohnswarts 5y ago
  I think you should change your question to "which -parts- of firefox will be redone in rust and is there a road map for that?"
deleted 5y ago
[deleted]
blub 5y ago
Two pieces of information that I found interesting: * Rust had poor support for sanitizers ("Rust, which has much less mature support for sanitizers)". Work was done to improve this, but it's not clear how well it works now - it would be worth a blog post in itself IMO. This was surprising, because at least I assumed that it works flawlessly thanks to clang. * Rust / C++ bridges are predictably problematic and weaken Rust's guarantees ("races in C++ code being obfuscated by passing through Rust"). This is the problem to solve for most companies working in a traditionally C and C++ oriented domain, as the C* code will dwarf the Rust code. Because of the shared memory space any problems in that code will be just as critical as if the entire code was unsafe. It would be quite awesome if Rust were able to isolate unsafe code not just at compile time, but also at runtime, by e.g. offering the option of executing the unsafe code in a separate address space. This would require some manual work for moving the data back and forth, but it could offer the tools to support such a work mode e.g. unsafe blocks could take parameters. I wish Mozilla would be more transparent about the absolutely normal and typical difficulties they must be encountering in a mixed code base. I have the feeling that there's plenty of them, but they don't want to emphasise them in order not to discourage Rust adoption.
- zaarn 5y ago
  In terms of Rust guarantees with C* code; the Rust parts of the code will be fine. Which means if you have an issue, you can easily isolate it to the C* code (probably doesn't help a lot). You can probably isolate things if you use IPC (I believe there is a few crates for that), but it wouldn't prevent the result from being wrong, only from unsafe behaviour stomping the address space. You could likely avoid copy if you relied on SHM for larger data patterns. On the other hand, integrating Rust and C* isn't terribly hard, there is automated tools that generate bindings for C/Rust so you don't really have to think about it much other than unpacking the unsafe data from C*.
- codeflo 5y ago
  I worked on a mixed C++ and C# codebase where we switched from in-process calls to IPC for that very reason, taking the efficiency hit. Having the .NET runtime running in the same address space confused any sanitizer or leak detector. You rely on those tools to manage a large C++ codebase in practice. Also, because the .NET GC churns through a lot of memory, statistically almost every problem on the C++ turned up on the C# side first, sometimes with very confusing effects. For example, .NET has a compacting GC, so an object can get corrupted and then randomly moved somewhere else. We didn't have that kind of problem often, but when we did, it was pure hell to debug.
ballerburg9006 5y ago
I almost thought it was an April fools joke.

470 points by caution 5y ago

> Overall Rust appears to be fulfilling one of its original design goals: allowing us to write more concurrent code safely. Both WebRender and Stylo are very large and pervasively multi-threaded, but have had minimal threading issues. What issues we did find were mistakes in the implementations of low-level and explicitly unsafe multithreading abstractions — and those mistakes were simple to fix. > This is in contrast to many of our C++ races, which often involved things being randomly accessed on different threads with unclear semantics, necessitating non-trivial refactorings of the code.

chowells 5y ago
Yeah, that was the part that stood out to me as well. I thought it was pretty intuitively obvious that having your data races confined to small segments is better than allowing them to be spread over the entire codebase, but a lot of people act like it negates the advantage of working in a safer language entirely. It's nice to have empirical reports that back up the "obvious" though. Sometimes "obvious" things turn out to be wrong. It's always good to check them.
- setr 5y ago
  > but a lot of people act like it negates the advantage of working in a safer language entirely. I think the underlying assumption is more along the lines of: the easy cases are made easier, and the hard cases are made impossible — or rather the concern being that rust hasn’t made the problem easier, it just moved it around. Which is often the case with claims of solving long-lasting problems
  - kibwen 5y ago
    > the concern being that rust hasn’t made the problem easier, it just moved it around. This is largely the point of Rust, though. Rust takes problems that used to manifest at runtime and causes them to manifest at compile-time, shortening the feedback loop drastically. It also takes problems that used to manifest all over the codebase and roots their causes in the leaf nodes of the program that are making use of `unsafe` blocks. Moving around the problem turns out to be a valuable way of addressing the problem, without ignoring any of the essential complexity or making anything fundamentally impossible.
    - wtetzner 5y ago
      > shortening the feedback loop drastically More importantly, forcing you to catch them. Problems at runtime don't get caught until that condition actually occurs during an actual run of the software.
  - pdpi 5y ago
    Not all forms of moving things around are equivalent. Some just keep the problem as diffuse as it was before, others restrict the problem to a smaller area. Rust does not, by any means, solve this problem. What it does do is help you prove that the problem is contained to a smaller subset of your codebase that you can audit/reason about much more easily.
  - chriswarbo 5y ago
    > Which is often the case with claims of solving long-lasting problems This is what I really like about programming language theory: problems which are literally undecidable in existing languages, can be made almost trivial by designing a language with that problem in mind. The trick is to have the language keep track of the information needed for the solution, so we're not dealing with "black boxes". A classic example is annotating every variable with a type, and propagating that information through compound expressions; lets us spot problems like trying to use a number as a string, without worrying about the halting problem (we do this by forbidding some technically-valid programs, like `1 + (true? 10 : "hello")`). Of course, this leaves the much harder problems of (a) keeping the burden imposed by this extra information down to a reasonable level, and (b) trying to make the language compelling enough for people to actually use. Rust is one of the few examples of going all-in on a new ecosystem, and it actually being embraced. Often it's easier to "embed" a new language inside an existing language, to piggy-back on the existing tooling (e.g. the variety of auto-differentiating languages built in Python)
    - Quekid5 5y ago
      Yes, people often forget that TMs really are "pure runtime". Sure, the program itself is static, but nothing else is. There's no imposed semantics on anything beyond the pure mechanism of the interpreter itself. This lessens the 'problem' part of the Halting Problem drastically if you just impose some constraints on the programs, e.g. static types, maybe a borrow-checker, etc.
  - stouset 5y ago
    > I think the underlying assumption is more along the lines of: the easy cases are made easier, and the hard cases are made impossible — or rather the concern being that rust hasn’t made the problem easier, it just moved it around. Except in this case the article linked demonstrates that they've made things dramatically easier overall. Data races went from being endemic in a C++ codebase to being in a few low-level and easy-to-locate areas. The easy stuff stayed easy, the bulk of the rest was folded into the easy case, and even the hard cases appear to have been made easier to find, identify, and fix.
    - Quekid5 5y ago
      Once you pick away enough at the straws, the needle becomes a lost easier to find.
kibwen 5y ago
And when it comes to finding bugs in your unsafe concurrency primitives in Rust programs, there is a tool called Loom that helps with that, as a powerful complement to what ThreadSanitizer provides: https://github.com/tokio-rs/loom https://github.com/tokio-rs/loom . You can think of Loom sort of like QuickCheck, except the permutations are all the possible thread states of your program based on the memory orderings that you have specified. Here's a video that briefly talks about what Loom can do https://youtu.be/rMGWeSjctlY?t=7519 https://youtu.be/rMGWeSjctlY?t=7519 (it's part of a larger video about memory orderings, and is a great introduction to the topic).
- gamegoblin 5y ago
  Loom is really awesome, though it is focused on exhaustive testing, so not suitable for code that has a lot of possible interleavings (e.g. due to a ton of threads, or a large body of code). There is a new project out of AWS called Shuttle [1] which is like Loom, but it does random exploration instead of exhaustive exploration, which enables massively distributed testing of really complicated stuff. [1] https://github.com/awslabs/shuttle https://github.com/awslabs/shuttle
  - kibwen 5y ago
    Excellent, thanks for the tip! I have to ask though, given a tool that does exhaustive testing (like Loom), shouldn't it be trivial to adapt it to do random testing merely by randomizing rather than enumerating the test inputs? Is there something more going on that I'm not seeing?
  - wyldfire 5y ago
    Is there anything like Loom or Shuttle for C/C++? Or are these only implemented in rust and they'd still work with C/C++?
    - kibwen 5y ago
      I vaguely recall that Loom is an implementation of a research paper that purported to implement this for C/C++, but I don't know if that implementation is available/production-ready. In the worst case it should be relatively straightforward to back-port Loom to C++, since Rust also uses the C memory model.
evmar 5y ago
It's interesting, I read this and had the opposite conclusion -- people are still writing racy code using Rust. I mean, it's great that you can write threaded code with more confidence, but it seems the conclusion here is that Rust gives you more confidence but not absolute confidence and you still need the normal stable of runtime detection tooling. The second fix[1] in particular resembles the bad pattern of "sprinkle atomics it until it works" you see in non-Rust languages. (If this comment raises your hackles, please read the "no true Scotsman[2]" fallacy before responding.) [1]: https://hg.mozilla.org/integration/autoland/rev/044f8b594c71 https://hg.mozilla.org/integration/autoland/rev/044f8b594c71 [2]: https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman
- brundolf 5y ago
  > What issues we did find were mistakes in the implementations of low-level and explicitly unsafe multithreading abstractions (emphasis mine) The important thing is that the races only happened in unsafe { } blocks. It's well-established that these blocks should basically be treated like C++ in terms of scrutiny, but (I believe) you can still have roughly "absolute confidence" in non-unsafe code. It's true and interesting that unsafe { } blocks deserve sanitization analysis like C++ code does - and maybe this hasn't been discussed enough - but I don't think it's fair to suggest that the enormous language benefits are outweighed by some false sense of confidence. The proportions in that idea just seem way out of whack.
  - benschulz 5y ago
    AFAIU one can create races in safe code using atomics with insufficient ordering. (That notwithstanding, I agree that Rust is a vast improvement over C/++.)
    - steveklabnik 5y ago
      "races" is too general. Safe Rust cannot have data races. It can have race conditions.
WalterBright 5y ago
D takes a different approach. D has a type constructor `shared`. Only types marked as `shared` (or `immutable`) can be accessed by multiple threads. If you've got concurrency problems, the scope of the problems are immediately reduced to the subset of the code typed as `shared`. In addition, `shared` types cannot be accessed directly. They can only be accessed via library routines, such as D's atomic library. Having an `immutable` data type also helps, as immutable data can be accessed by multiple threads without need for synchronization.
- eximius 5y ago
  It's not that different. Rust has Sync + Send + &refs and &mut refs.
- secondcoming 5y ago
  Does this approach have to deal with potential recursive locking?
  - WalterBright 5y ago
    Not by itself. But D's prototype ownership-borrowing system can help with that.
- d110af5ccf 5y ago
  Did D ever get around to specifying a formal memory model? Because when I tried to adopt D a couple years ago I felt there was a lot of ambiguity surrounding the semantics of `shared`, particularly when interfacing with C and C++ code. I ended up just casting to and from `shared`, and that seemed to work, but it was pretty verbose for highly parallel code and I was never quite sure under what circumstances doing that might violate the compiler's invariants. Also such casting appeared to eliminate most of the benefits, since what appeared to be local data might have been cast and had a pointer sent across to another thread elsewhere. In the end `shared`, @nogc antics (the language internals weren't fully annotated), closures requiring the GC (compare to C++), and the friction of interfacing with highly templated C++ code such as the Eigen library caused me to abandon the attempt. I sure learned a lot in the process though!
deleted 5y ago
[deleted]

ChiefOBrien 5y ago

Who knew using already existing data race analysis tools is much more feasible than shoehoning a huge project into a brand new language. Way to go mozilla!

Jweb_Guru 5y ago
If you believe that this tool has found all the data races in the C++ code, or even a majority of them, I have a bridge to sell you.
- ChiefOBrien 5y ago
  Oh come on now. If it was really important for them then why did they chose C++ a long time ago after deciding to rewrite Netscape from scratch? Their history is full of dumb choices and now suddenly going full Rust is the most important thing because... the codebase is lousy? It's not gonna bring back any new/old users, and right now the web needs a healthy marketshare.
  - steveklabnik 5y ago
    > now suddenly going full Rust is the most important thing As said above, Mozilla does not seem to have ever suggested, in the past or now, that "going full Rust" is a goal, let alone "the most important thing."
    - ChiefOBrien 5y ago
      Yeah and Webrender, Stylo, and Servo just happen to appear out of thin air.
      - dralley 5y ago
        Firefox has something like 8 million lines of C++. Stylo replaced about 150,000 lines of that, and WebRender is more of a fast path than a true replacement of anything. Both of those rewrites came with massive speedups due to the amount of parallelism they were able to use, which was impractical in C++. Other parts of Firefox are not as critical or parallelizable, hence why they aren't being replaced any time soon. You seem to be quite aggressively misinformed, if you are not deliberately trolling.

wyldfire 5y ago

> What is ThreadSanitizer? ThreadSanitizer (TSan) is compile-time instrumentation to detect data races according to the C/C++ memory model on Linux. It is important to note that these data races are considered undefined behavior within the C/C++ specification. TSan, ASan, UBSan - if you are writing code and your toolchain supports these, you should use them. Over the past 6-7+ years I've used them, I have never seen a false positive (I'm sure some have occasionally existed but they seem rare in practice). > However, developers claimed on various occasions that a particular report must be a false positive. In all of these cases, it turned out that TSan was indeed right and the problem was just very subtle and hard to understand. Yes, I have seen this phenomenon!

inetknght 5y ago
Address Sanitizer -- never seen a false positive. Ever. UBSan -- again never had a false positive. Thread Sanitizer does have false positives in some circumstances. They're hard to verify to be falsy and in many ways it's better to refactor to eliminate the complaint because you'll end up with better cleaner code. For example, Qt uses those atomic fences that the article describes [0] [1]. [0]: TSan does not produce false positive data race reports when properly deployed, which includes instrumenting all code that is loaded into the process and avoiding primitives that TSan doesn’t understand (such as atomic fences). [1]: https://lists.qt-project.org/pipermail/interest/2014-March/011481.html https://lists.qt-project.org/pipermail/interest/2014-March/0...
- nyanpasu64 5y ago
  I've gotten false positives in ASan when trying to instrument a Win32 application code compiled in MSVC using MSVC's then-experimental ASan support, using the wrong combination of the maze of ASan libraries (static library combined with dynamic MFC). And I also get startup crashes when running MSVC ASan binaries in Wine. But it's obscure beta use-cases and some may be fixed by now.
- derf_ 5y ago
  I have seen false positives in ASAN. In Firefox, media threads use a reduced stack size (128 kB), due to, historically, 32-bit address-space concerns with lots of media elements on a page. Opus, in its default configuration, requires a fair amount of working stack memory (a few dozen kB in some cases). "A few dozen" is much smaller than 128 kB, but apparently enabling ASAN increases the stack usage, quite a bit. So we got crashes under ASAN when it overflowed the stack that would not have been possible without ASAN enabled. https://bugzilla.mozilla.org/show_bug.cgi?id=750231 https://bugzilla.mozilla.org/show_bug.cgi?id=750231

BlueTemplar 5y ago

Pff, I always suspected that they were a bunch of racists...

dang 5y ago
Please stop posting unsubstantive and/or flamebait comments to HN. It looks like you've been doing it a lot, and it's not what this site is for. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- BlueTemplar 5y ago
  Well, it was supposed to be a joke, but I now realize that it was in bad taste, so I apologize.

xvilka 5y ago

Do they have a roadmap for complete Firefox conversion to Rust?

steveklabnik 5y ago
It's been a while, but I don't believe that is a goal, so I don't believe there's a roadmap for it.
jdashg 5y ago
The current roadmap for that is "never". :) There just isn't enough ROI for rewriting all old code.
- villasv 5y ago
  I'd go further and say that the ROI is not only "insufficient", it is negligible. Unless your legacy code is written in a language/framework where available experts are disappearing (which is not the case here), there is practically zero value on rewriting code that works as intended.
  - zozbot234 5y ago
    Plenty of legacy code was reimplemented from C++ to Java (a memory-safe language, not unlike Rust) when the latter was released. The ROI is definitely there. But given the scale of that task, it makes sense to pick the lowest-hanging fruit first. AIUI, most of the effort wrt. "oxidation" is now around WebRender, specifically making it the default render backend.
- jhgb 5y ago
  Surely at one point the return for getting rid of the rest of the C++ code would be getting rid of the C++ build process?
stjohnswarts 5y ago
I think you should change your question to "which -parts- of firefox will be redone in rust and is there a road map for that?"

deleted 5y ago

[deleted]

blub 5y ago

Two pieces of information that I found interesting: * Rust had poor support for sanitizers ("Rust, which has much less mature support for sanitizers)". Work was done to improve this, but it's not clear how well it works now - it would be worth a blog post in itself IMO. This was surprising, because at least I assumed that it works flawlessly thanks to clang. * Rust / C++ bridges are predictably problematic and weaken Rust's guarantees ("races in C++ code being obfuscated by passing through Rust"). This is the problem to solve for most companies working in a traditionally C and C++ oriented domain, as the C* code will dwarf the Rust code. Because of the shared memory space any problems in that code will be just as critical as if the entire code was unsafe. It would be quite awesome if Rust were able to isolate unsafe code not just at compile time, but also at runtime, by e.g. offering the option of executing the unsafe code in a separate address space. This would require some manual work for moving the data back and forth, but it could offer the tools to support such a work mode e.g. unsafe blocks could take parameters. I wish Mozilla would be more transparent about the absolutely normal and typical difficulties they must be encountering in a mixed code base. I have the feeling that there's plenty of them, but they don't want to emphasise them in order not to discourage Rust adoption.

zaarn 5y ago
In terms of Rust guarantees with C* code; the Rust parts of the code will be fine. Which means if you have an issue, you can easily isolate it to the C* code (probably doesn't help a lot). You can probably isolate things if you use IPC (I believe there is a few crates for that), but it wouldn't prevent the result from being wrong, only from unsafe behaviour stomping the address space. You could likely avoid copy if you relied on SHM for larger data patterns. On the other hand, integrating Rust and C* isn't terribly hard, there is automated tools that generate bindings for C/Rust so you don't really have to think about it much other than unpacking the unsafe data from C*.
codeflo 5y ago
I worked on a mixed C++ and C# codebase where we switched from in-process calls to IPC for that very reason, taking the efficiency hit. Having the .NET runtime running in the same address space confused any sanitizer or leak detector. You rely on those tools to manage a large C++ codebase in practice. Also, because the .NET GC churns through a lot of memory, statistically almost every problem on the C++ turned up on the C# side first, sometimes with very confusing effects. For example, .NET has a compacting GC, so an object can get corrupted and then randomly moved somewhere else. We didn't have that kind of problem often, but when we did, it was pure hell to debug.

ballerburg9006 5y ago

I almost thought it was an April fools joke.