9 ms·

When an implementation monoculture might be the right thing

86 points by jsnell 8y ago

xvilka 8y ago
While Clang is definitely good, GCC is quickly developing last years. Since they want to move everything into Rust, why they care about C++ parts right now?
- ash_gti 8y ago
  I don't think they will stop using c++ anytime soon.
- sametmax 8y ago
  Because Firefox is huge, complexe, and porting it aller to rust is à several years long project.
whoisthemachine 8y ago
Glad that decisions such as this get questioned, even if they seem like the inevitably correct decision. Sometimes engineering choices require uneasy decisions; ideally you can move forward with those decisions while keeping the code base cross compiler safe enough to switch to another compiler if needed in the future.
taeric 8y ago
Monocultures like this interest me. We spent a ton of time in grade school learning about how monocultures were terrible long term strategies. And yet, it seems they are actively pursued everywhere. It is easy to see why this is a problem for biological things. Just look at how close we are to losing the current bananas most in the US know. I'm pretty sure avocados are the same. In technology, though, we seem to embrace the monoculture for things. Languages, implementations, everything. They all seem to converge to a single one. I think I understand why this happens in many cases, but I can't help but shake a feeling that it is off. Indeed, I agree with this blog post, but I still feel that something is off. Am I alone in that?
- classichasclass 8y ago
  Everyone wants one grand unifying tool/standard/instruction set/architecture/theory/system to handle all use cases because it seems elegant to do so. (See obligatory xkcd which I won't post, but you know which one I mean.) It feels right to harmonize everything, even when it actually isn't, or it stifles innovation, or may lead to long-term technical debt.
- jack9 8y ago
  > Monocultures like this interest me. Just because they call it a monoculture, doesn't mean it has anything to do with social behavior per se. It's a technical choice. Might as well ask why every function isn't implemented in X languages or X styles. Using a singular tool in a build is the default. It's a surprise to me, that they still were trying to support a bunch of disparate compilers. Let someone else handle that, while you run the race of feature implementation, which is hard enough.
  - taeric 8y ago
    Your first sentence is confusing to me. Monoculture is not, necessarily, a social thing. Hence why I mentioned bananas and avocados. :) That said, I think I agree with your overall point. With the amusing counter that most new languages do go way the hell out of their way to start implementing everything. In large to keep you in that language. Even in the web services world, it amuses me to see so many folks try and cram everything into a single service where they can. The flip side, of course, is you do have a finite budge of energy to make things happen. It makes little sense to spend that budget solving problems you don't have. Worse, you could rathole into solving technical challenges that you may never actually have a need to solve. All the while leaving your customers dry. So, yes, there is a tradeoff. I fully agree with that. Which is why this topic is interesting to me. All of the incentives seem to be to embrace monoculture thinking in most that you do. I can't help but think that is off in the long view, though. :(
- qznc 8y ago
  There is also economics, where they have this scale from perfect competition over oligopoly to monopoly. What happens in reality differs very much: many competitors for farmers, few competitors for car manufacturers, no competition for government institutions. Currently, Firefox works in a compiler oligopoly. It seems they would prefer a LLVM monopoly for better efficiency. Btw the Linux kernel only supports GCC, so Mozilla is not alone.
camgunz 8y ago
I think multiple implementations is more or less a C/C++ thing. If you look at other platforms (Ruby, Python, PHP, Haskell, Java, Go, Rust, Scala, Lua, Perl) there's basically _the_ version and maybe some ancillary implementations. In fairness, it's hard enough to build a language people use that solves real problems. Asking someone to do it more than once, or to do it once but create specific enough documentation and tests so someone else can also do it -- and then get someone else to do it -- is practically impossible.
- asfasgasg 8y ago
  The difficulty, huge though it is, is secondary to the lack of utility. Even with C, for a long time there was only a single viable, mainstream compiler on any given platform. Clang being the main exception today.
  - RossBencina 8y ago
    > Even with C, for a long time there was only a single viable, mainstream compiler on any given platform. I don't think this is even remotely true. On Windows in the 90's we had Borland, Watcom, Microsoft compilers. Today there is at least MSVC and icc. On Mac there was Think C, Metrowerks Code Warrior, and others. For embedded systems there are multiple commercial compiler vendors, as well as the open source tools.
    - asfasgasg 8y ago
      It's been a minute since Borland or Watcom were mainstream C compilers on Windows. Consider the year between ~2000-2015. Based on my recollection, it seems like MSVC was used at least 75% of the time, with the other compilers each having a small slice of the remaining quarter. Today, very seldom do you encounter a program that was compiled with anything other than MSVC, although with Clang that's starting to change. I don't know too much about the history of Mac compilers. You may be right. Back in the days, it was easier to make a competitive C compiler, because the accretion of advantages to the dominant compilers was not so large. Today, the moat is too wide and too deep. There's also no reason to build a new compiler, because nobody is going to pay you for it unless it's substantially better than the free ones, which by this point are pretty damn good.
zokier 8y ago
> Whereas Firefox engineering is the only “consumer” of the compiler(s), and so we should optimize for that single consumer Seems really shortsighted view. What about distro package maintainers? Downstream derivatives? Porters to obscure platforms? Random developers/potential contributors?
- classichasclass 8y ago
  It would, in effect, make gcc (and MSVC) into "obscure platforms," or at least non-tier-1. I remember when Mozilla had trouble when they ran out of virtual memory space in the linker (see https://bugzilla.mozilla.org/show_bug.cgi?id=709193 https://bugzilla.mozilla.org/show_bug.cgi?id=709193 , but this wasn't the first time that happened) and caused serious churn for the Windows build because Visual Studio was their only option. To be sure, this was exacerbated because VS was and is a proprietary product, but if you're going to bet that clang doesn't suffer some similar critical failure in the future, maybe you shouldn't bet every single tier-1 build of your core product.
  - rebelwebmaster 8y ago
    In fact, Firefox currently can't build with the latest Visual Studio 2017 release due to internal compiler errors. https://bugzilla.mozilla.org/show_bug.cgi?id=1458247 https://bugzilla.mozilla.org/show_bug.cgi?id=1458247
  - systoll 8y ago
    The alternative is betting that none of [clang or gcc or MSVC etc] experience similar critical failures. If your plan for MSVC causing issues is to let it be broken with MSVC –– then MSVC is already not a tier-one platform. And if you're going to avoid failures caused by issues in MSVC, the existence of other compiler options doesn't change the work you have to do to achieve that.
mwcampbell 8y ago
Is Mozilla thinking about also using the same C++ standard library (libc++) on all platforms? Is it even feasible to use libc++ on Windows yet? IIRC, clang-cl uses the VC++ libraries.
bla2 8y ago
It's weird how gcc and msvc end up in the same basket here. Mozilla say they're all about openness. gcc is a free and open source compiler, while msvc is proprietary and closed source. I can see the case for keeping gcc supported, but if you don't have to support msvc, why would Mozilla give it more attention than for example icc?
- bumblebritches5 8y ago
  open != copyleft. go whine about your communist utopia products losing relevence elsewhere.
foxhop 8y ago
It's 2033 and a Mozilla Engineer is writing the same blog post but this time talking about Firefox adopting webkit... : |
- sanxiyn 8y ago
  You know what? A Mozilla Engineer did write a blog post talking about Firefox adopting WebKit... in 2007. https://robert.ocallahan.org/2018/01/ancient-browser-wars-history-md5-hashed.html https://robert.ocallahan.org/2018/01/ancient-browser-wars-hi...
stcredzero 8y ago
What often happens with standards, is that politics rears its ugly head and they become technically compromised. Some of them become absolutely toothless! C++ is sufficiently complicated, to the point where the abstractions will leak and devils will creep in with the few details of the myriads that sneak past, despite the best efforts of standard writers. Smalltalk is small enough to be succinctly describable, but politics saddled it with a completely toothless standard that basically allowed anything. What's needed are clear, succinct, moderately opinionated standards on clear, succinct, minimalist technologies.
- blub 8y ago
  C++ was first standardized in 1998, but it was created in 85. The standard is still changing, today in 2018. It's easy enough to say what's needed, but is it even possible to have that for languages with a potential life span of half a century or more?
  - reacweb 8y ago
    The Ada95 standard was covering a language far more rich than C++98 (it included all multitasking and did not rely on C standard). Its level of clarity and concision is incredible compared to C++98. Both languages were comparable at that time (same age, same usage).
  - stcredzero 8y ago
    It's easy enough to say what's needed, but is it even possible to have that for languages with a potential life span of half a century or more? Back in the early 2000's Camp Smalltalk folks actually came up with a way to have a "toothy" Smalltalk standard, by using homomorphisms. The idea was that to be an official Smalltalk, you didn't have to have exactly X, Y, and Z, but you needed to have something that could map 1-to-1 to X, Y, and Z. The intention was that "standard" Smalltalks would be able to be automatically translated between one another. One of factors why such an exercise was plausible, is the extremely minimalist nature of Smalltalk. C++ is huge. The surface area for standard-breaking problems is partly decided by the size of the language, as well as its age and the number of "vendors." We cannot stop time. It's not certain that limiting "vendors" is all good. We can also choose to minimize the size of the language.
JdeBP 8y ago
I wonder what their thoughts about OpenWatcom C++ and DigitalMars C++ were. * https://digitalmars.com/ https://digitalmars.com/ * http://openwatcom.org/ http://openwatcom.org/

86 points by jsnell 8y ago

While Clang is definitely good, GCC is quickly developing last years. Since they want to move everything into Rust, why they care about C++ parts right now?

ash_gti 8y ago
I don't think they will stop using c++ anytime soon.
sametmax 8y ago
Because Firefox is huge, complexe, and porting it aller to rust is à several years long project.

whoisthemachine 8y ago

Glad that decisions such as this get questioned, even if they seem like the inevitably correct decision. Sometimes engineering choices require uneasy decisions; ideally you can move forward with those decisions while keeping the code base cross compiler safe enough to switch to another compiler if needed in the future.

taeric 8y ago

Monocultures like this interest me. We spent a ton of time in grade school learning about how monocultures were terrible long term strategies. And yet, it seems they are actively pursued everywhere. It is easy to see why this is a problem for biological things. Just look at how close we are to losing the current bananas most in the US know. I'm pretty sure avocados are the same. In technology, though, we seem to embrace the monoculture for things. Languages, implementations, everything. They all seem to converge to a single one. I think I understand why this happens in many cases, but I can't help but shake a feeling that it is off. Indeed, I agree with this blog post, but I still feel that something is off. Am I alone in that?

classichasclass 8y ago
Everyone wants one grand unifying tool/standard/instruction set/architecture/theory/system to handle all use cases because it seems elegant to do so. (See obligatory xkcd which I won't post, but you know which one I mean.) It feels right to harmonize everything, even when it actually isn't, or it stifles innovation, or may lead to long-term technical debt.
jack9 8y ago
> Monocultures like this interest me. Just because they call it a monoculture, doesn't mean it has anything to do with social behavior per se. It's a technical choice. Might as well ask why every function isn't implemented in X languages or X styles. Using a singular tool in a build is the default. It's a surprise to me, that they still were trying to support a bunch of disparate compilers. Let someone else handle that, while you run the race of feature implementation, which is hard enough.
- taeric 8y ago
  Your first sentence is confusing to me. Monoculture is not, necessarily, a social thing. Hence why I mentioned bananas and avocados. :) That said, I think I agree with your overall point. With the amusing counter that most new languages do go way the hell out of their way to start implementing everything. In large to keep you in that language. Even in the web services world, it amuses me to see so many folks try and cram everything into a single service where they can. The flip side, of course, is you do have a finite budge of energy to make things happen. It makes little sense to spend that budget solving problems you don't have. Worse, you could rathole into solving technical challenges that you may never actually have a need to solve. All the while leaving your customers dry. So, yes, there is a tradeoff. I fully agree with that. Which is why this topic is interesting to me. All of the incentives seem to be to embrace monoculture thinking in most that you do. I can't help but think that is off in the long view, though. :(
qznc 8y ago
There is also economics, where they have this scale from perfect competition over oligopoly to monopoly. What happens in reality differs very much: many competitors for farmers, few competitors for car manufacturers, no competition for government institutions. Currently, Firefox works in a compiler oligopoly. It seems they would prefer a LLVM monopoly for better efficiency. Btw the Linux kernel only supports GCC, so Mozilla is not alone.

camgunz 8y ago

I think multiple implementations is more or less a C/C++ thing. If you look at other platforms (Ruby, Python, PHP, Haskell, Java, Go, Rust, Scala, Lua, Perl) there's basically _the_ version and maybe some ancillary implementations. In fairness, it's hard enough to build a language people use that solves real problems. Asking someone to do it more than once, or to do it once but create specific enough documentation and tests so someone else can also do it -- and then get someone else to do it -- is practically impossible.

asfasgasg 8y ago
The difficulty, huge though it is, is secondary to the lack of utility. Even with C, for a long time there was only a single viable, mainstream compiler on any given platform. Clang being the main exception today.
- RossBencina 8y ago
  > Even with C, for a long time there was only a single viable, mainstream compiler on any given platform. I don't think this is even remotely true. On Windows in the 90's we had Borland, Watcom, Microsoft compilers. Today there is at least MSVC and icc. On Mac there was Think C, Metrowerks Code Warrior, and others. For embedded systems there are multiple commercial compiler vendors, as well as the open source tools.
  - asfasgasg 8y ago
    It's been a minute since Borland or Watcom were mainstream C compilers on Windows. Consider the year between ~2000-2015. Based on my recollection, it seems like MSVC was used at least 75% of the time, with the other compilers each having a small slice of the remaining quarter. Today, very seldom do you encounter a program that was compiled with anything other than MSVC, although with Clang that's starting to change. I don't know too much about the history of Mac compilers. You may be right. Back in the days, it was easier to make a competitive C compiler, because the accretion of advantages to the dominant compilers was not so large. Today, the moat is too wide and too deep. There's also no reason to build a new compiler, because nobody is going to pay you for it unless it's substantially better than the free ones, which by this point are pretty damn good.

zokier 8y ago

> Whereas Firefox engineering is the only “consumer” of the compiler(s), and so we should optimize for that single consumer Seems really shortsighted view. What about distro package maintainers? Downstream derivatives? Porters to obscure platforms? Random developers/potential contributors?

classichasclass 8y ago
It would, in effect, make gcc (and MSVC) into "obscure platforms," or at least non-tier-1. I remember when Mozilla had trouble when they ran out of virtual memory space in the linker (see https://bugzilla.mozilla.org/show_bug.cgi?id=709193 https://bugzilla.mozilla.org/show_bug.cgi?id=709193 , but this wasn't the first time that happened) and caused serious churn for the Windows build because Visual Studio was their only option. To be sure, this was exacerbated because VS was and is a proprietary product, but if you're going to bet that clang doesn't suffer some similar critical failure in the future, maybe you shouldn't bet every single tier-1 build of your core product.
- rebelwebmaster 8y ago
  In fact, Firefox currently can't build with the latest Visual Studio 2017 release due to internal compiler errors. https://bugzilla.mozilla.org/show_bug.cgi?id=1458247 https://bugzilla.mozilla.org/show_bug.cgi?id=1458247
- systoll 8y ago
  The alternative is betting that none of [clang or gcc or MSVC etc] experience similar critical failures. If your plan for MSVC causing issues is to let it be broken with MSVC –– then MSVC is already not a tier-one platform. And if you're going to avoid failures caused by issues in MSVC, the existence of other compiler options doesn't change the work you have to do to achieve that.

mwcampbell 8y ago

Is Mozilla thinking about also using the same C++ standard library (libc++) on all platforms? Is it even feasible to use libc++ on Windows yet? IIRC, clang-cl uses the VC++ libraries.

bla2 8y ago

It's weird how gcc and msvc end up in the same basket here. Mozilla say they're all about openness. gcc is a free and open source compiler, while msvc is proprietary and closed source. I can see the case for keeping gcc supported, but if you don't have to support msvc, why would Mozilla give it more attention than for example icc?

bumblebritches5 8y ago
open != copyleft. go whine about your communist utopia products losing relevence elsewhere.

foxhop 8y ago

It's 2033 and a Mozilla Engineer is writing the same blog post but this time talking about Firefox adopting webkit... : |

sanxiyn 8y ago
You know what? A Mozilla Engineer did write a blog post talking about Firefox adopting WebKit... in 2007. https://robert.ocallahan.org/2018/01/ancient-browser-wars-history-md5-hashed.html https://robert.ocallahan.org/2018/01/ancient-browser-wars-hi...

stcredzero 8y ago

What often happens with standards, is that politics rears its ugly head and they become technically compromised. Some of them become absolutely toothless! C++ is sufficiently complicated, to the point where the abstractions will leak and devils will creep in with the few details of the myriads that sneak past, despite the best efforts of standard writers. Smalltalk is small enough to be succinctly describable, but politics saddled it with a completely toothless standard that basically allowed anything. What's needed are clear, succinct, moderately opinionated standards on clear, succinct, minimalist technologies.

blub 8y ago
C++ was first standardized in 1998, but it was created in 85. The standard is still changing, today in 2018. It's easy enough to say what's needed, but is it even possible to have that for languages with a potential life span of half a century or more?
- reacweb 8y ago
  The Ada95 standard was covering a language far more rich than C++98 (it included all multitasking and did not rely on C standard). Its level of clarity and concision is incredible compared to C++98. Both languages were comparable at that time (same age, same usage).
- stcredzero 8y ago
  It's easy enough to say what's needed, but is it even possible to have that for languages with a potential life span of half a century or more? Back in the early 2000's Camp Smalltalk folks actually came up with a way to have a "toothy" Smalltalk standard, by using homomorphisms. The idea was that to be an official Smalltalk, you didn't have to have exactly X, Y, and Z, but you needed to have something that could map 1-to-1 to X, Y, and Z. The intention was that "standard" Smalltalks would be able to be automatically translated between one another. One of factors why such an exercise was plausible, is the extremely minimalist nature of Smalltalk. C++ is huge. The surface area for standard-breaking problems is partly decided by the size of the language, as well as its age and the number of "vendors." We cannot stop time. It's not certain that limiting "vendors" is all good. We can also choose to minimize the size of the language.

JdeBP 8y ago

I wonder what their thoughts about OpenWatcom C++ and DigitalMars C++ were. * https://digitalmars.com/ https://digitalmars.com/ * http://openwatcom.org/ http://openwatcom.org/