5 ms·

Not that I really see away around it, given the size and feature set of Chrome, but those build requirements are just crazy. It kinda throws the open source and

Not that I really see away around it, given the size and feature set of Chrome, but those build requirements are just crazy. It kinda throws the open source and "everyone can contribute" model out the window, if you can't afford a pretty insane workstation then you're going to have a bad time. I doubt that Firefox is better, I seem to remember that building Firefox and the VIA C3 processor years back as around half a day of compiling, but was also an extremely poorly choose CPU for the task.

creesch 2y ago
> It kinda throws the how open source and "everyone can contribute" model out the window, if you can't afford a pretty insane workstation then you're going to have a bad time. That, it also means you are going to spend a lot of time on it before you can even attempt to do anything. Overall, there can be a pretty substantial amount of effort involved before you are even ready to make a PR of any kind. Then it remains to be seen if it is well received by the people who can approve it. You mention Firefox, my dealings with various contributors and people at Mozilla over the years would make me very hesitant to even consider diving into the deep end. To be fair, I have had good interactions with various people as well. But a lot of communication also has been just outright difficult. All of this also throws out the "if you don't like it, you can just fork it" mindset.
- cxr 2y ago
  "Patches welcome" vs "patches wanted"[1]. 1. <https://www.colbyrussell.com/2013/08/06/patches-wanted.html https://www.colbyrussell.com/2013/08/06/patches-wanted.html>
  - creesch 2y ago
    That little snippet doesn't quite provide enough historical context for me to agree or disagree with it.
    - cxr 2y ago
      Huh?
      - creesch 2y ago
        The thing you linked starts with > The phrase "patches welcome" has come not to be. Which I am already missing context for to make sense out of. Where was this phrase used, by who? I get a general idea of what it is about and what the intention is but it isn't all that clear as I am missing the surrounding context.
        cxr 2y ago
        K. To start with, it's been a common phrase in open source since before "open source" was invented and fairly self-explanatory; it means what it means on its face. Even if that weren't the case and someone were just encountering it for the first time in 2024, a 2-second attempt to get acquainted will not leave anyone wondering. There are 200 results for "patches welcome" on HN: <https://hn.algolia.com/?dateRange=all&page=5&prefix=false&query=%22patches%20welcome%22&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=5&prefix=false&qu...> The top results on Google for the phrase "patches welcome", most being posts written around the 4-, 5-, 6-year mark after the post I linked to, are all based on the same premise: "patches welcome" is (a) common, and (b) not what anyone wants to hear. <https://www.google.com/search?num=100&q=%22patches%20welcome%22 https://www.google.com/search?num=100&q=%22patches%20welcome...> E.g.: > "Patches Welcome". We've all seen it in the Open Source community. Nothing makes me angrier than these two words. <https://fy.blackhats.net.au/blog/2016-02-17-patches-welcome/ https://fy.blackhats.net.au/blog/2016-02-17-patches-welcome/> But I'm also missing some context here; my "Huh?" was rather more directed at the part of the remark that says "[...] for me to agree or disagree with it".
madeofpalk 2y ago
> I doubt that Firefox is better Firefox builds are seemingly more modular. I was hacking on the Firefox devtools a while back, which from source involved downloading a pre-build main brower binary and building just devtools from your source. This made it significantly faster due to not having to build the rest of Firefox from source. Of course, this will all depend on which part of the source code you're changing.
- rafram 2y ago
  Firefox isn’t very modular. There’s just a C++ part and a JS part and they can be built independently. My understanding is that new developers are encouraged to focus on the JS. The devtools team is entirely JS, with some fancy modern affordances (TypeScript type definitions!) that teams working on the older parts of the JS codebase don’t have.
  - cxr 2y ago
    The build process for the rest of Firefox could be as modular as devtools. When I was involved with Mozilla (and for a while after), it was a frequent criticism of mine that no one was prioritizing making the contribution process as easy as possible for people who didn't have an email address ending in @mozilla.com. Zotero has (had?) a really cool build system. Zotero is written on an Electron-like architecture, except it uses a Gecko-based runtime instead of Blink. (Firefox is the same way and has always been this way, and this predates not just Electron, but Chrome/Chromium, too.) Similar to the way that Electron apps' build scripts generally work by downloading a prebuilt Electron (rather than requiring developers build it from source), Zotero works by downloading a release build of Firefox, unarchiving everything, sweeping away all the XHTML/XUL/CSS/JS that comprises the Firefox UI and application logic, and then swapping in the components that make up Zotero. (In other words, just how it should work 90+% of contributors who want to submit patches against Firefox nightlies, too.)
izacus 2y ago
At which point was Chromium even attempting a "everyone can contribute" model of opensource? Like most corporate OSS projects, community contributions aren't something they rely on.
badsectoracula 2y ago
Depends on how much you really have hyped yourself on the task :-P. Back in early 2000s a friend of mine was mentioning on IRC how middle clicking on a scrollbar in Mozilla under X11 didn't jump to the clicked point like in other X11 GUI programs. I was full into the Mozilla hype back then (the project was basically at its apex of coolness :-P) and wanted to get people into it, so i thought "this is opensource, right? I can do it myself and convince people how great Mozilla is". Problem being, i had a Pentium MMX @ 200MHz with 128MB of RAM. It took at least six hours to do a build (most likely more, i felt asleep at some point during the night, then woke up ~5-6h later, was still compiling and i left to go out with a friend and was done when i came back some time later). Even if i didn't make a change and tried to recompile it took half an hour. Fortunately i had already done some GUI programs by that point (and even tried to make my own GUI systems and toolkits) so i had a rough idea where to look and it didn't took too long to figure out the CPP file where scrollbar behavior was implemented (though it did take some extra hours). I did submit it and remember being impressed by the process of having my patch to be "reviewed" and then "super reviewed", thinking that it now makes sense how Mozilla is of high quality (remember, i was a teenager big into the hype, everything was filtered through the most positive lens). FWIW i was asked to make a couple of minor changes but it was merged in. I don't think i'd have as much patience these days, the last time i tried to build some relatively complex piece of software so i can contribute to it was with Krita, but that took only about an hour (including getting everything needed to build) - and it didn't work at the end (it built but crashed at startup) so i decided to try some other time as i had lost interest by that point to debug why it didn't work :-P.
userbinator 2y ago
My solution for a few "bugs considered features" in huge open-source software like Firefox was to just patch the binary. Much easier than figuring out how to build it, and with only the change I wanted.
- lxgr 2y ago
  Do you have experience in disassembling binaries? How do you find the relevant part of the executable, and what tools are you using for this?
  - saagarjha 2y ago
    The big tools in this space are IDA Pro, Ghidra, and you can cross-reference that with the source code to get what you want.
  - userbinator 2y ago
    Yes, several decades of experience. Mainly Windbg now but I used to use SoftICE a lot. Look for error (or otherwise) messages/codes and use breakpoints to guide your exploration. Of course, the source code is also available, but sometimes it's even more difficult to follow than the binary if what you're looking for is obfuscated in several layers of compiled-out indirections.
high_na_euv 2y ago
Cpp compile times are shame as hell This shit shouldnt need this many resources for debug builds
- jmb99 2y ago
  45 minute build for 32 million lines of code seems pretty reasonable, to me. What are some projects of equal complexity with lower build time in other languages?
  - high_na_euv 2y ago
    There are no projects with equal complexity, let alone in other langs that im aware of
    - willhslade 2y ago
      It's bigger than Linux?
      - high_na_euv 2y ago
        They use the same compilers tho
qingcharles 2y ago
The onerous requirements are just for a first build, though? I used to leave things like that running overnight in the old days. All subsequent builds are just going to be compiling the one or two files you changed and then linking in the other 99.9%, which isn't going to take very long.
devsda 2y ago
> I doubt that Firefox is better, I seem to remember that building Firefox and the VIA C3 processor years back as around half a day of compiling, but was also an extremely poorly choose CPU for the task. Around 6-7 years back, I was able to make a change & build Firefox from source on a mid-range gaming laptop without much fiddling. I think the build took may be around an hour or more and it was not too long to stand out. I haven't tried building chromium to compare but from my past experience, Firefox's build was not too challenging for first time contributors.
mdaniel 2y ago
> I doubt that Firefox is better For a long time I would build Firefox from source every morning, and I don't have the build logs anymore but I would guess it was in the range of 60 to 75 minutes. Comparing building anything on a VIA C3 is not serious, nor is using some 5400rpm disk for the same task
- cpeterso 2y ago
  I can make a clean build of Firefox in less than 15 minutes on my MacBook Air M2, C++, Rust, and all. If you’re working on a frontend feature that only needs to modify Firefox’s frontend JS code, you can use “artifact builds” (prebuilt object files) so you don’t need to recompile the native code.
  - mdaniel 2y ago
    > less than 15 minutes ... Rust, and all Wow, your rust experience is way different from mine (not just FF, I mean invocations of rustc are where time goes to die). I also readily admit that I struggled constantly with trying to get sccache to behave sanely, so I'm open to that being part of the difference, too
    - cpeterso 2y ago
      FWIW, local sccache has never been a speedup for me. I've never tried to configure remote sccache.
kccqzy 2y ago
Open source doesn't imply "everyone can contribute" at all. The most famous example is perhaps SQLite, which is fully open source (even in the public domain) but contributions are not welcome.
cryptonector 2y ago
A browser is as complex (more) as an operating system. Building and working on one is just going to require beefy build servers or workstations.
kllrnohj 2y ago
16 core CPU, 64GB RAM, and 100GB of storage is definitely not a budget build, but I don't really see how that would be a "pretty insane workstation", either. That's firmly in consumer space these days and has been for a few years. But that's also not a requirement, either, that's just what it took for a 40 minute clean build time.
- doublepg23 2y ago
  Yeah that's a very doable build on AM4 Ryzen. I'd hope someone with C++ chops to hack on Chrome would be able to afford that.
- dieortin 2y ago
  Those requirements exceed what 99% of developers have, so I would agree that a workstation with those specs is pretty insane.
  - kllrnohj 2y ago
    ~5% of steam users have 16 core CPUs & 22% are more than 8 cores: https://store.steampowered.com/hwsurvey/cpus/ https://store.steampowered.com/hwsurvey/cpus/ No idea where you're pulling your 99% figure from, but it seems quite unlikely. Regardless, if compiling Chromium or similar large code bases is a thing you want to do regularly, it's unlikely to present a significant burden.
- koyote 2y ago
  I'd argue that it's not the 'default' workstation most developers who do not have specific requirements will be running, especially on the RAM side. My personal machine is very mid-spec'ed because my personal projects are all small and don't really require much. Even my work machine tops out at 32Gb RAM which has always been plenty, even for our comparatively large codebases (although more is always welcome :)).
  - kllrnohj 2y ago
    RAM is cheap and easily upgraded after the fact, though. Going from 32gb to 64gb is $100 I certainly wouldn't spend that for a one-off patch to an open source project, but if you're doing this regularly it's not exactly a steep upgrade either
mccr8 2y ago
Firefox uses unified builds, where a bunch of .cpp files are globbed together and compiled at once. That helps a lot, but a build still takes a bit of time unless you are on an absurdly fast machine. Chrome used to also support this, called "jumbo builds", but they didn't want to deal with the maintenance overhead. Presumably all of the Chrome developers employed by Google are using some kind of massive distributed build infrastructure so there's little impact of slower builds on individual developer productivity, so the use case of building on a single computer is not as prioritized.

creesch 2y ago

> It kinda throws the how open source and "everyone can contribute" model out the window, if you can't afford a pretty insane workstation then you're going to have a bad time. That, it also means you are going to spend a lot of time on it before you can even attempt to do anything. Overall, there can be a pretty substantial amount of effort involved before you are even ready to make a PR of any kind. Then it remains to be seen if it is well received by the people who can approve it. You mention Firefox, my dealings with various contributors and people at Mozilla over the years would make me very hesitant to even consider diving into the deep end. To be fair, I have had good interactions with various people as well. But a lot of communication also has been just outright difficult. All of this also throws out the "if you don't like it, you can just fork it" mindset.

cxr 2y ago
"Patches welcome" vs "patches wanted"[1]. 1. <https://www.colbyrussell.com/2013/08/06/patches-wanted.html https://www.colbyrussell.com/2013/08/06/patches-wanted.html>
- creesch 2y ago
  That little snippet doesn't quite provide enough historical context for me to agree or disagree with it.
  - cxr 2y ago
    Huh?
    - creesch 2y ago
      The thing you linked starts with > The phrase "patches welcome" has come not to be. Which I am already missing context for to make sense out of. Where was this phrase used, by who? I get a general idea of what it is about and what the intention is but it isn't all that clear as I am missing the surrounding context.
      - cxr 2y ago
        K. To start with, it's been a common phrase in open source since before "open source" was invented and fairly self-explanatory; it means what it means on its face. Even if that weren't the case and someone were just encountering it for the first time in 2024, a 2-second attempt to get acquainted will not leave anyone wondering. There are 200 results for "patches welcome" on HN: <https://hn.algolia.com/?dateRange=all&page=5&prefix=false&query=%22patches%20welcome%22&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=5&prefix=false&qu...> The top results on Google for the phrase "patches welcome", most being posts written around the 4-, 5-, 6-year mark after the post I linked to, are all based on the same premise: "patches welcome" is (a) common, and (b) not what anyone wants to hear. <https://www.google.com/search?num=100&q=%22patches%20welcome%22 https://www.google.com/search?num=100&q=%22patches%20welcome...> E.g.: > "Patches Welcome". We've all seen it in the Open Source community. Nothing makes me angrier than these two words. <https://fy.blackhats.net.au/blog/2016-02-17-patches-welcome/ https://fy.blackhats.net.au/blog/2016-02-17-patches-welcome/> But I'm also missing some context here; my "Huh?" was rather more directed at the part of the remark that says "[...] for me to agree or disagree with it".

madeofpalk 2y ago

> I doubt that Firefox is better Firefox builds are seemingly more modular. I was hacking on the Firefox devtools a while back, which from source involved downloading a pre-build main brower binary and building just devtools from your source. This made it significantly faster due to not having to build the rest of Firefox from source. Of course, this will all depend on which part of the source code you're changing.

rafram 2y ago
Firefox isn’t very modular. There’s just a C++ part and a JS part and they can be built independently. My understanding is that new developers are encouraged to focus on the JS. The devtools team is entirely JS, with some fancy modern affordances (TypeScript type definitions!) that teams working on the older parts of the JS codebase don’t have.
- cxr 2y ago
  The build process for the rest of Firefox could be as modular as devtools. When I was involved with Mozilla (and for a while after), it was a frequent criticism of mine that no one was prioritizing making the contribution process as easy as possible for people who didn't have an email address ending in @mozilla.com. Zotero has (had?) a really cool build system. Zotero is written on an Electron-like architecture, except it uses a Gecko-based runtime instead of Blink. (Firefox is the same way and has always been this way, and this predates not just Electron, but Chrome/Chromium, too.) Similar to the way that Electron apps' build scripts generally work by downloading a prebuilt Electron (rather than requiring developers build it from source), Zotero works by downloading a release build of Firefox, unarchiving everything, sweeping away all the XHTML/XUL/CSS/JS that comprises the Firefox UI and application logic, and then swapping in the components that make up Zotero. (In other words, just how it should work 90+% of contributors who want to submit patches against Firefox nightlies, too.)

izacus 2y ago

At which point was Chromium even attempting a "everyone can contribute" model of opensource? Like most corporate OSS projects, community contributions aren't something they rely on.

badsectoracula 2y ago

Depends on how much you really have hyped yourself on the task :-P. Back in early 2000s a friend of mine was mentioning on IRC how middle clicking on a scrollbar in Mozilla under X11 didn't jump to the clicked point like in other X11 GUI programs. I was full into the Mozilla hype back then (the project was basically at its apex of coolness :-P) and wanted to get people into it, so i thought "this is opensource, right? I can do it myself and convince people how great Mozilla is". Problem being, i had a Pentium MMX @ 200MHz with 128MB of RAM. It took at least six hours to do a build (most likely more, i felt asleep at some point during the night, then woke up ~5-6h later, was still compiling and i left to go out with a friend and was done when i came back some time later). Even if i didn't make a change and tried to recompile it took half an hour. Fortunately i had already done some GUI programs by that point (and even tried to make my own GUI systems and toolkits) so i had a rough idea where to look and it didn't took too long to figure out the CPP file where scrollbar behavior was implemented (though it did take some extra hours). I did submit it and remember being impressed by the process of having my patch to be "reviewed" and then "super reviewed", thinking that it now makes sense how Mozilla is of high quality (remember, i was a teenager big into the hype, everything was filtered through the most positive lens). FWIW i was asked to make a couple of minor changes but it was merged in. I don't think i'd have as much patience these days, the last time i tried to build some relatively complex piece of software so i can contribute to it was with Krita, but that took only about an hour (including getting everything needed to build) - and it didn't work at the end (it built but crashed at startup) so i decided to try some other time as i had lost interest by that point to debug why it didn't work :-P.

userbinator 2y ago

My solution for a few "bugs considered features" in huge open-source software like Firefox was to just patch the binary. Much easier than figuring out how to build it, and with only the change I wanted.

lxgr 2y ago
Do you have experience in disassembling binaries? How do you find the relevant part of the executable, and what tools are you using for this?
- saagarjha 2y ago
  The big tools in this space are IDA Pro, Ghidra, and you can cross-reference that with the source code to get what you want.
- userbinator 2y ago
  Yes, several decades of experience. Mainly Windbg now but I used to use SoftICE a lot. Look for error (or otherwise) messages/codes and use breakpoints to guide your exploration. Of course, the source code is also available, but sometimes it's even more difficult to follow than the binary if what you're looking for is obfuscated in several layers of compiled-out indirections.

high_na_euv 2y ago

Cpp compile times are shame as hell This shit shouldnt need this many resources for debug builds

jmb99 2y ago
45 minute build for 32 million lines of code seems pretty reasonable, to me. What are some projects of equal complexity with lower build time in other languages?
- high_na_euv 2y ago
  There are no projects with equal complexity, let alone in other langs that im aware of
  - willhslade 2y ago
    It's bigger than Linux?
    - high_na_euv 2y ago
      They use the same compilers tho

qingcharles 2y ago

The onerous requirements are just for a first build, though? I used to leave things like that running overnight in the old days. All subsequent builds are just going to be compiling the one or two files you changed and then linking in the other 99.9%, which isn't going to take very long.

devsda 2y ago

> I doubt that Firefox is better, I seem to remember that building Firefox and the VIA C3 processor years back as around half a day of compiling, but was also an extremely poorly choose CPU for the task. Around 6-7 years back, I was able to make a change & build Firefox from source on a mid-range gaming laptop without much fiddling. I think the build took may be around an hour or more and it was not too long to stand out. I haven't tried building chromium to compare but from my past experience, Firefox's build was not too challenging for first time contributors.

mdaniel 2y ago

> I doubt that Firefox is better For a long time I would build Firefox from source every morning, and I don't have the build logs anymore but I would guess it was in the range of 60 to 75 minutes. Comparing building anything on a VIA C3 is not serious, nor is using some 5400rpm disk for the same task

cpeterso 2y ago
I can make a clean build of Firefox in less than 15 minutes on my MacBook Air M2, C++, Rust, and all. If you’re working on a frontend feature that only needs to modify Firefox’s frontend JS code, you can use “artifact builds” (prebuilt object files) so you don’t need to recompile the native code.
- mdaniel 2y ago
  > less than 15 minutes ... Rust, and all Wow, your rust experience is way different from mine (not just FF, I mean invocations of rustc are where time goes to die). I also readily admit that I struggled constantly with trying to get sccache to behave sanely, so I'm open to that being part of the difference, too
  - cpeterso 2y ago
    FWIW, local sccache has never been a speedup for me. I've never tried to configure remote sccache.

kccqzy 2y ago

Open source doesn't imply "everyone can contribute" at all. The most famous example is perhaps SQLite, which is fully open source (even in the public domain) but contributions are not welcome.

cryptonector 2y ago

A browser is as complex (more) as an operating system. Building and working on one is just going to require beefy build servers or workstations.

kllrnohj 2y ago

16 core CPU, 64GB RAM, and 100GB of storage is definitely not a budget build, but I don't really see how that would be a "pretty insane workstation", either. That's firmly in consumer space these days and has been for a few years. But that's also not a requirement, either, that's just what it took for a 40 minute clean build time.

doublepg23 2y ago
Yeah that's a very doable build on AM4 Ryzen. I'd hope someone with C++ chops to hack on Chrome would be able to afford that.
dieortin 2y ago
Those requirements exceed what 99% of developers have, so I would agree that a workstation with those specs is pretty insane.
- kllrnohj 2y ago
  ~5% of steam users have 16 core CPUs & 22% are more than 8 cores: https://store.steampowered.com/hwsurvey/cpus/ https://store.steampowered.com/hwsurvey/cpus/ No idea where you're pulling your 99% figure from, but it seems quite unlikely. Regardless, if compiling Chromium or similar large code bases is a thing you want to do regularly, it's unlikely to present a significant burden.
koyote 2y ago
I'd argue that it's not the 'default' workstation most developers who do not have specific requirements will be running, especially on the RAM side. My personal machine is very mid-spec'ed because my personal projects are all small and don't really require much. Even my work machine tops out at 32Gb RAM which has always been plenty, even for our comparatively large codebases (although more is always welcome :)).
- kllrnohj 2y ago
  RAM is cheap and easily upgraded after the fact, though. Going from 32gb to 64gb is $100 I certainly wouldn't spend that for a one-off patch to an open source project, but if you're doing this regularly it's not exactly a steep upgrade either

mccr8 2y ago

Firefox uses unified builds, where a bunch of .cpp files are globbed together and compiled at once. That helps a lot, but a build still takes a bit of time unless you are on an absurdly fast machine. Chrome used to also support this, called "jumbo builds", but they didn't want to deal with the maintenance overhead. Presumably all of the Chrome developers employed by Google are using some kind of massive distributed build infrastructure so there's little impact of slower builds on individual developer productivity, so the use case of building on a single computer is not as prioritized.