6 ms·

Firefox ships ~100 binaries for 100 different targets, from mobile phones, to desktops, tablets, ... x86, arm, sparc, ppc, riscv, ... X linux, windows, macos,

Firefox ships ~100 binaries for 100 different targets, from mobile phones, to desktops, tablets, ... x86, arm, sparc, ppc, riscv, ... X linux, windows, macos, freebds, openbsd, netbsd, .... X arch linux, debian, ubuntu .. At some point you need to say this is what we want to ship, and branch, and make sure that not only all tests pass, but that the installers work, run the benchmark suite that might not be run on every PR, make sure the updates work on all systems, appstores, etc. Accept new patches to fix bugs, etc. I suppose they might have a release team that does nothing but releases, and if creating a full release takes a week (no idea how long it takes honestly), then maybe they could release each week. But I don't think they can release a new version every day, because probably running their whole CI takes longer. E.g. before shipping a version of Rust, the RC is used to build and run tests of all packages in the Rust package repository. This takes 4 full days. One could probably throw money at the problem and speed it up, but at least right now, releases just cannot technically happen faster, unless you are willing to compromise on quality.

aquabeagle 7y ago
Mozilla only provides binaries for Linux 32-bit, Linux 64-bit, macOS, win32, and win64: http://ftp.mozilla.org/pub/firefox/releases/69.0/ http://ftp.mozilla.org/pub/firefox/releases/69.0/ All of the various Linux distributions just repackage those binaries. iOS and Android are distributed only through the App Store and Google Play. The BSDs and other ("tier 3") platforms have to compile from source and distribute their own packages: https://developer.mozilla.org/en-US/docs/Mozilla/Supported_build_configurations https://developer.mozilla.org/en-US/docs/Mozilla/Supported_b...
- acqq 7y ago
  > All of the various Linux distributions just repackage those binaries Debian and Ubuntu don’t build from sources?
  - throwaway170919 7y ago
    They do. You can notice by looking at the build time in the version string. This makes it impossible to share a profile between two GNU/Linux distributions (or even the official binary and the distribution binary) without hacking with the current versions, even if they are effectively the same version: one will complain that the profile has been run with a more recent version. (But I sure expect and want my distributions to rebuild Firefox - I trust Debian for making sure no bit of proprietary code is here, and makes tracking opt-in, more than Mozilla)
- earenndil 7y ago
  Most linuces build their own firefoxes from source, same as the BSDs.
  - ggg3 7y ago
    it mostly was the case to remove the logo at some point in time :/
- kingofpandora 7y ago
  > All of the various Linux distributions just repackage those binaries. Source please. Never mind, it's simply wrong so don't even worry about it!
  - Wowfunhappy 7y ago
    Even if you’re right and they’re wrong, the snark was unwarranted IMO. You should either correct the GP or request more information (like a source), not ask rhetorically.
  - ajsnigrutin 7y ago
    Gentoo does it for firefox-bin: /usr/portage/www-client/firefox-bin/firefox-bin-69.0.ebuild MOZ_HTTP_URI="https://archive.mozilla.org/pub/mozilla.org/${MOZ_PN}/releases/" ... SRC_URI="${SRC_URI} amd64? ( ${MOZ_HTTP_URI%/}/${MOZ_PV}/linux-x86_64/en-US/${MOZ_P}.tar.bz2 -> ${PN}_x86_64-${PV}.tar.bz2 ) x86? ( ${MOZ_HTTP_URI%/}/${MOZ_PV}/linux-i686/en-US/${MOZ_P}.tar.bz2 -> ${PN}_i686-${PV}.tar.bz2 )"
    - brennebeck 7y ago
      Doesn’t gentoo essentially do that for (most?) everything?
      - moviuro 7y ago
        Only massive packages (firefox, chromium, KDE, Gnome, etc.). The entire promise of gentoo is the opposite: fine-tuning and local ~~heater~~ compilation.
- sha666sum 7y ago
  Although this comment was already debunked, I'll add Arch's Firefox PKGBUILD here [1]. And to save you a click, they compile it from a source tarball. [1] https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/firefox https://git.archlinux.org/svntogit/packages.git/tree/trunk/P...
  - JetSpiegel 7y ago
    They even need the Google API key... https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/firefox#n37 https://git.archlinux.org/svntogit/packages.git/tree/trunk/P...
LinuxBender 7y ago
Not to mention, different builds per language for each OS. [1] This has never really made sense to me, but I'm sure there must be a good reason. [1] - https://ftp.mozilla.org/pub/firefox/releases/68.1.0esr/linux-x86_64/ https://ftp.mozilla.org/pub/firefox/releases/68.1.0esr/linux...
- kevingadd 7y ago
  My educated guess is that the localization data for every language is pretty massive, so per-region downloads can include the text for their region without dragging along a few extra megabytes of text/images for other regions. If you swap regions after installing it can probably download on-demand. Historically reducing your download size by a couple megabytes has had measurable positive impact on conversion rates (from webpage visit -> install) so I expect this tradeoff made a lot of sense in the past even if it's less important now. EDIT: One other thing worth note is that there's zh-CN and zh-TW, and I can imagine the Chinese government being very irritated about the idea of their release having separate Taiwan loc text/images baked in since according to them Taiwan doesn't exist. Major software like Windows has already had CN-specific releases for this (among other) reason(s).
  - Hello71 7y ago
    zh-CN and zh-TW has nothing to do with that, and I'm virtually certain there are no "Taiwan images". in applications which provide zh-CN and zh-TW, including Firefox, the former means Simplified Chinese and the latter means Traditional Chinese. theoretically, it could mean Taiwanese Chinese, but then there would also be zh-HK, zh-MO, zh-SG...
  - blaesus 7y ago
    `zh-CN` and `zh-TW` are generally just old-school ways to say "simplified Chinese" and "Traditional Chinese" (two similar writing systems). The confusion comes in as the Chinese language family has multiple dialects and variants, and there are multiple writing systems. The current practice is really a dimension-reduction, where a N×2 matrix [(CN, TW, HK, SG, MY...) × (simplified, traditional)] collapsed into a 2×1 vector (TW, CN). The language labels should have been `zh-hans` and `zh-hant` if one means to differentiate the writing systems, and not the underlying linguistic variants. According to Beijing government, Taiwan absolutely exists, it's just that it is not a sovereign state. Hong Kong has been handed over to China for more than twenty years and people in Hong Kong continue to use zh-HK – Beijing government seems to be OK with that.
nickm12 7y ago
Right. Desktop software is also much more difficult to QA than server software because of the lack of control over the environment and the difficulty of rollback.
- nextos 7y ago
  Also, for desktop software, I have found that small releases are a lot less risky. You experience small issues, one by one, instead of a massive amount of bugs coming all of a sudden. For example, last time I updated a Linux distribution with 6-month releases this was the case. Whereas with a rolling release one, it's just much more pleasant.
  - deckarep 7y ago
    This is true of releasing code in general. Changes that accumulate over time actually increase risk whether it be desktop or server-side.
chrisseaton 7y ago
They must already be doing a full build, test, and benchmark for each platform for each PR to their codebase in CI, presumably in parallel to make it run reasonably fast. It doesn’t seem like much of an extra step to release those binaries they’re already building hundreds of times a day.
- uranusjr 7y ago
  They already do that with Firefox Nightly. But most users expect a more stable product than that.
  - chrisseaton 7y ago
    How can it be more stable than a build that passes CI? If they have some extra tests to run to check it’s stable why aren’t those in CI?
    - pingyong 7y ago
      It can be more stable by having nightly users test it beforehand for a while and report any new bugs.
      - chrisseaton 7y ago
        What are those users doing that can’t be automated and run in CI?
        chopin 7y ago
        Test on exotic hardware, for example. Or exotic combinations of libraries in the OS which FF depends on.
        chrisseaton 7y ago
        Those should be in the CI test matrix then - we shouldn’t be relying on manual testing done specially for releases.
fragmede 7y ago
They build downloadable binaries every 12 hours, called Firefox nightly, which is to say the bottleneck for full releases isn't building or automated testing. Mind you, the nightlies are experimental, and not for general use.
- fluffything 7y ago
  Building is in general just a small part of the release pipeline.

aquabeagle 7y ago

Mozilla only provides binaries for Linux 32-bit, Linux 64-bit, macOS, win32, and win64: http://ftp.mozilla.org/pub/firefox/releases/69.0/ http://ftp.mozilla.org/pub/firefox/releases/69.0/ All of the various Linux distributions just repackage those binaries. iOS and Android are distributed only through the App Store and Google Play. The BSDs and other ("tier 3") platforms have to compile from source and distribute their own packages: https://developer.mozilla.org/en-US/docs/Mozilla/Supported_build_configurations https://developer.mozilla.org/en-US/docs/Mozilla/Supported_b...

acqq 7y ago
> All of the various Linux distributions just repackage those binaries Debian and Ubuntu don’t build from sources?
- throwaway170919 7y ago
  They do. You can notice by looking at the build time in the version string. This makes it impossible to share a profile between two GNU/Linux distributions (or even the official binary and the distribution binary) without hacking with the current versions, even if they are effectively the same version: one will complain that the profile has been run with a more recent version. (But I sure expect and want my distributions to rebuild Firefox - I trust Debian for making sure no bit of proprietary code is here, and makes tracking opt-in, more than Mozilla)
earenndil 7y ago
Most linuces build their own firefoxes from source, same as the BSDs.
- ggg3 7y ago
  it mostly was the case to remove the logo at some point in time :/
kingofpandora 7y ago
> All of the various Linux distributions just repackage those binaries. Source please. Never mind, it's simply wrong so don't even worry about it!
- Wowfunhappy 7y ago
  Even if you’re right and they’re wrong, the snark was unwarranted IMO. You should either correct the GP or request more information (like a source), not ask rhetorically.
- ajsnigrutin 7y ago
  Gentoo does it for firefox-bin: /usr/portage/www-client/firefox-bin/firefox-bin-69.0.ebuild MOZ_HTTP_URI="https://archive.mozilla.org/pub/mozilla.org/${MOZ_PN}/releases/" ... SRC_URI="${SRC_URI} amd64? ( ${MOZ_HTTP_URI%/}/${MOZ_PV}/linux-x86_64/en-US/${MOZ_P}.tar.bz2 -> ${PN}_x86_64-${PV}.tar.bz2 ) x86? ( ${MOZ_HTTP_URI%/}/${MOZ_PV}/linux-i686/en-US/${MOZ_P}.tar.bz2 -> ${PN}_i686-${PV}.tar.bz2 )"
  - brennebeck 7y ago
    Doesn’t gentoo essentially do that for (most?) everything?
    - moviuro 7y ago
      Only massive packages (firefox, chromium, KDE, Gnome, etc.). The entire promise of gentoo is the opposite: fine-tuning and local ~~heater~~ compilation.
sha666sum 7y ago
Although this comment was already debunked, I'll add Arch's Firefox PKGBUILD here [1]. And to save you a click, they compile it from a source tarball. [1] https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/firefox https://git.archlinux.org/svntogit/packages.git/tree/trunk/P...
- JetSpiegel 7y ago
  They even need the Google API key... https://git.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/firefox#n37 https://git.archlinux.org/svntogit/packages.git/tree/trunk/P...

LinuxBender 7y ago

Not to mention, different builds per language for each OS. [1] This has never really made sense to me, but I'm sure there must be a good reason. [1] - https://ftp.mozilla.org/pub/firefox/releases/68.1.0esr/linux-x86_64/ https://ftp.mozilla.org/pub/firefox/releases/68.1.0esr/linux...

kevingadd 7y ago
My educated guess is that the localization data for every language is pretty massive, so per-region downloads can include the text for their region without dragging along a few extra megabytes of text/images for other regions. If you swap regions after installing it can probably download on-demand. Historically reducing your download size by a couple megabytes has had measurable positive impact on conversion rates (from webpage visit -> install) so I expect this tradeoff made a lot of sense in the past even if it's less important now. EDIT: One other thing worth note is that there's zh-CN and zh-TW, and I can imagine the Chinese government being very irritated about the idea of their release having separate Taiwan loc text/images baked in since according to them Taiwan doesn't exist. Major software like Windows has already had CN-specific releases for this (among other) reason(s).
- Hello71 7y ago
  zh-CN and zh-TW has nothing to do with that, and I'm virtually certain there are no "Taiwan images". in applications which provide zh-CN and zh-TW, including Firefox, the former means Simplified Chinese and the latter means Traditional Chinese. theoretically, it could mean Taiwanese Chinese, but then there would also be zh-HK, zh-MO, zh-SG...
- blaesus 7y ago
  `zh-CN` and `zh-TW` are generally just old-school ways to say "simplified Chinese" and "Traditional Chinese" (two similar writing systems). The confusion comes in as the Chinese language family has multiple dialects and variants, and there are multiple writing systems. The current practice is really a dimension-reduction, where a N×2 matrix [(CN, TW, HK, SG, MY...) × (simplified, traditional)] collapsed into a 2×1 vector (TW, CN). The language labels should have been `zh-hans` and `zh-hant` if one means to differentiate the writing systems, and not the underlying linguistic variants. According to Beijing government, Taiwan absolutely exists, it's just that it is not a sovereign state. Hong Kong has been handed over to China for more than twenty years and people in Hong Kong continue to use zh-HK – Beijing government seems to be OK with that.

nickm12 7y ago

Right. Desktop software is also much more difficult to QA than server software because of the lack of control over the environment and the difficulty of rollback.

nextos 7y ago
Also, for desktop software, I have found that small releases are a lot less risky. You experience small issues, one by one, instead of a massive amount of bugs coming all of a sudden. For example, last time I updated a Linux distribution with 6-month releases this was the case. Whereas with a rolling release one, it's just much more pleasant.
- deckarep 7y ago
  This is true of releasing code in general. Changes that accumulate over time actually increase risk whether it be desktop or server-side.

chrisseaton 7y ago

They must already be doing a full build, test, and benchmark for each platform for each PR to their codebase in CI, presumably in parallel to make it run reasonably fast. It doesn’t seem like much of an extra step to release those binaries they’re already building hundreds of times a day.

uranusjr 7y ago
They already do that with Firefox Nightly. But most users expect a more stable product than that.
- chrisseaton 7y ago
  How can it be more stable than a build that passes CI? If they have some extra tests to run to check it’s stable why aren’t those in CI?
  - pingyong 7y ago
    It can be more stable by having nightly users test it beforehand for a while and report any new bugs.
    - chrisseaton 7y ago
      What are those users doing that can’t be automated and run in CI?
      - chopin 7y ago
        Test on exotic hardware, for example. Or exotic combinations of libraries in the OS which FF depends on.
        chrisseaton 7y ago
        Those should be in the CI test matrix then - we shouldn’t be relying on manual testing done specially for releases.

fragmede 7y ago

They build downloadable binaries every 12 hours, called Firefox nightly, which is to say the bottleneck for full releases isn't building or automated testing. Mind you, the nightlies are experimental, and not for general use.

fluffything 7y ago
Building is in general just a small part of the release pipeline.