3 ms·

How far do you think something like Docker/Vagrant build images would go to solve that problem? I imagine an image with the full toolchain installed, geared to

How far do you think something like Docker/Vagrant build images would go to solve that problem? I imagine an image with the full toolchain installed, geared towards using it as a kind of "firefox-compiler" CLI (so coupled with a few shell scripts), that only requires you to know how to install Docker|Vagrant. This could easily be bundled with the source code.

annnnd 10y ago
My thoughts exactly. Whenever I tinker with some new software I try to do it in a Docker container (writing all relevant commands to Dockerfile) so I can remove the changes easily when I want to.
lmm 10y ago
I think that would be more overhead, at least to the kind of people who have the native-code experience to be able to contribute to Firefox. (As more of the browser becomes Rust-based it will hopefully be more accessible for contribution, but once you're in the Rust world Cargo solves your problem a lot better than Docker does). And what do you do about the people left behind by Docker's extremely limited platform support?
- vidarh 10y ago
  A Dockerfile is basically a set of instructions to run to produce a working image. Nothing prevents you from running those instructions outside of Docker. And nothing prevents you from structuring your Dockerfiles like: FROM suitable-base-image ADD ./install.sh /tmp RUN chmod a+rx /tmp/install.sh && /tmp/install.sh and write a script to install the dependencies that people can opt to run outside of Docker if they prefer. The point is not to take away peoples ability to pull together all of the dependencies on their own, but to make replicating the build environment trivial. Personally I don't want install scripts etc. polluting my server or laptop setups - I always spin up containers to run builds in these days, because it means I at the end have a repeatable description of how to set it up. But once someone has done that job once, it's silly for everyone else to have to repeat it.
  - lmm 10y ago
    Install scripts were always the most opaque and difficult to use way to get a piece of software. The problem with Docker is that it encourages an opaque style of dependencies where you stop wondering or tracking which things are required and which are accidental. You stop depending on "foolib version 2.3 or later" and start depending on "foolib version 2.4.1p2". You stop being able to export the dependency graph and inspect it, because all you can do with a dockerfile is run it. Docker maybe makes sense as a "compilation target" - generating a dockerfile of your dependencies to make it easy for people to use makes sense. But it's not rich or structured enough to be the canonical record of what dependencies your project needs.
    - vidarh 10y ago
      > But it's not rich or structured enough to be the canonical record of what dependencies your project needs. I'm not arguing for it to be that, and I largly agree with you. As my example shows, the Dockerfile doesn't need to contain any detail of the dependencies - you can "outsource" that entirely to whatever mechanism you wish, as long as you can easily add whatever you need to apply those dependencies to a suitable base image to the Dockerfile. But most of the time I don't care about precision - I just want a means of getting a working build environment quickly. If it pulls in 100MB of unnecessary libraries, I don't care, if it gets me an uncomplicated way of building the project without having to assemble the dependencies myself. For a lot of projects I come across, my experience is that because the list of build dependencies are often never automatically tested, it's a crapshoot whether or not their build instructions and dependency list will yield a working build environment on any given system. "Even" a plain Dockerfile that is regularly used to rebuild the build environment is far superior to a text file of dependencies that is rarely to never verified. If you want to go one better and storing more precise instructions and then use them to generate a build script or Dockerfile, then awesome. The main thing is that absent automated testing of the build environment too, the list of dependencies isn't worth the bytes they're stored in. And a simple way of regularly testing the build environment is to rebuild it automatically. My experience is that the dependency lists for projects that automatically rebuild the build environment - whether as Docker containers or in a chroot or any number of other ways - tends to be far more reliable. And if you first can automatically build the build environment, it makes sense to make it easy for everyone to use that automatically built build environment.
vidarh 10y ago
My thought too. I write the build processes for all my newer software explicitly as a two stage process: A Dockerfile with all the dependencies, including pre-requisite Makefiles etc., and a simple Makefile that has targets to (re-)build the build container and run builds in it. If people want to install all the build dependencies themselves, that's fine, but there's really no reason to make people go through that hassle. Automating and isolating it also makes it far more liely that the dependency list remains accurate.
hashhar 10y ago
I actually published such an image. You can find it at https://hub.docker.com/r/hashhar/mozilla-build/ https://hub.docker.com/r/hashhar/mozilla-build/ It only includes the build tools though. You need the source code separately. I think I have provided enough info in the README to get you going.

annnnd 10y ago

My thoughts exactly. Whenever I tinker with some new software I try to do it in a Docker container (writing all relevant commands to Dockerfile) so I can remove the changes easily when I want to.

lmm 10y ago

I think that would be more overhead, at least to the kind of people who have the native-code experience to be able to contribute to Firefox. (As more of the browser becomes Rust-based it will hopefully be more accessible for contribution, but once you're in the Rust world Cargo solves your problem a lot better than Docker does). And what do you do about the people left behind by Docker's extremely limited platform support?

vidarh 10y ago
A Dockerfile is basically a set of instructions to run to produce a working image. Nothing prevents you from running those instructions outside of Docker. And nothing prevents you from structuring your Dockerfiles like: FROM suitable-base-image ADD ./install.sh /tmp RUN chmod a+rx /tmp/install.sh && /tmp/install.sh and write a script to install the dependencies that people can opt to run outside of Docker if they prefer. The point is not to take away peoples ability to pull together all of the dependencies on their own, but to make replicating the build environment trivial. Personally I don't want install scripts etc. polluting my server or laptop setups - I always spin up containers to run builds in these days, because it means I at the end have a repeatable description of how to set it up. But once someone has done that job once, it's silly for everyone else to have to repeat it.
- lmm 10y ago
  Install scripts were always the most opaque and difficult to use way to get a piece of software. The problem with Docker is that it encourages an opaque style of dependencies where you stop wondering or tracking which things are required and which are accidental. You stop depending on "foolib version 2.3 or later" and start depending on "foolib version 2.4.1p2". You stop being able to export the dependency graph and inspect it, because all you can do with a dockerfile is run it. Docker maybe makes sense as a "compilation target" - generating a dockerfile of your dependencies to make it easy for people to use makes sense. But it's not rich or structured enough to be the canonical record of what dependencies your project needs.
  - vidarh 10y ago
    > But it's not rich or structured enough to be the canonical record of what dependencies your project needs. I'm not arguing for it to be that, and I largly agree with you. As my example shows, the Dockerfile doesn't need to contain any detail of the dependencies - you can "outsource" that entirely to whatever mechanism you wish, as long as you can easily add whatever you need to apply those dependencies to a suitable base image to the Dockerfile. But most of the time I don't care about precision - I just want a means of getting a working build environment quickly. If it pulls in 100MB of unnecessary libraries, I don't care, if it gets me an uncomplicated way of building the project without having to assemble the dependencies myself. For a lot of projects I come across, my experience is that because the list of build dependencies are often never automatically tested, it's a crapshoot whether or not their build instructions and dependency list will yield a working build environment on any given system. "Even" a plain Dockerfile that is regularly used to rebuild the build environment is far superior to a text file of dependencies that is rarely to never verified. If you want to go one better and storing more precise instructions and then use them to generate a build script or Dockerfile, then awesome. The main thing is that absent automated testing of the build environment too, the list of dependencies isn't worth the bytes they're stored in. And a simple way of regularly testing the build environment is to rebuild it automatically. My experience is that the dependency lists for projects that automatically rebuild the build environment - whether as Docker containers or in a chroot or any number of other ways - tends to be far more reliable. And if you first can automatically build the build environment, it makes sense to make it easy for everyone to use that automatically built build environment.

vidarh 10y ago

My thought too. I write the build processes for all my newer software explicitly as a two stage process: A Dockerfile with all the dependencies, including pre-requisite Makefiles etc., and a simple Makefile that has targets to (re-)build the build container and run builds in it. If people want to install all the build dependencies themselves, that's fine, but there's really no reason to make people go through that hassle. Automating and isolating it also makes it far more liely that the dependency list remains accurate.

hashhar 10y ago

I actually published such an image. You can find it at https://hub.docker.com/r/hashhar/mozilla-build/ https://hub.docker.com/r/hashhar/mozilla-build/ It only includes the build tools though. You need the source code separately. I think I have provided enough info in the README to get you going.