7 ms·

As someone who worked on porting Microsoft .NET Core to new platforms, that's definitely a pain I felt. At first, very few assumptions was made about the Unix

As someone who worked on porting Microsoft .NET Core to new platforms, that's definitely a pain I felt. At first, very few assumptions was made about the Unix bootstrapping process, and the mantra was that you should build the managed components on a Windows machine which already had .NET. Just to get something going, the Unix community decided to use mono to bootstrap the process on non-Windows platforms. This effectively meant you could bootstrap .NET Core on almost any platform where mono was around. That gave quite some platform reach out of the box, and on overall an improvement much welcomed by Microsoft. But later, after Unixes like Linux and OSX become better supported, the bootstrapping method changed to downloading a previous build of .NET Core for the current platform and using that to build the latest sources. I.e. going self-bootstrapping, which sounds fancy on paper. The cost of that? Suddenly it just wasn't doable (without considerable knowledge and effort) to bootstrap .NET Core from source any more on platforms not currently considered first class. I'm not just talking about "strange" outlier OSes like FreeBSD, NetBSD or Alpine Linux or whatever. Just trying to build for Fedora 24 (which doesn't have published packages) fails. This is because the Fedora 23 packages are linked against library-versions which are no longer in Fedora 24. You'd think that be easy to fix, but now, even with Fedora 25 betas starting to ship, there's still no official Fedora 24 package exists from Microsoft, so Fedora users (and developers, and porters, etc) are SOL. So now basically just trying to bootstrap for a previously supported mainstream Linux-distro is seriously PITA, next to impossible. In my honest opinion that's a major regression just for the sake of being self-bootstrapping. So yeah. You should never underestimate the value of having secondary platforms available for bootstrapping your own. You may come to depend on it.

lmm 10y ago
> I'm not just talking about "strange" outlier OSes like FreeBSD, NetBSD or Alpine Linux or whatever. Just trying to build for Fedora 24 (which doesn't have published packages) fails. This is because the Fedora 23 packages are linked against library-versions which are no longer in Fedora 24. This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous.
- wtallis 10y ago
  > If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. As a general-purpose application framework/runtime, .NET probably has far broader dependencies than any one application. Maintaining binary compatibility for a dynamically-linked .NET framework is a tall order.
- josteink 10y ago
  > This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. I've seen issues like this for significantly smaller packages on Ubuntu. One example being exactimage, which for some reason weren't included in one single release around 14.04. Trying to copy a single binary across versions here, would cause similar breakage because of the versions of linked dependencies. I think this is "symptomatic" of the Linux world and actually fairly common in software where source is assumed available and recompiling using that source a feasible task. Things like this is usually just not problem in the real world when things are properly packaged. Again: for .NET the problem was not the dependency itself, but the root bootstrapping process.
- snuxoll 10y ago
  Fedora maintains binary compatibility within a specific release, but major versions of software are updated every 6 months and if the ABI is different a soname bump will happen. Unless software in the Fedora repositories flat out will not build or run with a newer ABI-release of a dependency older versions of the library aren't typically packaged. In all honesty it's virtually never an issue, most libraries maintain API compatibility and everything gets a mass-rebuild before alphas start to ship (though everyone tries to notify affected maintainers when they are going to do a soname bump on the mailing list) - so as long as you don't depend on old libraries for bootstrap of new package versions it's fine. There's plenty of compilers in the repositories that bootstrap themselves every release without giving grief, so Microsoft needs to either provide an external bootstrap mechanism again or reduce the dependencies needed to get it bootstrapped to preferably nothing more than glibc.
masklinn 10y ago
Wouldn't a good cross-compilation story work for bootstrapping? Once you've got the system to run on platform A, you can cross-compile for B and use those artefacts to bootstrap B.
- justincormack 10y ago
  Yes that's very useful. Go has that, NetBSD has that, its getting more common at last. It doesn't solvevall the problems but has other benefits.
- Nullabillity 10y ago
  That doesn't really work for a system like Nix (or derivatives like Guix), where binary packages are only considered an optimization, and packages are always rebuilt without having the old version, or other user state, available.
  - oneplane 10y ago
    But Nix and Guix need a base compiler to start with, right?
    - vertex-four 10y ago
      Indeed, though the goal is to keep the closure of base dependencies as small as possible. They don't really want to be depending on anything other than a C/C++ compiler and a shell.
      - Ericson2314 10y ago
        Actually, criss compiling is a solution here. Hydra should be able to deal with packages on one platform depending on packages from another.
        davexunit 10y ago
        How does cross compiling help? How do you build the first Java from source?
Scaevolus 10y ago
Maybe releasing a container with .net core included and using that to bootstrap on other platforms could work? You would run the container with the source code and build directories from the host mounted and run the appropriate command. It's not perfect, since now you have a dependency on Docker (or equivalent), but maybe it's still an improvement? There's also some efforts like [snap](http://snapcraft.io/ http://snapcraft.io/) to make it easier to release binaries with all their dependencies packaged together, but I expect making it work for a complex project like .NET is difficult.
- loeg 10y ago
  We call them "statically linked binaries". Yeah, that would work well for bootstrapping.
  - Scaevolus 10y ago
    Statically linking binaries doesn't help when there's a bunch of auxiliary resources (bytecode/class libraries) that the compiler needs to run. You can bake that into one binary in additional segments, but it can be fiddly and require more extensive code changes to make various fopen() calls read from the binary instead of a standalone file.
    - loeg 10y ago
      Fortunately, this is not the problem described above.
snuxoll 10y ago
Compilers that bootstrap themselves should try to eliminate external dependencies needed to just get the bootstrap going for this exact reason. Hell, Ruby uses itself as part of the build process and one of the first things it does is compile `miniruby` which is a small enough variant of the runtime and libraries that it can run `rake` and a couple other tools. This is actually really sad to hear, because I've been waiting to see proper RPM's for the CoreCLR in Fedora for a long time, but having to constantly bring in new binaries to bootstrap the package is a huge headache that few package maintainers would be willing to take. Maybe provide the option to build a statically linked build that could be shoved into a different package and used to bootstrap a new dynamic version?
int_19h 10y ago
That sounds like it's partly due to poor componentization and/or dependency graph of the framework - correct me if I'm wrong here... When I think of bootstrapping something like .NET, I would assume that most high-level stuff (i.e. large parts of the standard library, compiler and other tooling etc) are all 100% managed. So to run those, all you need is the VM and the part of stdlib that is implemented in native - the rest is just bytecode, and shouldn't have any OS dependencies, so you could easily redistribute and reuse the older build. On the other hand, VM and native parts of stdlib, I would expect to be written in C or C++, and so, while that's where all those pesky dependencies are, that's also the code that you should be able to build with just as C/C++ compiler. So the process would work like so: 1. Build VM. 2. Build minimal stdlib (native parts). 3. Use the preceding to run an older build of full stdlib and compiler, precompiled into bytecode. 4. Use the preceding to rebuild the compiler. 5. Use the preceding to rebuild the stdlib. Which is also something that a fairly simple script should be able to tackle, such that the dev only needs to do "make bootstrap" or something similar. So, where does this picture break?
- josteink 10y ago
  > 1. Build VM. 2. Build minimal stdlib (native parts). 3. Use the preceding to run an older build of full stdlib and compiler, precompiled into bytecode. 4. Use the preceding to rebuild the compiler. 5. Use the preceding to rebuild the stdlib. ... So, where does this picture break? It breaks at point 3. There's 2 distinct things which breaks here, much due to what you call poor componentization. To build the stdlib, you need Nuget (which is written in .NET) to resolve binary dependencies and the Roslyn compiler (also written in .NET). These two components are wrapped up in a simple to use all-in-one command line utility, much like Clojure's leininginen: Basically one tool to rule them all. This tool is also written in .NET. This means you need a prebuilt .NET version, including the base stdlib, to run the toolchain and compiler needed to build the stdlib itself. In theory this should not have to present any issues. Basically so far so good. The problem is that what you need to download will contain 1. corerun/corehost runtimes used to bootstrap the process (platform native binaries) and 2. The managed stdlib, which also contains platform specific binaries (Win32 system-calls on Windows, glibc on Linux, another Libc on FreeBSD, etc). These are not all bundled in one big package where binaries for all platforms are included, but as separate packages per supported platform (Windows, OSX, Fedora 23, Ubuntu 14, Ubuntu 16, etc) So on a previously unbootstrapped platform you can run into any of the following issues: 1. Native components are dynamically linked, and may fail to resolve libraries if they have had their soname/version bumped, even though you can build these components yourself just fine. This is obviously a structural issue with the build process, which IMO should be fairly doable to solve. 2. Because the compiler itself is .NET -based, it relies on the managed stdlib, and this this library itself must be built for your platform (FreeBSD's implementation differs slightly from Linux, etc). So on a platform where nobody has ever done any work before, the download for step 2 will fail and the build will crash. To rectify this you will basically need to get acquainted with dotnet's multitude of individual repos, how they tie together, in what order they should be built, how their build-outputs should be combined, and all kinds of un/under-documented stuff, and do that manually... Phew! ... To package a base "SDK" which can be used to initiate the normal coreclr build-process, which previously failed. You may need to fake this SDK for several dependent repos. That single step is incredibly complex, and I think only a handful of people on the whole internet knows how this is done. Compare this to how things were previously: The process was bootstrapped automatically using mono, and every single coreclr developer knew how to get things up and running. That this was greenlit without any further objections is something I find hard to believe.
danudey 10y ago
Likewise, in order to build Go from source you need to already have Go 1.4 installed. In an OS where you by default download source and compile it (e.g. Gentoo, MacPorts on OS X), this means downloading an old, unsupported, and unmaintained version of Go just to build it to run a newer version. I had an issue recently where, on my laptop, Go 1.4 refused to build, and now I have no versions of Go on my laptop. I could install from binaries from the website, but that wouldn't satisfy dependencies so I'm stuck for now.

lmm 10y ago

> I'm not just talking about "strange" outlier OSes like FreeBSD, NetBSD or Alpine Linux or whatever. Just trying to build for Fedora 24 (which doesn't have published packages) fails. This is because the Fedora 23 packages are linked against library-versions which are no longer in Fedora 24. This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous.

wtallis 10y ago
> If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. As a general-purpose application framework/runtime, .NET probably has far broader dependencies than any one application. Maintaining binary compatibility for a dynamically-linked .NET framework is a tall order.
josteink 10y ago
> This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. I've seen issues like this for significantly smaller packages on Ubuntu. One example being exactimage, which for some reason weren't included in one single release around 14.04. Trying to copy a single binary across versions here, would cause similar breakage because of the versions of linked dependencies. I think this is "symptomatic" of the Linux world and actually fairly common in software where source is assumed available and recompiling using that source a feasible task. Things like this is usually just not problem in the real world when things are properly packaged. Again: for .NET the problem was not the dependency itself, but the root bootstrapping process.
snuxoll 10y ago
Fedora maintains binary compatibility within a specific release, but major versions of software are updated every 6 months and if the ABI is different a soname bump will happen. Unless software in the Fedora repositories flat out will not build or run with a newer ABI-release of a dependency older versions of the library aren't typically packaged. In all honesty it's virtually never an issue, most libraries maintain API compatibility and everything gets a mass-rebuild before alphas start to ship (though everyone tries to notify affected maintainers when they are going to do a soname bump on the mailing list) - so as long as you don't depend on old libraries for bootstrap of new package versions it's fine. There's plenty of compilers in the repositories that bootstrap themselves every release without giving grief, so Microsoft needs to either provide an external bootstrap mechanism again or reduce the dependencies needed to get it bootstrapped to preferably nothing more than glibc.

masklinn 10y ago

Wouldn't a good cross-compilation story work for bootstrapping? Once you've got the system to run on platform A, you can cross-compile for B and use those artefacts to bootstrap B.

justincormack 10y ago
Yes that's very useful. Go has that, NetBSD has that, its getting more common at last. It doesn't solvevall the problems but has other benefits.
Nullabillity 10y ago
That doesn't really work for a system like Nix (or derivatives like Guix), where binary packages are only considered an optimization, and packages are always rebuilt without having the old version, or other user state, available.
- oneplane 10y ago
  But Nix and Guix need a base compiler to start with, right?
  - vertex-four 10y ago
    Indeed, though the goal is to keep the closure of base dependencies as small as possible. They don't really want to be depending on anything other than a C/C++ compiler and a shell.
    - Ericson2314 10y ago
      Actually, criss compiling is a solution here. Hydra should be able to deal with packages on one platform depending on packages from another.
      - davexunit 10y ago
        How does cross compiling help? How do you build the first Java from source?

Scaevolus 10y ago

Maybe releasing a container with .net core included and using that to bootstrap on other platforms could work? You would run the container with the source code and build directories from the host mounted and run the appropriate command. It's not perfect, since now you have a dependency on Docker (or equivalent), but maybe it's still an improvement? There's also some efforts like [snap](http://snapcraft.io/ http://snapcraft.io/) to make it easier to release binaries with all their dependencies packaged together, but I expect making it work for a complex project like .NET is difficult.

loeg 10y ago
We call them "statically linked binaries". Yeah, that would work well for bootstrapping.
- Scaevolus 10y ago
  Statically linking binaries doesn't help when there's a bunch of auxiliary resources (bytecode/class libraries) that the compiler needs to run. You can bake that into one binary in additional segments, but it can be fiddly and require more extensive code changes to make various fopen() calls read from the binary instead of a standalone file.
  - loeg 10y ago
    Fortunately, this is not the problem described above.

snuxoll 10y ago

Compilers that bootstrap themselves should try to eliminate external dependencies needed to just get the bootstrap going for this exact reason. Hell, Ruby uses itself as part of the build process and one of the first things it does is compile `miniruby` which is a small enough variant of the runtime and libraries that it can run `rake` and a couple other tools. This is actually really sad to hear, because I've been waiting to see proper RPM's for the CoreCLR in Fedora for a long time, but having to constantly bring in new binaries to bootstrap the package is a huge headache that few package maintainers would be willing to take. Maybe provide the option to build a statically linked build that could be shoved into a different package and used to bootstrap a new dynamic version?

int_19h 10y ago

That sounds like it's partly due to poor componentization and/or dependency graph of the framework - correct me if I'm wrong here... When I think of bootstrapping something like .NET, I would assume that most high-level stuff (i.e. large parts of the standard library, compiler and other tooling etc) are all 100% managed. So to run those, all you need is the VM and the part of stdlib that is implemented in native - the rest is just bytecode, and shouldn't have any OS dependencies, so you could easily redistribute and reuse the older build. On the other hand, VM and native parts of stdlib, I would expect to be written in C or C++, and so, while that's where all those pesky dependencies are, that's also the code that you should be able to build with just as C/C++ compiler. So the process would work like so: 1. Build VM. 2. Build minimal stdlib (native parts). 3. Use the preceding to run an older build of full stdlib and compiler, precompiled into bytecode. 4. Use the preceding to rebuild the compiler. 5. Use the preceding to rebuild the stdlib. Which is also something that a fairly simple script should be able to tackle, such that the dev only needs to do "make bootstrap" or something similar. So, where does this picture break?

josteink 10y ago
> 1. Build VM. 2. Build minimal stdlib (native parts). 3. Use the preceding to run an older build of full stdlib and compiler, precompiled into bytecode. 4. Use the preceding to rebuild the compiler. 5. Use the preceding to rebuild the stdlib. ... So, where does this picture break? It breaks at point 3. There's 2 distinct things which breaks here, much due to what you call poor componentization. To build the stdlib, you need Nuget (which is written in .NET) to resolve binary dependencies and the Roslyn compiler (also written in .NET). These two components are wrapped up in a simple to use all-in-one command line utility, much like Clojure's leininginen: Basically one tool to rule them all. This tool is also written in .NET. This means you need a prebuilt .NET version, including the base stdlib, to run the toolchain and compiler needed to build the stdlib itself. In theory this should not have to present any issues. Basically so far so good. The problem is that what you need to download will contain 1. corerun/corehost runtimes used to bootstrap the process (platform native binaries) and 2. The managed stdlib, which also contains platform specific binaries (Win32 system-calls on Windows, glibc on Linux, another Libc on FreeBSD, etc). These are not all bundled in one big package where binaries for all platforms are included, but as separate packages per supported platform (Windows, OSX, Fedora 23, Ubuntu 14, Ubuntu 16, etc) So on a previously unbootstrapped platform you can run into any of the following issues: 1. Native components are dynamically linked, and may fail to resolve libraries if they have had their soname/version bumped, even though you can build these components yourself just fine. This is obviously a structural issue with the build process, which IMO should be fairly doable to solve. 2. Because the compiler itself is .NET -based, it relies on the managed stdlib, and this this library itself must be built for your platform (FreeBSD's implementation differs slightly from Linux, etc). So on a platform where nobody has ever done any work before, the download for step 2 will fail and the build will crash. To rectify this you will basically need to get acquainted with dotnet's multitude of individual repos, how they tie together, in what order they should be built, how their build-outputs should be combined, and all kinds of un/under-documented stuff, and do that manually... Phew! ... To package a base "SDK" which can be used to initiate the normal coreclr build-process, which previously failed. You may need to fake this SDK for several dependent repos. That single step is incredibly complex, and I think only a handful of people on the whole internet knows how this is done. Compare this to how things were previously: The process was bootstrapped automatically using mono, and every single coreclr developer knew how to get things up and running. That this was greenlit without any further objections is something I find hard to believe.

danudey 10y ago

Likewise, in order to build Go from source you need to already have Go 1.4 installed. In an OS where you by default download source and compile it (e.g. Gentoo, MacPorts on OS X), this means downloading an old, unsupported, and unmaintained version of Go just to build it to run a newer version. I had an issue recently where, on my laptop, Go 1.4 refused to build, and now I have no versions of Go on my laptop. I could install from binaries from the website, but that wouldn't satisfy dependencies so I'm stuck for now.