8 ms·

This is a little sad because this means we will have to keep around old versions of GCC with GCJ in order to be able to bootstrap the JDK from source (without a

by rekado 10y ago

This is a little sad because this means we will have to keep around old versions of GCC with GCJ in order to be able to bootstrap the JDK from source (without a binary bootstrap JDK). Only OpenJDK6 (with the extensive IcedTea patches) can be bootstrapped with GCJ alone, so we also have to keep OpenJDK6 around in order to be able to build the latest JDK. Bootstrapping becomes harder and more legacy software needs to be accumulated in order to actually build new software from source. (I'm working on GNU Guix, a functional package manager, and I wrote the package definitions for the JDKs.)

josteink 10y ago
As someone who worked on porting Microsoft .NET Core to new platforms, that's definitely a pain I felt. At first, very few assumptions was made about the Unix bootstrapping process, and the mantra was that you should build the managed components on a Windows machine which already had .NET. Just to get something going, the Unix community decided to use mono to bootstrap the process on non-Windows platforms. This effectively meant you could bootstrap .NET Core on almost any platform where mono was around. That gave quite some platform reach out of the box, and on overall an improvement much welcomed by Microsoft. But later, after Unixes like Linux and OSX become better supported, the bootstrapping method changed to downloading a previous build of .NET Core for the current platform and using that to build the latest sources. I.e. going self-bootstrapping, which sounds fancy on paper. The cost of that? Suddenly it just wasn't doable (without considerable knowledge and effort) to bootstrap .NET Core from source any more on platforms not currently considered first class. I'm not just talking about "strange" outlier OSes like FreeBSD, NetBSD or Alpine Linux or whatever. Just trying to build for Fedora 24 (which doesn't have published packages) fails. This is because the Fedora 23 packages are linked against library-versions which are no longer in Fedora 24. You'd think that be easy to fix, but now, even with Fedora 25 betas starting to ship, there's still no official Fedora 24 package exists from Microsoft, so Fedora users (and developers, and porters, etc) are SOL. So now basically just trying to bootstrap for a previously supported mainstream Linux-distro is seriously PITA, next to impossible. In my honest opinion that's a major regression just for the sake of being self-bootstrapping. So yeah. You should never underestimate the value of having secondary platforms available for bootstrapping your own. You may come to depend on it.
- lmm 10y ago
  > I'm not just talking about "strange" outlier OSes like FreeBSD, NetBSD or Alpine Linux or whatever. Just trying to build for Fedora 24 (which doesn't have published packages) fails. This is because the Fedora 23 packages are linked against library-versions which are no longer in Fedora 24. This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous.
  - wtallis 10y ago
    > If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. As a general-purpose application framework/runtime, .NET probably has far broader dependencies than any one application. Maintaining binary compatibility for a dynamically-linked .NET framework is a tall order.
  - josteink 10y ago
    > This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. I've seen issues like this for significantly smaller packages on Ubuntu. One example being exactimage, which for some reason weren't included in one single release around 14.04. Trying to copy a single binary across versions here, would cause similar breakage because of the versions of linked dependencies. I think this is "symptomatic" of the Linux world and actually fairly common in software where source is assumed available and recompiling using that source a feasible task. Things like this is usually just not problem in the real world when things are properly packaged. Again: for .NET the problem was not the dependency itself, but the root bootstrapping process.
  - snuxoll 10y ago
    Fedora maintains binary compatibility within a specific release, but major versions of software are updated every 6 months and if the ABI is different a soname bump will happen. Unless software in the Fedora repositories flat out will not build or run with a newer ABI-release of a dependency older versions of the library aren't typically packaged. In all honesty it's virtually never an issue, most libraries maintain API compatibility and everything gets a mass-rebuild before alphas start to ship (though everyone tries to notify affected maintainers when they are going to do a soname bump on the mailing list) - so as long as you don't depend on old libraries for bootstrap of new package versions it's fine. There's plenty of compilers in the repositories that bootstrap themselves every release without giving grief, so Microsoft needs to either provide an external bootstrap mechanism again or reduce the dependencies needed to get it bootstrapped to preferably nothing more than glibc.
lmm 10y ago
Was bootstrapping GCJ from source any easier than bootstrapping OpenJDK would be? GCC itself is very complex and while it has a bootstrap process that still relies on having a C compiler AIUI.
- davexunit 10y ago
  GCC is in the set of bootstrap binaries, which needs to be kept as small as possible. It's imperative to raise awareness of this issue so that language implementors provide a way to bootstrap their compilers. For example, Guile Scheme can bootstrap itself from source using an interpreter written in C.
rjsw 10y ago
I would have thought that Jikes was a better bootstrap compiler than GCJ.
- nn3 10y ago
  Isn't jikes written in Java? You would need to bootstrap it first.
  - correktlogic 10y ago
    I was a contributor to Jikes back in the day. The open sourced IBM Jikes compiler was written in C++. But it didn't produce native code. It produced .class files from .java files.
    - rjsw 10y ago
      You need to be able to produce .class files, as well as C headers, to bootstrap the Sun JDK. You don't need to compile Java to native code.
      - correktlogic 10y ago
        Jikes didn't have any javah-like functionality to generate .h files to my knowledge.
- mickronome 10y ago
  I think Jikes RVM needed some kind of bytecode interpreter to build itself, but I don't think it would need to be very advanced. Slow would also be OK since the part getting compiled was not terribly big. Don't know if it could help in any practical way, but it's a fun piece of technology!
zck 10y ago
This is completely off-topic, but I've been having a hell of a time trying to figure out how to, on Guix, install the JDKs and then set JAVA_HOME properly. Is there documentation about how to do that? I'm sure I'm just missing something. I'm happy to talk either here, or my email's in my profile, if you prefer.
civodul 10y ago
Seconded. With GCJ, Java had a bootstrapping path from C/C++, unlike all the current single-implementation languages: Rust, GHC, etc. Having this path is important from a security and user freedom viewpoint: we could build OpenJDK from source. Not having such a bootstrap path means relying on huge opaque binary blobs, which is a net loss.
- pcwalton 10y ago
  Well, for Rust the binary blob isn't opaque: you can go back through the source code history and find the exact source for every revision of the blob, all the way back to the OCaml bootstrap implementation. There's no loss of "user freedom" at any step, just a lot of complexity (but compared to the complexity of, say, GCC it's a drop in the bucket).

by rekado 10y ago

josteink 10y ago

As someone who worked on porting Microsoft .NET Core to new platforms, that's definitely a pain I felt. At first, very few assumptions was made about the Unix bootstrapping process, and the mantra was that you should build the managed components on a Windows machine which already had .NET. Just to get something going, the Unix community decided to use mono to bootstrap the process on non-Windows platforms. This effectively meant you could bootstrap .NET Core on almost any platform where mono was around. That gave quite some platform reach out of the box, and on overall an improvement much welcomed by Microsoft. But later, after Unixes like Linux and OSX become better supported, the bootstrapping method changed to downloading a previous build of .NET Core for the current platform and using that to build the latest sources. I.e. going self-bootstrapping, which sounds fancy on paper. The cost of that? Suddenly it just wasn't doable (without considerable knowledge and effort) to bootstrap .NET Core from source any more on platforms not currently considered first class. I'm not just talking about "strange" outlier OSes like FreeBSD, NetBSD or Alpine Linux or whatever. Just trying to build for Fedora 24 (which doesn't have published packages) fails. This is because the Fedora 23 packages are linked against library-versions which are no longer in Fedora 24. You'd think that be easy to fix, but now, even with Fedora 25 betas starting to ship, there's still no official Fedora 24 package exists from Microsoft, so Fedora users (and developers, and porters, etc) are SOL. So now basically just trying to bootstrap for a previously supported mainstream Linux-distro is seriously PITA, next to impossible. In my honest opinion that's a major regression just for the sake of being self-bootstrapping. So yeah. You should never underestimate the value of having secondary platforms available for bootstrapping your own. You may come to depend on it.

lmm 10y ago
> I'm not just talking about "strange" outlier OSes like FreeBSD, NetBSD or Alpine Linux or whatever. Just trying to build for Fedora 24 (which doesn't have published packages) fails. This is because the Fedora 23 packages are linked against library-versions which are no longer in Fedora 24. This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous.
- wtallis 10y ago
  > If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. As a general-purpose application framework/runtime, .NET probably has far broader dependencies than any one application. Maintaining binary compatibility for a dynamically-linked .NET framework is a tall order.
- josteink 10y ago
  > This sounds like more of an issue with Fedora than anything else. If Fedora 24 can't run programs for Fedora 23 (not ancient programs that happened to work on Fedora 23, but programs that were specifically built for Fedora 23), that's pretty ridiculous. I've seen issues like this for significantly smaller packages on Ubuntu. One example being exactimage, which for some reason weren't included in one single release around 14.04. Trying to copy a single binary across versions here, would cause similar breakage because of the versions of linked dependencies. I think this is "symptomatic" of the Linux world and actually fairly common in software where source is assumed available and recompiling using that source a feasible task. Things like this is usually just not problem in the real world when things are properly packaged. Again: for .NET the problem was not the dependency itself, but the root bootstrapping process.
- snuxoll 10y ago
  Fedora maintains binary compatibility within a specific release, but major versions of software are updated every 6 months and if the ABI is different a soname bump will happen. Unless software in the Fedora repositories flat out will not build or run with a newer ABI-release of a dependency older versions of the library aren't typically packaged. In all honesty it's virtually never an issue, most libraries maintain API compatibility and everything gets a mass-rebuild before alphas start to ship (though everyone tries to notify affected maintainers when they are going to do a soname bump on the mailing list) - so as long as you don't depend on old libraries for bootstrap of new package versions it's fine. There's plenty of compilers in the repositories that bootstrap themselves every release without giving grief, so Microsoft needs to either provide an external bootstrap mechanism again or reduce the dependencies needed to get it bootstrapped to preferably nothing more than glibc.

lmm 10y ago

Was bootstrapping GCJ from source any easier than bootstrapping OpenJDK would be? GCC itself is very complex and while it has a bootstrap process that still relies on having a C compiler AIUI.

davexunit 10y ago
GCC is in the set of bootstrap binaries, which needs to be kept as small as possible. It's imperative to raise awareness of this issue so that language implementors provide a way to bootstrap their compilers. For example, Guile Scheme can bootstrap itself from source using an interpreter written in C.

rjsw 10y ago

I would have thought that Jikes was a better bootstrap compiler than GCJ.

nn3 10y ago
Isn't jikes written in Java? You would need to bootstrap it first.
- correktlogic 10y ago
  I was a contributor to Jikes back in the day. The open sourced IBM Jikes compiler was written in C++. But it didn't produce native code. It produced .class files from .java files.
  - rjsw 10y ago
    You need to be able to produce .class files, as well as C headers, to bootstrap the Sun JDK. You don't need to compile Java to native code.
    - correktlogic 10y ago
      Jikes didn't have any javah-like functionality to generate .h files to my knowledge.
mickronome 10y ago
I think Jikes RVM needed some kind of bytecode interpreter to build itself, but I don't think it would need to be very advanced. Slow would also be OK since the part getting compiled was not terribly big. Don't know if it could help in any practical way, but it's a fun piece of technology!

zck 10y ago

This is completely off-topic, but I've been having a hell of a time trying to figure out how to, on Guix, install the JDKs and then set JAVA_HOME properly. Is there documentation about how to do that? I'm sure I'm just missing something. I'm happy to talk either here, or my email's in my profile, if you prefer.

civodul 10y ago

Seconded. With GCJ, Java had a bootstrapping path from C/C++, unlike all the current single-implementation languages: Rust, GHC, etc. Having this path is important from a security and user freedom viewpoint: we could build OpenJDK from source. Not having such a bootstrap path means relying on huge opaque binary blobs, which is a net loss.

pcwalton 10y ago
Well, for Rust the binary blob isn't opaque: you can go back through the source code history and find the exact source for every revision of the blob, all the way back to the OCaml bootstrap implementation. There's no loss of "user freedom" at any step, just a lot of complexity (but compared to the complexity of, say, GCC it's a drop in the bucket).