4 ms·

> From an operating systems point of view I would go for an intermediate language (similar to .NET or Java bytecode) that I also could do some on-the-fly verifi

> From an operating systems point of view I would go for an intermediate language (similar to .NET or Java bytecode) that I also could do some on-the-fly verification of and only have very little compiled for the actual cpu (at least beyond dedicated embedded systems). There are technical reasons we don't already have a 'universal bytecode' like this. One of them is that it doesn't fit the build model of C and C++. C deliberately doesn't shield the programmer from all details of the underlying architecture, and neither does C++. For instance, the values of sizeof(long) and LONG_MAX depend on the platform, and must be known at compile-time. This is the reason LLVM bitcode cannot be used as an intermediate representation for C programs the way Java bytecode is an IR for Java programs. IRs are already widely used where they make good technical sense. Java, .Net, Python, Lua, and Erlang, each define their own. None of them would work well as a universal intermediate representation. Programs for the Inferno operating-system used a bytecode representation, but modern OSs don't tend to offer 'grand abstractions' like this. See this wise comment from 2015: https://news.ycombinator.com/item?id=9807777 https://news.ycombinator.com/item?id=9807777

throwaway4good 6y ago
WebAssembly is an example of an IL that it is supported by many languages. It solves a lot of problems related to distribution of software and migration between different processor architectures. The Huawei Harmony OS is already touting its verification and how it lets a lot of user code run in the kernel process. Add some just in time compilation and it would be quite a thing.
- my123 6y ago
  Singularity and Midori existed too. + WebAssembly just isn't efficient for translating quite some patterns to it, at least at this point, with a quite significant perf decrease. (yes, it's better than JS in general, but that's not saying much)
pjmlp 6y ago
Sure it does, mainframe language environments and .NET have long supported C and C++. Back in the J2ME days there was a Swedish startup trying to sell a C and C++ version of what was basically a competitor to J2ME. The only reason why LLVM bitcode doesn't do this is political, meaning the LLVM designers don't want to follow down this path.
- MaxBarraclough 6y ago
  > mainframe language environments and .NET have long supported C and C++ I should have been more clear: I'm not saying you can't define an IR language as a target for a C compiler, and then have that IR run on various platforms. As you say, that's already been done with solutions like C++/CLI, and compiling C++ to JavaScript or to WebAssembly. My point is that this isn't the same as what Java bytecode does for Java. When you compile Java to Java bytecode, there's no platform-sensitivity in that compilation step. You can run javac on Windows, and on FreeBSD, and you'll get identical .class files from both. C is importantly different from Java, in two regards: 1. C permits platform-specific use of the preprocessor, so that the programmer can for instance activate a specially tailored Windows-specific version of a function if and only if the target platform is 64-bit Windows. (I'm not fond of the term conditional compilation to describe this, but it seems to have stuck.) 2. Properties of the underlying platform are revealed to the programmer at compile-time, such as in sizeof(long) If you treat your IR as the compilation target, you have to commit to a virtual platform that might not match the underlying platform. You'll need to decide a value for sizeof(long), and the size of pointer types, etc. You can do this, sure enough, but it's presumably going to make things awkward and introduce a possible performance penalty. More importantly though, it's also going to break the way C programmers tailor their programs for different operating systems, how they cope with the availability of different features and optional libraries, etc. Consider the build-specific details that tend to be handled by autotools. Platform-specific preprocessor decisions could also happen in any header file that you rely on. This means it fails to act as a universal portable intermediate representation for C programs. A single universal IR blob isn't going to cope with something like this: void initialize_graphics() { #ifdef USE_DIRECT3D initialize_direct3d(); #else initialize_vulkan(); #endif } Depending on the platform, the function body changes completely. You could single out the IR as a distinct platform: void initialize_graphics() { #if defined WEBASSEMBLY_BUILD initialize_webassembly_graphics(); #elif defined USE_DIRECT3D initialize_direct3d(); #else initialize_vulkan(); #endif } Java, by design, is unable to express compile-time decisions of this sort. Compilation from .java to .class isn't 'lossy' the way compiling C is. To put all this more succinctly: with Java you get the build, with C you get a build. With Java, a .class file a function of a Java source file, whereas with C, a compiled object file is a function of both a C compilation unit and the platform. > The only reason why LLVM bitcode doesn't do this is political, meaning the LLVM designers don't want to follow down this path. On top of what I've mentioned, I doubt they want to be tied to perfect backward-compatibility for LLVM bitcode. Not sure that counts as political though. Of course, Google already gave this a go, with PNaCl.
thu2111 6y ago
You don't need a common IL to abstract the CPU. You just have to more tightly connect the compilers with the shipped computer. Azul Vega did this with JVM bytecode (the Vega chips had an undocumented and sometimes changing ISA), but with GraalVM/Truffle technology you don't need a bytecode anymore. For example, GraalVM can JIT compile LLVM bitcode. Non-portability of bitcode is hardly an issue. OK, fix sizeof(long) to 64 bits, no big deal. GraalVM/Truffle can JIT compile bitcode to the underlying ISA which can then evolve free from backwards compatibility constraints. You can also JIT compiled Python, Ruby, R, JavaScript, Rust, FORTRAN, etc, all with the open source code that exists today. If I were designing a new computer architecture from scratch this is the way I'd go. If you don't need to freeze or even properly document your CPU ISA it gives you way more ability to quickly deploy performance upgrades, quickly iterate on CPU designs etc. You can have a custom OS that bundles GraalVM and QEMU, for the cases where you can't recompile apps.

throwaway4good 6y ago

WebAssembly is an example of an IL that it is supported by many languages. It solves a lot of problems related to distribution of software and migration between different processor architectures. The Huawei Harmony OS is already touting its verification and how it lets a lot of user code run in the kernel process. Add some just in time compilation and it would be quite a thing.

my123 6y ago
Singularity and Midori existed too. + WebAssembly just isn't efficient for translating quite some patterns to it, at least at this point, with a quite significant perf decrease. (yes, it's better than JS in general, but that's not saying much)

pjmlp 6y ago

Sure it does, mainframe language environments and .NET have long supported C and C++. Back in the J2ME days there was a Swedish startup trying to sell a C and C++ version of what was basically a competitor to J2ME. The only reason why LLVM bitcode doesn't do this is political, meaning the LLVM designers don't want to follow down this path.

MaxBarraclough 6y ago
> mainframe language environments and .NET have long supported C and C++ I should have been more clear: I'm not saying you can't define an IR language as a target for a C compiler, and then have that IR run on various platforms. As you say, that's already been done with solutions like C++/CLI, and compiling C++ to JavaScript or to WebAssembly. My point is that this isn't the same as what Java bytecode does for Java. When you compile Java to Java bytecode, there's no platform-sensitivity in that compilation step. You can run javac on Windows, and on FreeBSD, and you'll get identical .class files from both. C is importantly different from Java, in two regards: 1. C permits platform-specific use of the preprocessor, so that the programmer can for instance activate a specially tailored Windows-specific version of a function if and only if the target platform is 64-bit Windows. (I'm not fond of the term conditional compilation to describe this, but it seems to have stuck.) 2. Properties of the underlying platform are revealed to the programmer at compile-time, such as in sizeof(long) If you treat your IR as the compilation target, you have to commit to a virtual platform that might not match the underlying platform. You'll need to decide a value for sizeof(long), and the size of pointer types, etc. You can do this, sure enough, but it's presumably going to make things awkward and introduce a possible performance penalty. More importantly though, it's also going to break the way C programmers tailor their programs for different operating systems, how they cope with the availability of different features and optional libraries, etc. Consider the build-specific details that tend to be handled by autotools. Platform-specific preprocessor decisions could also happen in any header file that you rely on. This means it fails to act as a universal portable intermediate representation for C programs. A single universal IR blob isn't going to cope with something like this: void initialize_graphics() { #ifdef USE_DIRECT3D initialize_direct3d(); #else initialize_vulkan(); #endif } Depending on the platform, the function body changes completely. You could single out the IR as a distinct platform: void initialize_graphics() { #if defined WEBASSEMBLY_BUILD initialize_webassembly_graphics(); #elif defined USE_DIRECT3D initialize_direct3d(); #else initialize_vulkan(); #endif } Java, by design, is unable to express compile-time decisions of this sort. Compilation from .java to .class isn't 'lossy' the way compiling C is. To put all this more succinctly: with Java you get the build, with C you get a build. With Java, a .class file a function of a Java source file, whereas with C, a compiled object file is a function of both a C compilation unit and the platform. > The only reason why LLVM bitcode doesn't do this is political, meaning the LLVM designers don't want to follow down this path. On top of what I've mentioned, I doubt they want to be tied to perfect backward-compatibility for LLVM bitcode. Not sure that counts as political though. Of course, Google already gave this a go, with PNaCl.

thu2111 6y ago

You don't need a common IL to abstract the CPU. You just have to more tightly connect the compilers with the shipped computer. Azul Vega did this with JVM bytecode (the Vega chips had an undocumented and sometimes changing ISA), but with GraalVM/Truffle technology you don't need a bytecode anymore. For example, GraalVM can JIT compile LLVM bitcode. Non-portability of bitcode is hardly an issue. OK, fix sizeof(long) to 64 bits, no big deal. GraalVM/Truffle can JIT compile bitcode to the underlying ISA which can then evolve free from backwards compatibility constraints. You can also JIT compiled Python, Ruby, R, JavaScript, Rust, FORTRAN, etc, all with the open source code that exists today. If I were designing a new computer architecture from scratch this is the way I'd go. If you don't need to freeze or even properly document your CPU ISA it gives you way more ability to quickly deploy performance upgrades, quickly iterate on CPU designs etc. You can have a custom OS that bundles GraalVM and QEMU, for the cases where you can't recompile apps.