5 ms·

Sorry for being naive, but how can it be possible that nobody had thought of this approach and this approach would have actually worked? What have been prevent

Sorry for being naive, but how can it be possible that nobody had thought of this approach and this approach would have actually worked? What have been preventing people from coming up with something like this?

mmastrac 6y ago
I think it's been pondered a bunch, but the legwork required to build the headers for each system so that you can execute the same binaries on every platform was tough.
Sebb767 6y ago
I can think of two reasons: It's arguably a hack and it's not needed. While this is awesome, it can easily break (zsh already seems to do so) and, as the author states, it is not meant for GUI apps or programs relying deeply on OS APIs, which includes a lot of apps. Additionaly, it does not solve an important problem. Doing a build for each supported OS is not that hard and easily automated. The actual problem is to be able to reuse code and this is solved for the most part.
- jart 6y ago
  Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI and a WIN32 GUI in the same binary! Even if you turn the WIN32 off, it'll still run the TUI inside the Windows Command Prompt. As for breakages, Cosmopolitan only depends on stable kernel ABIs with long term stability promises. The binaries you build will work on all Linux distros dating back to RHEL5, Windows versions dating back to Vista, etc. There's no reason to believe there will be any radical breaking changes to these ABIs. Why is it so stable? Alignment of self-interest. By using canonical kernel interfaces, we ensure platforms can't break us without first breaking themselves.
  - ludocode 6y ago
    > Cosmopolitan only depends on stable kernel ABIs with long term stability promises Unfortunately, on macOS a stable kernel ABI does not exist. macOS has had kernel ABI breakage before, even recently, and they explicitly warn against making your own system calls. On Windows things are more stable but that's only because Windows cares about backwards compatibility to the point of preserving the behavior of even the most badly written software. Linux is the only one of the three that promises a stable kernel ABI. On macOS and Windows the system libc is the only safe way to talk to the kernel.
    - moonchild 6y ago
      Ntdll is the kernel ABI on windows; it is stable, and cosmopolitan uses it. Not sure what you mean by ‘on Windows thing are more stable’; syscall numbers change all the time. See https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/
      - jart 6y ago
        Cosmopolitan uses the stable WIN32 API on Windows. The word ABI is used loosely in this context. It's somewhat accurate since Cosmopolitan still configures the assembler to generate the binary linkage by hand. NTDLL APIs are only used on rare occasions, such as sched_yield(). When it is used, it's used in accordance with Microsoft's recommendations: namely having fallback. It'd be great if we were authorized to use the SYSCALL instruction on Windows, however we have to respect Microsoft's recommendations if we want long-term stability promises. Mac however is a different story. One of the reasons Apple came back is they adopted the UNIX interface. They literally copied the magic numbers right out of the UNIX codebase. They shouldn't have the right to stand on the shoulders of giants, pull an about face, and then declare it their own internal API. Apple actually broke every single Go application a few years ago by doing that, and I'm astounded that Google didn't do more to push back. We need to give Apple as much feedback as possible about keeping the SYSCALL interface stable. It's the right thing to do.
        6y ago
choppaface 6y ago
Not an answer, but Splunk currently has a large deployment of a C++ application that supports a large variety of platforms. At least as of a few years ago, they deployed C++ indexing and querying nodes on customer machines. They've spent a great deal of time to accommodate the peculiarities of enterprise customers, old compilers, etc. One might imagine that a "build once run anywhere" platform would benefit them, but they've found a business model where they were able to work around the peculiarities and still be profitable. So while there's probably some interesting historical reasons that explain why there's no "JVM for C++", Splunk is a good example of a modern technical / business model where conquering platform fragmentation was feasible.
jart 6y ago
Author here. Here's the best explanation I've thought of so far: > If it's this easy, why has no one done this before? The best answer I can tell is it requires an minor ABI change, where C preprocessor macros relating to system interfaces need to be symbolic. This is barely an issue, except in cases like switch(errno){case EINVAL:...}. https://justine.lol/ape.html https://justine.lol/ape.html It also took an act of willpower. This is a novel approach to building software that says the conventional wisdom is wrong. When I first stumbled upon the idea, I couldn't believe it was even possible, until I actually did it. Now I'm glad that I did. Actually Portable Executable has been so useful for my work. I used to work on the TensorFlow team, so I've seen the worst c/c++ build toil imaginable, and looking back at those experiences I just can't believe how I managed to get by for so long without having portable build-once binaries without interpreters.
- jes 6y ago
  Have you tried to code-sign executables produced by this method?
  - GordonS 6y ago
    On Windows, Authenticode signatures rely on and use PE header - I'd also love if the author could answer this one. If it does work, I'd also like to know why it works :)
  - umanghere 6y ago
    On macOS, these executables can be signed with a detached signature. Surprisingly, the embedded codesignature also works (but the signature is stored in an extended attribute on the filesystem).
- moonchild 6y ago
  Why not keep constants constant (so your switch example works), but have an extra translation step in front of the syscalls?
- qlk1123 6y ago
  Fair enough and inspiring. However I am still wondering how to advocate changes in such a scale. For example, Linux introduces some fixes for Y2038 problem during the past few years. I suppose the community of this C library should do something for those changes, right?

mmastrac 6y ago

I think it's been pondered a bunch, but the legwork required to build the headers for each system so that you can execute the same binaries on every platform was tough.

Sebb767 6y ago

I can think of two reasons: It's arguably a hack and it's not needed. While this is awesome, it can easily break (zsh already seems to do so) and, as the author states, it is not meant for GUI apps or programs relying deeply on OS APIs, which includes a lot of apps. Additionaly, it does not solve an important problem. Doing a build for each supported OS is not that hard and easily automated. The actual problem is to be able to reuse code and this is solved for the most part.

jart 6y ago
Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI and a WIN32 GUI in the same binary! Even if you turn the WIN32 off, it'll still run the TUI inside the Windows Command Prompt. As for breakages, Cosmopolitan only depends on stable kernel ABIs with long term stability promises. The binaries you build will work on all Linux distros dating back to RHEL5, Windows versions dating back to Vista, etc. There's no reason to believe there will be any radical breaking changes to these ABIs. Why is it so stable? Alignment of self-interest. By using canonical kernel interfaces, we ensure platforms can't break us without first breaking themselves.
- ludocode 6y ago
  > Cosmopolitan only depends on stable kernel ABIs with long term stability promises Unfortunately, on macOS a stable kernel ABI does not exist. macOS has had kernel ABI breakage before, even recently, and they explicitly warn against making your own system calls. On Windows things are more stable but that's only because Windows cares about backwards compatibility to the point of preserving the behavior of even the most badly written software. Linux is the only one of the three that promises a stable kernel ABI. On macOS and Windows the system libc is the only safe way to talk to the kernel.
  - moonchild 6y ago
    Ntdll is the kernel ABI on windows; it is stable, and cosmopolitan uses it. Not sure what you mean by ‘on Windows thing are more stable’; syscall numbers change all the time. See https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/
    - jart 6y ago
      Cosmopolitan uses the stable WIN32 API on Windows. The word ABI is used loosely in this context. It's somewhat accurate since Cosmopolitan still configures the assembler to generate the binary linkage by hand. NTDLL APIs are only used on rare occasions, such as sched_yield(). When it is used, it's used in accordance with Microsoft's recommendations: namely having fallback. It'd be great if we were authorized to use the SYSCALL instruction on Windows, however we have to respect Microsoft's recommendations if we want long-term stability promises. Mac however is a different story. One of the reasons Apple came back is they adopted the UNIX interface. They literally copied the magic numbers right out of the UNIX codebase. They shouldn't have the right to stand on the shoulders of giants, pull an about face, and then declare it their own internal API. Apple actually broke every single Go application a few years ago by doing that, and I'm astounded that Google didn't do more to push back. We need to give Apple as much feedback as possible about keeping the SYSCALL interface stable. It's the right thing to do.
      - 6y ago

choppaface 6y ago

Not an answer, but Splunk currently has a large deployment of a C++ application that supports a large variety of platforms. At least as of a few years ago, they deployed C++ indexing and querying nodes on customer machines. They've spent a great deal of time to accommodate the peculiarities of enterprise customers, old compilers, etc. One might imagine that a "build once run anywhere" platform would benefit them, but they've found a business model where they were able to work around the peculiarities and still be profitable. So while there's probably some interesting historical reasons that explain why there's no "JVM for C++", Splunk is a good example of a modern technical / business model where conquering platform fragmentation was feasible.

jart 6y ago

Author here. Here's the best explanation I've thought of so far: > If it's this easy, why has no one done this before? The best answer I can tell is it requires an minor ABI change, where C preprocessor macros relating to system interfaces need to be symbolic. This is barely an issue, except in cases like switch(errno){case EINVAL:...}. https://justine.lol/ape.html https://justine.lol/ape.html It also took an act of willpower. This is a novel approach to building software that says the conventional wisdom is wrong. When I first stumbled upon the idea, I couldn't believe it was even possible, until I actually did it. Now I'm glad that I did. Actually Portable Executable has been so useful for my work. I used to work on the TensorFlow team, so I've seen the worst c/c++ build toil imaginable, and looking back at those experiences I just can't believe how I managed to get by for so long without having portable build-once binaries without interpreters.

jes 6y ago
Have you tried to code-sign executables produced by this method?
- GordonS 6y ago
  On Windows, Authenticode signatures rely on and use PE header - I'd also love if the author could answer this one. If it does work, I'd also like to know why it works :)
- umanghere 6y ago
  On macOS, these executables can be signed with a detached signature. Surprisingly, the embedded codesignature also works (but the signature is stored in an extended attribute on the filesystem).
moonchild 6y ago
Why not keep constants constant (so your switch example works), but have an extra translation step in front of the syscalls?
qlk1123 6y ago
Fair enough and inspiring. However I am still wondering how to advocate changes in such a scale. For example, Linux introduces some fixes for Y2038 problem during the past few years. I suppose the community of this C library should do something for those changes, right?