13 ms·

Cosmopolitan Libc 1.0

545 points by Orochikaku 5y ago

stabbles 5y ago
Is there any way to use cosmopolitan with shared libraries? In some cases licensing doesn't allow one to make a single executable.
- SXX 5y ago
  In what case licensing is a problem? For LGPL at least you can just provide object files and this will satisfy all requirements since user will be able to link any library version.
sunmountain 5y ago
Just awesome (tm).
slimsag 5y ago
Super exciting to see Cosmopolitan progress further! For those wondering what it is, from the website[0]: > Cosmopolitan makes C a build-once run-anywhere language, similar to Java, except it doesn't require interpreters or virtual machines be installed beforehand. Cosmo provides the same portability benefits as high-level languages like Go and Rust, but it doesn't invent a new language and you won't need to configure a CI system to build separate binaries for each operating system. What Cosmopolitan focuses on is fixing C by decoupling it from platforms, so it can be pleasant to use for writing small unix programs that are easily distributed to a much broader audience. [0] https://justine.lol/cosmopolitan/index.html https://justine.lol/cosmopolitan/index.html
- jart 5y ago
  Some previous discussions on HN here: - https://news.ycombinator.com/item?id=26271117 https://news.ycombinator.com/item?id=26271117 - https://news.ycombinator.com/item?id=25556286 https://news.ycombinator.com/item?id=25556286 - https://news.ycombinator.com/item?id=24256883 https://news.ycombinator.com/item?id=24256883
- KMag 5y ago
  > Cosmopolitan Libc makes C a build-once run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. ... and it only supports x86 (without binary translation), right? It's great to see progress like this, but it's poor form to suggest it's build-once run-anywhere in the same sense that Java is. As far as I can tell, it's not trivial to run these binaries on a RPi.
  - sigjuice 5y ago
    The 2-line intro says the format will run on a impressively long list of systems. Is it really that big of a deal that it doesn't specifically mention that it is x86-only? Please.
    - roca 5y ago
      The problem is, the landing page says it's "similar to Java" right in the first sentence and in the second sentence "same portability benefits as high-level languages like Go and Rust", and later on "excellent performance on modern desktops and servers" and "compared with glibc, you should expect Cosmopolitan to be almost as fast". The tricky part is, those claims are all true, they're just not all true at the same time. The portability claims are true (only if you ignore performance), the performance claims are true (only if you're on x86-64).
      - jart 5y ago
        We learned yesterday on HN that Java 1.0 was only able to run on two systems: Windows and Solaris. https://github.com/ufuu/Java-JDK10 https://github.com/ufuu/Java-JDK10 Cosmopolitan 1.0 runs on Windows, Linux, Mac, FreeBSD, OpenBSD, NetBSD, and boots from BIOS on bare metal all in a single 12kb file. https://justine.lol/cosmopolitan/howfat.html https://justine.lol/cosmopolitan/howfat.html If we consider that Java today is on version 17, then I'd ask you to imagine where Cosmopolitan v17 will be in the future.
        robertlagrant 5y ago
        The possibilities are of course endless; I think the discussion is about what is claimed today.
slimsag 5y ago
I wonder if anyone has tried using this with Zig (or Rust) yet? I think it'd be quite cool to have this as an out-of-the-box option one can just flip
- rationalfaith 5y ago
  yes, cause that language is a clear priority.
- girvo 5y ago
  It should be possible to get it working with Nim easily too, interestingly. I'll have a play with it tonight
  - planetis 5y ago
    already done: https://github.com/Yardanico/cosmonim https://github.com/Yardanico/cosmonim
    - girvo 5y ago
      Hah, of course it is! And it’s about as easy as I expected too. I got a PoC working nicely, but this looks a lot nicer than my work, with its asyncserv patch
tomcam 5y ago
Going from strength to strength in one ridiculously small package. It is thrilling to watch this runtime evolve.
tomcam 5y ago
You have to love this comment partway through the Linux/BSD deployment instructions, which read slightly more like the Borg manual than compilation docs: > Please note that your APE binary will assimilate itself as a conventional resident of your platform after the first run...
saagarjha 5y ago
Nit: XNU's version numbers number in the thousands, it's Darwin that is in the teens/early twenties. Darwin 15.6 came out in 2016, not 2018.
numlock86 5y ago
> Please note that your APE binary will assimilate itself as a conventional resident of your platform after the first run, so it can be fast and efficient for subsequent executions. Is my understanding correct that the binary changes itself when first run on the target platform? That sounds like it will trigger a lot of red lights with many automated defensive mechanisms like anti virus.
- slimsag 5y ago
  Yes, I think that's correct. However, see: https://github.com/jart/cosmopolitan/issues/90 https://github.com/jart/cosmopolitan/issues/90
  - jart 5y ago
    Author here. The people who make virus scanners are going to have their heads explode when they learn about redbean's new StoreAsset() function https://justine.lol/redbean/index.html#StoreAsset https://justine.lol/redbean/index.html#StoreAsset where the executable edits itself as though it were MongoDB. It took a pretty heroic hack to work around ETXTBSY where the executable unmaps itself from memory after loading and then remaps itself so it can restore the original APE header and modify the ZIP central directory. https://github.com/jart/cosmopolitan/blob/b8f38cf55d84bac7d735ebbe58e1d9bf1956de39/libc/runtime/openexecutable.S#L25-L189 https://github.com/jart/cosmopolitan/blob/b8f38cf55d84bac7d7... That said, I still upvote on VirusTotal each executable I publish, so you can verify they came from me, provided the executable is byte-for-byte identical to its original form. After you've modified it, it becomes yours, and you can upload your modified version to VirusTotal too and upvote that. In the future, I'd like to have smarter tools for verifying the authenticity of the non-zip bits of APE binaries, but that will come with time.
    - gravypod 5y ago
      As your repo grows do you think you'll eventually need to implement some blaze-like tooling? In previous treads you mentioned compiling the entire repo only takes a minute or two but I'm wondering if the reproducability guarantees would help an effort like this.
barosl 5y ago
> https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html > All you need to do is download the redbean.com program below, change the filename to .zip, add your content in a zip editing tool, and then change the extension back to .com. > That performance is thanks to zip and gzip using the same compression format, which enables kernelspace copies. Oh my. Having a web server executable which is a zip archive at the same time is a lovely idea. Have there been any other attempts similar to this?
- bayindirh 5y ago
  PoC||GTFO Magazine's almost every issue is multi format: https://www.alchemistowl.org/pocorgtfo/ https://www.alchemistowl.org/pocorgtfo/ From the latest issue: Technical Note:The electronic edition of this magazine is valid as both PDF and ZIP. The PDF has been cryptographically signed with a factored private key for the TI 83+ graphing calculator.
- twic 5y ago
  JAR files are ZIP archives, so any Java web server packed into a single JAR is such a thing, to an extent. I don't know of any server which actually makes use of that fact to serve compressed content, but it wouldn't be hard to do.
  - barbarbar 5y ago
    But that needs a JVM (and maintenance of that). Which is why redbean/cosmopolitan is so extremely clever.
    - jart 5y ago
      You'll never need to worry about showing your users how to uninstall the Ask Toolbar that came bundled with the Cosmopolitan Virtual Machine that needs to be installed before your users can run your APE binaries, because there isn't one. This project grants your software autonomy without any toilsome tradeoffs. Enjoy!
3v1n0 5y ago
How can this work in sandbox distributions? For example in a snap or flatpak package where a binary is in a read only mode and with limited permissions?
- lifthrasiir 5y ago
  There is a compile option named APE_NO_MODIFY_SELF which writes and launches a copy of itself into the tmpdir instead of self-modifying (with a huge caveat).
  - nonameiguess 5y ago
    Her comment here acknowledges this, but this will cause problems when /tmp is mounted noexec, which is fairly common for servers: https://github.com/jart/cosmopolitan/blob/da8a08fd58324a87f6ac673b7774512e861fe477/ape/ape.S#L549-L561 https://github.com/jart/cosmopolitan/blob/da8a08fd58324a87f6... I suppose this level of portability is more a feature if you're shipping to PCs anyway, though. If you're deploying to servers, you know the arch and OS ahead of time and there is no obvious downside I can think of to just targeting it directly.
    - mikepurvis 5y ago
      Well, or if you could do it during commissioning, like have a local Ansible step that pre-flavours the executable for the target, then pushes it over.
nabla9 5y ago
Changes... .. Add Fabrice Bellard's JavaScript engine to third party .. Add SQLite to third party
- dingdingdang 5y ago
  Along with https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html (also in cosmo distro now) this means full offline gui apps can be written to run across all major plaforms. Very impressive in my opinion :)
1vuio0pswjnm7 5y ago
Only major downside of all this amazing portability work I can see is that it seems to be owned by an online advertising services corporation (the author's employer).
- jart 5y ago
  Author here. I haven't worked for them for years. If you're concerned about ownership see https://justine.lol/cosmopolitan/license.html https://justine.lol/cosmopolitan/license.html
  - 1vuio0pswjnm7 5y ago
    When it says "Support Vector" does that just mean the version that is being used to test or does that mean, e.g., this will not work on pre-2018/2020 BSD. Is there a GNU make dependancy and if so, could it be eliminated so other makes would work as well.
- nabla9 5y ago
  Why you think so? It says in the license: Copyright 2020 Justine Alexandra Roberts Tunney ISC License (same as MIT or BSD with unnecessary text removed)
- deleted 5y ago
  [deleted]
gigatexal 5y ago
I had no idea Windows NT stood for New Technology. Also, this looks really, really interesting. This is a surely a project I will watch as until now I'd not known about it.
nicoburns 5y ago
Wouldn't it be nice if operating systems just collaborated on a shared executable format (that would just wrap the platform native ones) rather than having to hack around this with self-modifying code.
- delusional 5y ago
  The executable format is not the hard part of cross platform executable. It's pretty neat that Cosmopolitan has managed to include a cross platform loader within the executable itself, but the real magic is a cross platform standard library.
- smallstepforman 5y ago
  Well, we cannot get the various Linux distros to agree on base systems, even though they use the same kernel. There are at least 4 flavours of BSD. The irony is that poor little Haiku as a single unified system ends up having a larger community than a lot of the perceived larger, yet segregated distros.
- oblio 5y ago
  They'd never admit it explicitly, but OSes are actually fighting each other. Sometimes in a friendlier fashion, sometimes not. No way they'd agree on a common binary format.
motiejus 5y ago
I have been working with Bazel in my personal monorepo (~6 months now), and the fact that Justine, someone I believe knows Bazel/Blaze well, uses GNU Make, makes me re-think my decision: perhaps I should re-consider it. I have quite a few issues with Bazel, which require significant engineering hours to overcome, whereas I could just use gnu make and solve it quickly. Let's see if I can get some inspiration of how things can be built from this repo? `o/$(MODE)/depend`, a makefile-looking file with the full dependency tree. It is compiled using tool/build/mkdeps.c. So we have a Makefile generator (Justine, you mentioned elsewhere in this thread you didn't want to invent a build system? :)) The Makefile generator is very specific to this project: parses C files and creates that tree. It is damn fast and, so far, beautifully documented (at least the build parts I looked). Hell, even documentation lines in file preambles are 72 characters wide, and justified. Crazy. Do you manually justify those? You are also vendoring statically-built GCC, and the folder with executables is <10MB. LLVM C/C++ toolchain is hundreds of megs, compressed. I am certainly taking inspiration of being in tight control of the compiler toolchain, and beautiful documentation. Not sure I will write my Makefile generator, since my project is also not that big. Thanks. Cosmopolitan is giving me much more to look at than an αcτµαlly pδrταblε εxεcµταblε.
- oaw-bct-ar-bamf 5y ago
  I never understood how anyone can be fine with dynamic build dependencies? Each and every build tool should be version pinned so that two build runs today and 4 years ago produce the same (hex neutral) executable. Otherwise debugging issues becomes a nightmare...
  - gumby 5y ago
    I doubt many would disagree in principle but that's very expensive. You need to maintain old machines too (well, old OSes) as the old compiler may not run the same on newer versions (file locations, shared libraries, etc). Years ago I had a customer who wanted to freeze a toolset and wanted to be sure that any bug fix they requested changed only the lines of code relevant to the bug (they diffed the binaries and traced each change back to our source change to be sure). They paid an enormous premium for this capability.
YesThatTom2 5y ago
What do they mean by BIOS/UEFI support? Bare metal?
iainmerrick 5y ago
I agree with some of the other comments here, that while this is a very impressive hack (and this is Hacker News!) the real world value seems dubious. It looks like a very clever way of packaging an x86 fat binary for multiple platforms, without actually duplicating the code. To support ARM I assume it’ll need to be an actual fat binary, with both x86 and ARM code. At that point, unless I actually test the code myself on both architectures, how can I be confident it’s going to work properly? If you’re using very simple C constructs and not doing anything fancy, it should work, but it’s not clear to me that this approach is preferable to e.g. a Python script. If you’re doing fancy stuff, it’s a bit more chancy as C isn’t memory-safe and has tons of undefined behavior and platform-specified weirdness. Java is “run anywhere” because they’ve specified the JVM in massive detail and tried to ensure it actually works the same on all platforms. I don’t see how you can have that same confidence if you’re running machine code everywhere. I guess I just don’t see the use case where this is compelling. If I write a handy Unix utility in C, I’ll just keep the source code around, and compile it as needed. It might be handy if you need to move such a utility quickly from Unix to Windows, if you don’t have any dev tools set up. But I can’t think of a situation when I’ve needed that.
- BiteCode_dev 5y ago
  It can be useful to provide a single installer that works on any system, downloading the machine specific rest of the program.
  - varajelle 5y ago
    An installer without UI or network access?
    - GranPC 5y ago
      It looks like the runtime has networking support. https://github.com/jart/cosmopolitan/tree/master/libc/sock https://github.com/jart/cosmopolitan/tree/master/libc/sock
      - 5y ago
motiejus 5y ago
I am a bit worried about this: > Please note that your APE binary will assimilate itself as a conventional resident of your platform after the first run, so it can be fast and efficient for subsequent executions. I understand there may be no real way out, but this defeats part of the promise/purpose of PAE: assume I use a pae binary, know it's pae, and implicitly share it or copy it to a different machine. But once I have copied it from ~/bin/ it's not pae any more, it's optimized for my OS/arch! Assuming the first-run optimization step is necessary, would it make sense to provide `--deoptimize` or `--paeize` flag to the same binary, so it's easy to return to the original, without recompiling it from source? Does it lose information when it optimizes? Can that information be tucked away (with an optional flag or env variable) for this step? What happens if the binary is read-only?
- deleted 5y ago
  [deleted]
- Gaelan 5y ago
  Iirc, APE starts with a (Windows) PE header, which is also a valid shebang-less shell script; when invoked on Unix, the shell script replaces the beginning of the file with an ELF/Mach-O header and invokes itself.
hvis 5y ago
> GNU/Systemd That's funny.
adsharma 5y ago
I think there are a number of messages here: * Java is slowing our tools down * amalgamation sqlite style is showing the benefits of tightly written, standalone good old C. * By extension: monorepos are introducing complexity, since they depend on JVM for blaze/bazel. * x86 is pervasive in our industry People might also want to consider the cost/benefit trade-off for binary vs source compatibility. If you can code in a programming language that works across platforms, can be readily transpiled to one of the supported statically typed languages with a robust, small and fast toolchain, you have any number of packagers who can quickly make binaries for your platform of interest that makes it convenient to install. You get the benefit of better static analysis vs good old C.
deleted 5y ago
[deleted]

545 points by Orochikaku 5y ago

Is there any way to use cosmopolitan with shared libraries? In some cases licensing doesn't allow one to make a single executable.

SXX 5y ago
In what case licensing is a problem? For LGPL at least you can just provide object files and this will satisfy all requirements since user will be able to link any library version.

sunmountain 5y ago

Just awesome (tm).

slimsag 5y ago

Super exciting to see Cosmopolitan progress further! For those wondering what it is, from the website[0]: > Cosmopolitan makes C a build-once run-anywhere language, similar to Java, except it doesn't require interpreters or virtual machines be installed beforehand. Cosmo provides the same portability benefits as high-level languages like Go and Rust, but it doesn't invent a new language and you won't need to configure a CI system to build separate binaries for each operating system. What Cosmopolitan focuses on is fixing C by decoupling it from platforms, so it can be pleasant to use for writing small unix programs that are easily distributed to a much broader audience. [0] https://justine.lol/cosmopolitan/index.html https://justine.lol/cosmopolitan/index.html

jart 5y ago
Some previous discussions on HN here: - https://news.ycombinator.com/item?id=26271117 https://news.ycombinator.com/item?id=26271117 - https://news.ycombinator.com/item?id=25556286 https://news.ycombinator.com/item?id=25556286 - https://news.ycombinator.com/item?id=24256883 https://news.ycombinator.com/item?id=24256883
KMag 5y ago
> Cosmopolitan Libc makes C a build-once run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. ... and it only supports x86 (without binary translation), right? It's great to see progress like this, but it's poor form to suggest it's build-once run-anywhere in the same sense that Java is. As far as I can tell, it's not trivial to run these binaries on a RPi.
- sigjuice 5y ago
  The 2-line intro says the format will run on a impressively long list of systems. Is it really that big of a deal that it doesn't specifically mention that it is x86-only? Please.
  - roca 5y ago
    The problem is, the landing page says it's "similar to Java" right in the first sentence and in the second sentence "same portability benefits as high-level languages like Go and Rust", and later on "excellent performance on modern desktops and servers" and "compared with glibc, you should expect Cosmopolitan to be almost as fast". The tricky part is, those claims are all true, they're just not all true at the same time. The portability claims are true (only if you ignore performance), the performance claims are true (only if you're on x86-64).
    - jart 5y ago
      We learned yesterday on HN that Java 1.0 was only able to run on two systems: Windows and Solaris. https://github.com/ufuu/Java-JDK10 https://github.com/ufuu/Java-JDK10 Cosmopolitan 1.0 runs on Windows, Linux, Mac, FreeBSD, OpenBSD, NetBSD, and boots from BIOS on bare metal all in a single 12kb file. https://justine.lol/cosmopolitan/howfat.html https://justine.lol/cosmopolitan/howfat.html If we consider that Java today is on version 17, then I'd ask you to imagine where Cosmopolitan v17 will be in the future.
      - robertlagrant 5y ago
        The possibilities are of course endless; I think the discussion is about what is claimed today.

slimsag 5y ago

I wonder if anyone has tried using this with Zig (or Rust) yet? I think it'd be quite cool to have this as an out-of-the-box option one can just flip

rationalfaith 5y ago
yes, cause that language is a clear priority.
girvo 5y ago
It should be possible to get it working with Nim easily too, interestingly. I'll have a play with it tonight
- planetis 5y ago
  already done: https://github.com/Yardanico/cosmonim https://github.com/Yardanico/cosmonim
  - girvo 5y ago
    Hah, of course it is! And it’s about as easy as I expected too. I got a PoC working nicely, but this looks a lot nicer than my work, with its asyncserv patch

tomcam 5y ago

Going from strength to strength in one ridiculously small package. It is thrilling to watch this runtime evolve.

tomcam 5y ago

You have to love this comment partway through the Linux/BSD deployment instructions, which read slightly more like the Borg manual than compilation docs: > Please note that your APE binary will assimilate itself as a conventional resident of your platform after the first run...

saagarjha 5y ago

Nit: XNU's version numbers number in the thousands, it's Darwin that is in the teens/early twenties. Darwin 15.6 came out in 2016, not 2018.

numlock86 5y ago

> Please note that your APE binary will assimilate itself as a conventional resident of your platform after the first run, so it can be fast and efficient for subsequent executions. Is my understanding correct that the binary changes itself when first run on the target platform? That sounds like it will trigger a lot of red lights with many automated defensive mechanisms like anti virus.

slimsag 5y ago
Yes, I think that's correct. However, see: https://github.com/jart/cosmopolitan/issues/90 https://github.com/jart/cosmopolitan/issues/90
- jart 5y ago
  Author here. The people who make virus scanners are going to have their heads explode when they learn about redbean's new StoreAsset() function https://justine.lol/redbean/index.html#StoreAsset https://justine.lol/redbean/index.html#StoreAsset where the executable edits itself as though it were MongoDB. It took a pretty heroic hack to work around ETXTBSY where the executable unmaps itself from memory after loading and then remaps itself so it can restore the original APE header and modify the ZIP central directory. https://github.com/jart/cosmopolitan/blob/b8f38cf55d84bac7d735ebbe58e1d9bf1956de39/libc/runtime/openexecutable.S#L25-L189 https://github.com/jart/cosmopolitan/blob/b8f38cf55d84bac7d7... That said, I still upvote on VirusTotal each executable I publish, so you can verify they came from me, provided the executable is byte-for-byte identical to its original form. After you've modified it, it becomes yours, and you can upload your modified version to VirusTotal too and upvote that. In the future, I'd like to have smarter tools for verifying the authenticity of the non-zip bits of APE binaries, but that will come with time.
  - gravypod 5y ago
    As your repo grows do you think you'll eventually need to implement some blaze-like tooling? In previous treads you mentioned compiling the entire repo only takes a minute or two but I'm wondering if the reproducability guarantees would help an effort like this.

barosl 5y ago

> https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html > All you need to do is download the redbean.com program below, change the filename to .zip, add your content in a zip editing tool, and then change the extension back to .com. > That performance is thanks to zip and gzip using the same compression format, which enables kernelspace copies. Oh my. Having a web server executable which is a zip archive at the same time is a lovely idea. Have there been any other attempts similar to this?

bayindirh 5y ago
PoC||GTFO Magazine's almost every issue is multi format: https://www.alchemistowl.org/pocorgtfo/ https://www.alchemistowl.org/pocorgtfo/ From the latest issue: Technical Note:The electronic edition of this magazine is valid as both PDF and ZIP. The PDF has been cryptographically signed with a factored private key for the TI 83+ graphing calculator.
twic 5y ago
JAR files are ZIP archives, so any Java web server packed into a single JAR is such a thing, to an extent. I don't know of any server which actually makes use of that fact to serve compressed content, but it wouldn't be hard to do.
- barbarbar 5y ago
  But that needs a JVM (and maintenance of that). Which is why redbean/cosmopolitan is so extremely clever.
  - jart 5y ago
    You'll never need to worry about showing your users how to uninstall the Ask Toolbar that came bundled with the Cosmopolitan Virtual Machine that needs to be installed before your users can run your APE binaries, because there isn't one. This project grants your software autonomy without any toilsome tradeoffs. Enjoy!

3v1n0 5y ago

How can this work in sandbox distributions? For example in a snap or flatpak package where a binary is in a read only mode and with limited permissions?

lifthrasiir 5y ago
There is a compile option named APE_NO_MODIFY_SELF which writes and launches a copy of itself into the tmpdir instead of self-modifying (with a huge caveat).
- nonameiguess 5y ago
  Her comment here acknowledges this, but this will cause problems when /tmp is mounted noexec, which is fairly common for servers: https://github.com/jart/cosmopolitan/blob/da8a08fd58324a87f6ac673b7774512e861fe477/ape/ape.S#L549-L561 https://github.com/jart/cosmopolitan/blob/da8a08fd58324a87f6... I suppose this level of portability is more a feature if you're shipping to PCs anyway, though. If you're deploying to servers, you know the arch and OS ahead of time and there is no obvious downside I can think of to just targeting it directly.
  - mikepurvis 5y ago
    Well, or if you could do it during commissioning, like have a local Ansible step that pre-flavours the executable for the target, then pushes it over.

nabla9 5y ago

Changes... .. Add Fabrice Bellard's JavaScript engine to third party .. Add SQLite to third party

dingdingdang 5y ago
Along with https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html (also in cosmo distro now) this means full offline gui apps can be written to run across all major plaforms. Very impressive in my opinion :)

1vuio0pswjnm7 5y ago

Only major downside of all this amazing portability work I can see is that it seems to be owned by an online advertising services corporation (the author's employer).

jart 5y ago
Author here. I haven't worked for them for years. If you're concerned about ownership see https://justine.lol/cosmopolitan/license.html https://justine.lol/cosmopolitan/license.html
- 1vuio0pswjnm7 5y ago
  When it says "Support Vector" does that just mean the version that is being used to test or does that mean, e.g., this will not work on pre-2018/2020 BSD. Is there a GNU make dependancy and if so, could it be eliminated so other makes would work as well.
nabla9 5y ago
Why you think so? It says in the license: Copyright 2020 Justine Alexandra Roberts Tunney ISC License (same as MIT or BSD with unnecessary text removed)
deleted 5y ago
[deleted]

gigatexal 5y ago

I had no idea Windows NT stood for New Technology. Also, this looks really, really interesting. This is a surely a project I will watch as until now I'd not known about it.

nicoburns 5y ago

Wouldn't it be nice if operating systems just collaborated on a shared executable format (that would just wrap the platform native ones) rather than having to hack around this with self-modifying code.

delusional 5y ago
The executable format is not the hard part of cross platform executable. It's pretty neat that Cosmopolitan has managed to include a cross platform loader within the executable itself, but the real magic is a cross platform standard library.
smallstepforman 5y ago
Well, we cannot get the various Linux distros to agree on base systems, even though they use the same kernel. There are at least 4 flavours of BSD. The irony is that poor little Haiku as a single unified system ends up having a larger community than a lot of the perceived larger, yet segregated distros.
oblio 5y ago
They'd never admit it explicitly, but OSes are actually fighting each other. Sometimes in a friendlier fashion, sometimes not. No way they'd agree on a common binary format.

motiejus 5y ago

I have been working with Bazel in my personal monorepo (~6 months now), and the fact that Justine, someone I believe knows Bazel/Blaze well, uses GNU Make, makes me re-think my decision: perhaps I should re-consider it. I have quite a few issues with Bazel, which require significant engineering hours to overcome, whereas I could just use gnu make and solve it quickly. Let's see if I can get some inspiration of how things can be built from this repo? `o/$(MODE)/depend`, a makefile-looking file with the full dependency tree. It is compiled using tool/build/mkdeps.c. So we have a Makefile generator (Justine, you mentioned elsewhere in this thread you didn't want to invent a build system? :)) The Makefile generator is very specific to this project: parses C files and creates that tree. It is damn fast and, so far, beautifully documented (at least the build parts I looked). Hell, even documentation lines in file preambles are 72 characters wide, and justified. Crazy. Do you manually justify those? You are also vendoring statically-built GCC, and the folder with executables is <10MB. LLVM C/C++ toolchain is hundreds of megs, compressed. I am certainly taking inspiration of being in tight control of the compiler toolchain, and beautiful documentation. Not sure I will write my Makefile generator, since my project is also not that big. Thanks. Cosmopolitan is giving me much more to look at than an αcτµαlly pδrταblε εxεcµταblε.

oaw-bct-ar-bamf 5y ago
I never understood how anyone can be fine with dynamic build dependencies? Each and every build tool should be version pinned so that two build runs today and 4 years ago produce the same (hex neutral) executable. Otherwise debugging issues becomes a nightmare...
- gumby 5y ago
  I doubt many would disagree in principle but that's very expensive. You need to maintain old machines too (well, old OSes) as the old compiler may not run the same on newer versions (file locations, shared libraries, etc). Years ago I had a customer who wanted to freeze a toolset and wanted to be sure that any bug fix they requested changed only the lines of code relevant to the bug (they diffed the binaries and traced each change back to our source change to be sure). They paid an enormous premium for this capability.

YesThatTom2 5y ago

What do they mean by BIOS/UEFI support? Bare metal?

iainmerrick 5y ago

I agree with some of the other comments here, that while this is a very impressive hack (and this is Hacker News!) the real world value seems dubious. It looks like a very clever way of packaging an x86 fat binary for multiple platforms, without actually duplicating the code. To support ARM I assume it’ll need to be an actual fat binary, with both x86 and ARM code. At that point, unless I actually test the code myself on both architectures, how can I be confident it’s going to work properly? If you’re using very simple C constructs and not doing anything fancy, it should work, but it’s not clear to me that this approach is preferable to e.g. a Python script. If you’re doing fancy stuff, it’s a bit more chancy as C isn’t memory-safe and has tons of undefined behavior and platform-specified weirdness. Java is “run anywhere” because they’ve specified the JVM in massive detail and tried to ensure it actually works the same on all platforms. I don’t see how you can have that same confidence if you’re running machine code everywhere. I guess I just don’t see the use case where this is compelling. If I write a handy Unix utility in C, I’ll just keep the source code around, and compile it as needed. It might be handy if you need to move such a utility quickly from Unix to Windows, if you don’t have any dev tools set up. But I can’t think of a situation when I’ve needed that.

BiteCode_dev 5y ago
It can be useful to provide a single installer that works on any system, downloading the machine specific rest of the program.
- varajelle 5y ago
  An installer without UI or network access?
  - GranPC 5y ago
    It looks like the runtime has networking support. https://github.com/jart/cosmopolitan/tree/master/libc/sock https://github.com/jart/cosmopolitan/tree/master/libc/sock
    - 5y ago

motiejus 5y ago

I am a bit worried about this: > Please note that your APE binary will assimilate itself as a conventional resident of your platform after the first run, so it can be fast and efficient for subsequent executions. I understand there may be no real way out, but this defeats part of the promise/purpose of PAE: assume I use a pae binary, know it's pae, and implicitly share it or copy it to a different machine. But once I have copied it from ~/bin/ it's not pae any more, it's optimized for my OS/arch! Assuming the first-run optimization step is necessary, would it make sense to provide `--deoptimize` or `--paeize` flag to the same binary, so it's easy to return to the original, without recompiling it from source? Does it lose information when it optimizes? Can that information be tucked away (with an optional flag or env variable) for this step? What happens if the binary is read-only?

deleted 5y ago
[deleted]
Gaelan 5y ago
Iirc, APE starts with a (Windows) PE header, which is also a valid shebang-less shell script; when invoked on Unix, the shell script replaces the beginning of the file with an ELF/Mach-O header and invokes itself.

hvis 5y ago

> GNU/Systemd That's funny.

adsharma 5y ago

I think there are a number of messages here: * Java is slowing our tools down * amalgamation sqlite style is showing the benefits of tightly written, standalone good old C. * By extension: monorepos are introducing complexity, since they depend on JVM for blaze/bazel. * x86 is pervasive in our industry People might also want to consider the cost/benefit trade-off for binary vs source compatibility. If you can code in a programming language that works across platforms, can be readily transpiled to one of the supported statically typed languages with a robust, small and fast toolchain, you have any number of packagers who can quickly make binaries for your platform of interest that makes it convenient to install. You get the benefit of better static analysis vs good old C.

deleted 5y ago

[deleted]