6 ms·

I agree with some of the other comments here, that while this is a very impressive hack (and this is Hacker News!) the real world value seems dubious. It looks

I agree with some of the other comments here, that while this is a very impressive hack (and this is Hacker News!) the real world value seems dubious. It looks like a very clever way of packaging an x86 fat binary for multiple platforms, without actually duplicating the code. To support ARM I assume it’ll need to be an actual fat binary, with both x86 and ARM code. At that point, unless I actually test the code myself on both architectures, how can I be confident it’s going to work properly? If you’re using very simple C constructs and not doing anything fancy, it should work, but it’s not clear to me that this approach is preferable to e.g. a Python script. If you’re doing fancy stuff, it’s a bit more chancy as C isn’t memory-safe and has tons of undefined behavior and platform-specified weirdness. Java is “run anywhere” because they’ve specified the JVM in massive detail and tried to ensure it actually works the same on all platforms. I don’t see how you can have that same confidence if you’re running machine code everywhere. I guess I just don’t see the use case where this is compelling. If I write a handy Unix utility in C, I’ll just keep the source code around, and compile it as needed. It might be handy if you need to move such a utility quickly from Unix to Windows, if you don’t have any dev tools set up. But I can’t think of a situation when I’ve needed that.

BiteCode_dev 5y ago
It can be useful to provide a single installer that works on any system, downloading the machine specific rest of the program.
- varajelle 5y ago
  An installer without UI or network access?
  - GranPC 5y ago
    It looks like the runtime has networking support. https://github.com/jart/cosmopolitan/tree/master/libc/sock https://github.com/jart/cosmopolitan/tree/master/libc/sock
    - dingdingdang 5y ago
      It looks like the runtime has embedded local-gui-view-via-system-browser-server support. https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html - more useful info: http://redbean.justine.lol/redbean.lua?x=hi%20there%00%C0%80&y&z&z=%26 http://redbean.justine.lol/redbean.lua?x=hi%20there%00%C0%80...
      - jart 5y ago
        There's also WIN32 GUI support too. See this demo program called apelife which was built with Cosmopolitan Libc. https://justine.lol/apelife/index.html https://justine.lol/apelife/index.html It embeds a UNIX TUI that runs on both Windows and UNIX. It also embeds a WIN32 GUI too, in the same binary. There's also an example of how to build TUI panel applications in the examples/ folder. https://github.com/jart/cosmopolitan/blob/master/examples/panels.c https://github.com/jart/cosmopolitan/blob/master/examples/pa... The canonical example of a Cosmopolitan TUI would be https://justine.lol/blinkenlights/index.html https://justine.lol/blinkenlights/index.html I highly recommend writing TUIs because VT100+XTERM is universal these days and termios was easy to polyfill across operating systems. These things literally run in the CMD.EXE DOS command box on Windows 10 without needing any special #ifdef or conditions, which is wild, and it even works on metal via the serial uart port. make -j8 o//tool/viz/deathstar.com qemu-system-x86_64 -m 16 -nographic -fda o//tool/viz/deathstar.com You don't have to do anything special either. You just write your TUI program in the conventional UNIX style.
        deleted 5y ago
        [deleted]
        deleted 5y ago
        [deleted]
tstack 5y ago
I’m planning to use it in my terminal-based log file viewer (https://lnav.org https://lnav.org) to deploy an agent (over ssh) to remote machines to monitor their log files. CosmoC allows me to build a single agent binary that will work on lots of target machines and package that into the main lnav binary. I’ll still be building lnav packages for different platforms, but the agent build and deployment is vastly simpler with CosmoC. I’ll be using this feature in my day-to-day work because I interact with short-lived VMs running the software I’m developing. Being able to run lnav on my devbox and have it live-tailing logs on the VMs running the changes will be an improvement over the current workflow where I manually copy the files onto the devbox.
- sime2009 5y ago
  I'm looking at Cosmo with similar applications in mind; just-in-time delivery and execution of a small binary in a "remote" environment like a server, container, VM, or maybe embedded system.
- brabel 5y ago
  Hey! Lots of people at work love your lnav tool! Thanks for making it! Can't wait to try this feature.
1vuio0pswjnm7 5y ago
"I guess I just don't see the use case where this is compelling. If I write a handy Unix utility in C, I'll just keep the source code around, and compile it as needed." One use case is where the source and target are different operating systems, and either the target does not have a compiler installed or the source does not have a cross-compiler installed. I run into this often. I do not wish to install Python or Java on every computer. I have small form factor computers where I need to conserve space. Also, I prefer the size and speed of C over Java and Python. I think there may be a tad too much "marketing" and trying to be cute in the way this is presented but this work certainly has potential utility, whether or not it is used as intended. Even just reading through the hacks is educational. Programming does not always have to be commercially-oriented to be useful or interesting.
- iainmerrick 5y ago
  Programming does not always have to be commercially-oriented to be useful or interesting. Agreed! Thanks for the response, that does sound like an area where this could be useful. Although (speaking as somebody who doesn’t do a lot of embedded programming these days, so take this with a pinch of salt) I would have thought that if cross-compilation could be made much faster and easier, that might be a better solution overall. I really like what’s being done with Zig around cross-compilation.
- smasher164 5y ago
  > One use case is where the source and target are different architectures, and either the target does not have a compiler installed or the source does not have a cross-compiler installed. Do you mean different OSes? AFAIK, Cosmopolitan isn't cross-architecture.
  - coolreader18 5y ago
    IIRC it involves qemu on aarch64
    - smasher164 5y ago
twic 5y ago
IIRC on non-x86 platforms, it fires up QEMU!
tgamblin 5y ago
> I guess I just don’t see the use case where this is compelling. If I write a handy Unix utility in C, I’ll just keep the source code around, and compile it as needed. Isn't this exactly why you don't see the use case? You're willing to compile. As someone working on a cross-platform, cross-language packaging tool (https://github.com/spack/spack https://github.com/spack/spack), it's very appealing to not have to build for every OS and Linux distro. Currently we build binaries per-OS/per-distro. This would eliminate a couple dimensions from our combinatorial builds. We still care a lot about non-x86_64 architectures, so that's still an issue, but the work here is great for distributors of binaries. It seems like it has the potential to replace more cumbersome techniques like manylinux (https://github.com/pypa/manylinux https://github.com/pypa/manylinux).
- iainmerrick 5y ago
  Currently we build binaries per-OS/per-distro. What varies between Linux distros, for your purposes? Different libc? I would naively assume it’s all just ELF so it shouldn’t be that big a deal to make a portable Linux binary. That seems separate from swizzling ELF/Mach-O/PE, specializing the binary on first run(!), etc, which is all super cool but something I’d be wary of relying on as a solid platform. Maybe I’m being too cautious, though! Reading the Cosmopolitan docs, it has some really clever optimizations, and I think I’d be more excited about simply a small and fast libc over the flashier APE parts.
  - tgamblin 5y ago
    See https://www.python.org/dev/peps/pep-0513/#key-causes-of-inter-linux-binary-incompatibility https://www.python.org/dev/peps/pep-0513/#key-causes-of-inte.... Software built with a newer glibc is not compatible with software built with an older glibc. You can't just build an EFL binary on an arbitrary distro. Moreover, and probably more notably for cosmopolitan, binaries built with glibc aren't compatible with BSD libc or musl. IIUC, cosmopolitan works across the BSDs and musl-based Linux distros, as well.
    - jart 5y ago

BiteCode_dev 5y ago

It can be useful to provide a single installer that works on any system, downloading the machine specific rest of the program.

varajelle 5y ago
An installer without UI or network access?
- GranPC 5y ago
  It looks like the runtime has networking support. https://github.com/jart/cosmopolitan/tree/master/libc/sock https://github.com/jart/cosmopolitan/tree/master/libc/sock
  - dingdingdang 5y ago
    It looks like the runtime has embedded local-gui-view-via-system-browser-server support. https://justine.lol/redbean/index.html https://justine.lol/redbean/index.html - more useful info: http://redbean.justine.lol/redbean.lua?x=hi%20there%00%C0%80&y&z&z=%26 http://redbean.justine.lol/redbean.lua?x=hi%20there%00%C0%80...
    - jart 5y ago
      There's also WIN32 GUI support too. See this demo program called apelife which was built with Cosmopolitan Libc. https://justine.lol/apelife/index.html https://justine.lol/apelife/index.html It embeds a UNIX TUI that runs on both Windows and UNIX. It also embeds a WIN32 GUI too, in the same binary. There's also an example of how to build TUI panel applications in the examples/ folder. https://github.com/jart/cosmopolitan/blob/master/examples/panels.c https://github.com/jart/cosmopolitan/blob/master/examples/pa... The canonical example of a Cosmopolitan TUI would be https://justine.lol/blinkenlights/index.html https://justine.lol/blinkenlights/index.html I highly recommend writing TUIs because VT100+XTERM is universal these days and termios was easy to polyfill across operating systems. These things literally run in the CMD.EXE DOS command box on Windows 10 without needing any special #ifdef or conditions, which is wild, and it even works on metal via the serial uart port. make -j8 o//tool/viz/deathstar.com qemu-system-x86_64 -m 16 -nographic -fda o//tool/viz/deathstar.com You don't have to do anything special either. You just write your TUI program in the conventional UNIX style.
      - deleted 5y ago
        [deleted]
        deleted 5y ago
        [deleted]

tstack 5y ago

I’m planning to use it in my terminal-based log file viewer (https://lnav.org https://lnav.org) to deploy an agent (over ssh) to remote machines to monitor their log files. CosmoC allows me to build a single agent binary that will work on lots of target machines and package that into the main lnav binary. I’ll still be building lnav packages for different platforms, but the agent build and deployment is vastly simpler with CosmoC. I’ll be using this feature in my day-to-day work because I interact with short-lived VMs running the software I’m developing. Being able to run lnav on my devbox and have it live-tailing logs on the VMs running the changes will be an improvement over the current workflow where I manually copy the files onto the devbox.

sime2009 5y ago
I'm looking at Cosmo with similar applications in mind; just-in-time delivery and execution of a small binary in a "remote" environment like a server, container, VM, or maybe embedded system.
brabel 5y ago
Hey! Lots of people at work love your lnav tool! Thanks for making it! Can't wait to try this feature.

1vuio0pswjnm7 5y ago

"I guess I just don't see the use case where this is compelling. If I write a handy Unix utility in C, I'll just keep the source code around, and compile it as needed." One use case is where the source and target are different operating systems, and either the target does not have a compiler installed or the source does not have a cross-compiler installed. I run into this often. I do not wish to install Python or Java on every computer. I have small form factor computers where I need to conserve space. Also, I prefer the size and speed of C over Java and Python. I think there may be a tad too much "marketing" and trying to be cute in the way this is presented but this work certainly has potential utility, whether or not it is used as intended. Even just reading through the hacks is educational. Programming does not always have to be commercially-oriented to be useful or interesting.

iainmerrick 5y ago
Programming does not always have to be commercially-oriented to be useful or interesting. Agreed! Thanks for the response, that does sound like an area where this could be useful. Although (speaking as somebody who doesn’t do a lot of embedded programming these days, so take this with a pinch of salt) I would have thought that if cross-compilation could be made much faster and easier, that might be a better solution overall. I really like what’s being done with Zig around cross-compilation.
smasher164 5y ago
> One use case is where the source and target are different architectures, and either the target does not have a compiler installed or the source does not have a cross-compiler installed. Do you mean different OSes? AFAIK, Cosmopolitan isn't cross-architecture.
- coolreader18 5y ago
  IIRC it involves qemu on aarch64
  - smasher164 5y ago

twic 5y ago

IIRC on non-x86 platforms, it fires up QEMU!

tgamblin 5y ago

> I guess I just don’t see the use case where this is compelling. If I write a handy Unix utility in C, I’ll just keep the source code around, and compile it as needed. Isn't this exactly why you don't see the use case? You're willing to compile. As someone working on a cross-platform, cross-language packaging tool (https://github.com/spack/spack https://github.com/spack/spack), it's very appealing to not have to build for every OS and Linux distro. Currently we build binaries per-OS/per-distro. This would eliminate a couple dimensions from our combinatorial builds. We still care a lot about non-x86_64 architectures, so that's still an issue, but the work here is great for distributors of binaries. It seems like it has the potential to replace more cumbersome techniques like manylinux (https://github.com/pypa/manylinux https://github.com/pypa/manylinux).

iainmerrick 5y ago
Currently we build binaries per-OS/per-distro. What varies between Linux distros, for your purposes? Different libc? I would naively assume it’s all just ELF so it shouldn’t be that big a deal to make a portable Linux binary. That seems separate from swizzling ELF/Mach-O/PE, specializing the binary on first run(!), etc, which is all super cool but something I’d be wary of relying on as a solid platform. Maybe I’m being too cautious, though! Reading the Cosmopolitan docs, it has some really clever optimizations, and I think I’d be more excited about simply a small and fast libc over the flashier APE parts.
- tgamblin 5y ago
  See https://www.python.org/dev/peps/pep-0513/#key-causes-of-inter-linux-binary-incompatibility https://www.python.org/dev/peps/pep-0513/#key-causes-of-inte.... Software built with a newer glibc is not compatible with software built with an older glibc. You can't just build an EFL binary on an arbitrary distro. Moreover, and probably more notably for cosmopolitan, binaries built with glibc aren't compatible with BSD libc or musl. IIUC, cosmopolitan works across the BSDs and musl-based Linux distros, as well.
  - jart 5y ago