11 ms·

Cosmopolitan v3.5

366 points by sgammon 2y ago

WantonQuantum 2y ago
I assume the executables won't run on ARM platforms? So modern Macs are out, etc.
- nabla9 2y ago
  It works on AMD64 and ARM64 both. It makes two architecture fat binary, but removes all code not needed. Resulting binary is typically only 30-50% larger than 1-architecture binary.
  - mkl 2y ago
    More details of supported platforms at https://justine.lol/cosmo3/ https://justine.lol/cosmo3/ and https://github.com/jart/cosmopolitan/blob/master/tool/cosmocc/README.md https://github.com/jart/cosmopolitan/blob/master/tool/cosmoc..., both linked in the release notes.
- lukemerrick 2y ago
  Looks like there is both an ARM and x86 version according to the docs. Probably need two different binaries, but you still get cross-OS for each architecture.
  - eesmith 2y ago
    It's a single binary, cross-OS, cross-arch. Go to https://cosmo.zip/pub/cosmos/bin/ https://cosmo.zip/pub/cosmos/bin/ and download an executable, like https://cosmo.zip/pub/cosmos/bin/basename https://cosmo.zip/pub/cosmos/bin/basename . % curl -O https://cosmo.zip/pub/cosmos/bin/basename % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 663k 100 663k 0 0 440k 0 0:00:01 0:00:01 --:--:-- 441k % chmod +x basename On macOS with an M1: % arch arm64 % sha256sum basename 2e4cf8378b679dd0c2cdcc043bb1f21eaf464d3c3b636ed101fbc5e3230eb5fb basename % file ./basename ./basename: DOS/MBR boot sector; partition 1 : ID=0x7f, active, start-CHS (0x0,0,1), end-CHS (0x3ff,255,63), startsector 0, 4294967295 sectors % ./basename /dev/null.txt null.txt On FreeBSD with an amd64: % uname -sm FreeBSD amd64 % sha256sum basename 2e4cf8378b679dd0c2cdcc043bb1f21eaf464d3c3b636ed101fbc5e3230eb5fb basename % file basename basename: DOS/MBR boot sector; partition 1 : ID=0x7f, active, start-CHS (0x0,0,1), end-CHS (0x3ff,255,63), startsector 0, 4294967295 sectors % ./basename /dev/null.pdf null.pdf
    - alganet 2y ago
      Kind of. It can be read as a single binary, some supported systems will do that. In others, the executable is first parsed as a shell script. There's definitely more to it than a single binary. MZqFpD=' ø @ ' <<'justine0e5c7z' È ²@ë ëëHƒì1Ò½ ëéE ü‡>à¿ p1ÉŽÁúŽ×‰Ìûè ^îr ¸ PP 1ÿ¹ ó¤‡ÒÿêŽ ŽÙ¹ ¸P ŽÀ1À1ÿóª€ú@tè °1É0ö¿Pèo ŒÆƒÆ ŽÆOuóê ' SR´Ís1ÀÍrF¸¹ ¶ » ŽÃ1ÛÍr3´Ír-ˆÏ€ç?€áÀÐÁÐÁ†Í1öŽÆ¾‡÷¥¥¥¥¥¤“«®‘«’«Xª’[ÃZ€ò€1ÀÍr÷ë¡PQ†ÍÐÉÐÉÁ1Û°´ÍYXrþÀ:$v °þÆ:6)v0öAÃP1ÀÍXëÌ‰þ¬„Àt » ´ÍëòÃW¿¹&èèÿ_èäÿ¿Á&èÞÿóëü¹ ¾ …Àt QV—¾Ð&è ^YâîÃ‰ú…ÒtRV1É±Ê¬^ €îZ¬îBIyúÃ € ÿÿÿ ÿÿÿÿ Uª justine0e5c7z #'" o=$(command -v "$0") [ x"$1" != x--assimilate ] && type ape >/dev/null 2>&1 && exec ape "$o" "$@" t="${TMPDIR:-${HOME:-.}}/.ape-1.10" [ x"$1" != x--assimilate ] && [ -x "$t" ] && exec "$t" "$o" "$@" m=$(uname -m 2>/dev/null) || m=x86_64 if [ ! -d /Applications ]; then if [ x"$1" = x--assimilate ]; then if [ "$m" = x86_64 ] || [ "$m" = amd64 ]; then exec 7<> "$o" || exit 121 printf '\177ELF\2\1\1\11\0\0\0\0\0\0\0\0\2\0>\0\1\0\0\0vE@\0\0\0\0\0\510\013\000\000\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\100\0\70\0\006\000\0\0\0\0\0\0' >&7 exec 7<&- fi exit
      - bux93 2y ago
        Reminds me of a cool tool I once used, uudecode.com, which was a DOS binary that only used 7-bit characters and could decode uuencoded (base64 predecessor) files. Was useful for getting attachments through e-mail in the face of all kinds of filters.
        eesmith 2y ago
        https://news.ycombinator.com/item?id=16312317 https://news.ycombinator.com/item?id=16312317 - "A C89 compiler that produces executables that are also valid ASCII text files" http://tom7.org/abc/ http://tom7.org/abc/ The paper for SIGBOVIK 2017 is both a text file and an executable.
        metadat 2y ago
        Really near. Also discussed 11 months ago: https://news.ycombinator.com/item?id=36821985 https://news.ycombinator.com/item?id=36821985 (15 comments)
- d4rti 2y ago
  Works for sure: $ file ./bin/python ./bin/python: DOS/MBR boot sector; partition 1 : ID=0x7f, active, start-CHS (0x0,0,1), end-CHS (0x3ff,255,63), startsector 0, 4294967295 sectors $ ./bin/python Python 3.11.4 (heads/pypack1:6eea485, Jan 24 2024, 10:14:24) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.processor() 'arm'
  - 4gotunameagain 2y ago
    Out of curiosity, why are you running python packed with cosmopolitan ?
    - simonw 2y ago
      Presumably to demonstrate that it works on Mac ARM platforms, in response to the question about that.
      - 4gotunameagain 2y ago
        If was satisfied with my own presumption I wouldn't have asked ;)
        simonw 2y ago
        That Python app is a popular demo for Cosmopolitan. It's what I would have chosen for that demo, too! It's handy because it outputs a little bit of information about the current architecture on the first line when you start the shell. There's some interesting background on it in this issue: https://github.com/jart/cosmopolitan/issues/141 https://github.com/jart/cosmopolitan/issues/141
  - smcameron 2y ago
    I forget how I did this, but: $ file hello.com hello.com: αcτµαlly pδrταblε εxεcµταblε
- Aissen 2y ago
  It depends. It does not work on CentOS 8 / Rocky 8 for example: https://github.com/jart/cosmopolitan/issues/1151 https://github.com/jart/cosmopolitan/issues/1151
  - jart 2y ago
    Is that an embedded device with a small address space? APE used to require a 47bit address space. The release I published today fixes that. Maybe give it another try?
    - Aissen 2y ago
      No, it's a server with 256GB of RAM, but indeed the kernel config has CONFIG_ARM64_VA_BITS_48=y (and 52 for userspace). I have since moved away from this kernel, I'll see if I find the time to boot it again to test the latest memory map manager. Edit: I tried booting it, the issue is still present, I updated the github issue.
speedgoose 2y ago
The repository also contains the source code for redbean, which is a neat web server with lua and plenty of features. https://redbean.dev/ https://redbean.dev/ https://github.com/jart/cosmopolitan/tree/master/tool/net https://github.com/jart/cosmopolitan/tree/master/tool/net
- sgt_bilko 2y ago
  The Lua fullmoon web framework is built around redbean. https://github.com/pkulchenko/fullmoon https://github.com/pkulchenko/fullmoon
  - morbicer 2y ago
    Wow this is awesome, thanks for the tip
- ncruces 2y ago
  Having written a portable SQLite VFS from scratch, I'm a bit scared of how they blindly translate fcntl locks into LockFileEx in a forking web server. The opportunity for data corruption just freaks me out.
  - deleted 2y ago
    [deleted]
m1keil 2y ago
Cosmo looks so cool. Are there any big projects out there that use it for distribution?
- PhilippGille 2y ago
  Llamafile: https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile
mkl 2y ago
Related re Windows on ARM https://news.ycombinator.com/item?id=40772475 https://news.ycombinator.com/item?id=40772475.
trustno2 2y ago
I'm always somewhat scared - not sure if rationally or irrationally - about cosmopolitan. It's a cool hack but I somehow feel like it should not work.
- hiAndrewQuinn 2y ago
  I'm somewhat of the same mind, but I'm fairly sure a study of Operating Systems: Three Easy Steps would get me over the hump. There's no actual reason to suspect this thing would like, sidestep normal process management or memory virtualization or something and run amok... I think.
- alganet 2y ago
  It should work. We should have portable executables across many operating systems and architectures. The other expectation is scary.
  - cpach 2y ago
    There’s also Java, Python, Ruby, NodeJS etc etc.
  - 3836293648 2y ago
    Working across unices is reasonable. Adding Windows is less so
    - cpach 2y ago
      It’s quite rare to ship multi-Unix binaries, isn’t it? Cosmopolitan is a very cool hack indeed, but IMHO it’s not likely to gain widespread usage.
    - flohofwoe 2y ago
      A thin POSIX layer and ELF loader shouldn't be too much of a problem for Microsoft to implement if they wanted to do so (WinNT actually did have a POSIX personality at some point, but I don't think that's still supported). I'd also like to see a builtin WASM runtime in all operating systems.
auggierose 2y ago
Is there a UI library that goes with that?
- DrNosferatu 2y ago
  A Web Interface for Cosmopolitan Apps would be a great bridge for an UI.
  - auggierose 2y ago
    I am looking for an Electron replacement for writing my apps. Could cosmopolitan be the basis of something like that, or is that not really its purpose?
    - brabel 2y ago
      Lua is perfect for Redbean because it's tiny and made for embedding, which is this exactly use case. Something like TypeScript would require a transpiler to JS and a JS runtime, which as far as I know, even the smallest one would be much bigger than Lua. And Lua has well defined semantics to call C and be called by C, which may be hard to do with JS/TS.
      - DrNosferatu 2y ago
        Any examples and good tutorials on how to best go about it?
        brabel 2y ago
        Best go about using Redbean? The project is well documented: https://redbean.dev/ https://redbean.dev/ Just go through it, it answers all questions you may come up with IMO. Or you mean about embedding Lua? The official docs are great too: https://www.lua.org/pil/24.html https://www.lua.org/pil/24.html If you prefer a tutorial, there's plenty as Lua has been around for a long time. Search on Youtube for example and there's lots of videos. The top hit on DDG for me was https://lucasklassmann.com/blog/2019-02-02-embedding-lua-in-c/ https://lucasklassmann.com/blog/2019-02-02-embedding-lua-in-... which seems pretty straightforward to follow.
pixelpoet 2y ago
I checked two commits and they both had exactly the same number of chars per line, e.g. https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3df6b1f4a426f8ee89d3d20ac https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3... I enjoy doing this too sometimes and don't find it too difficult, but damn...
- whenlambo 2y ago
  https://www.youtube.com/watch?v=Y65FRxE7uMc https://www.youtube.com/watch?v=Y65FRxE7uMc
  - lioeters 2y ago
    I have found my people.
  - cgh 2y ago
    Amazing, thank you for this.
  - pixelpoet 2y ago
    I thought I was alone!! wipes tears of joy
  - bdjsiqoocwk 2y ago
    Well. Lorem Ipsum vs Donald Knuth. I think they both suck. I have a different hero. Linus Torvalds. He IS good AND actually delivers. I enjoyed the video. However, both the author and Donald Keith strike as two people who are mainly motivated by aesthetics (which is supremely ironic as they almost define themselves as being the opposite of Loren Ipsum). I like people who deliver results. I think the author mostly likes the appearance of someone who pays attention to detail. He says that in a movie if it's obvious that the coffee cups are empty, he notices and bothers him. 3 minutes later in his video he has a guy using his laptop at the beach where glare would make it impossible to see anything on the screen.
- lproven 2y ago
  Back in the pre-GUI era, I did this with all my posts and comments online. All fully-justified, both margins, without any extra whitespace: just careful choice of words, and occasional (but always normal standard) abbreviations. It was fun, once practised it barely slowed me down, and when people noticed it blew their minds. Then along came GUIs and proportional fonts and made it all invisible. :'(
Nevin1901 2y ago
The "actually portable executable header", fun to read through: https://raw.githubusercontent.com/jart/cosmopolitan/1.0/ape/ape.S https://raw.githubusercontent.com/jart/cosmopolitan/1.0/ape/...
- dirkc 2y ago
  I just scrolled for the images, I don't know about reading assembly for fun :)
j16sdiz 2y ago
in release note: > ... , it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format ... Did POSIX really _approve_ this? if yes, when?
- xiaq 2y ago
  Look for “POSIX” on https://justine.lol/cosmo3/ https://justine.lol/cosmo3/
  - oguz-ismail 2y ago
    POSIX now says: >The input file can be of any type, but the initial portion of the file intended to be parsed according to the shell grammar [...] shall not contain the NUL character. But the initial portion of, https://cosmo.zip/pub/cosmos/bin/cat https://cosmo.zip/pub/cosmos/bin/cat for example, does contain the NUL character.
    - jart 2y ago
      Cosmo author here. I'm familiar with the new rules. They were first implemented in https://github.com/freebsd/freebsd-src/commit/e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a https://github.com/freebsd/freebsd-src/commit/e0f5c1387df23c... which makes a clear intentional allowance for what APE is doing. If you look at the hex dump: main jart@luna:~/llamafile$ hexdump -C cat | head 00000000 4d 5a 71 46 70 44 3d 27 0a 0a 00 10 00 f8 00 00 |MZqFpD='........| 00000010 00 00 00 00 00 01 00 08 40 00 00 00 00 00 00 00 |........@.......| You'll notice there's no NUL characters on the first line, and that the subsequent NULs are escaped by a single-quoted string, which is legal. The rules used to be more restrictive but they relaxed the requirements specifically so I could work on APE. Jilles Tjoelker is one of the heroes who made that possible.
      - oguz-ismail 2y ago
        [T]he initial portion doesn't mean the first line, it means the script part of a file consisting of a shell script and a binary payload, separated by `exit', `exec whatever', etc. A good example is the Oracle Developer Studio installation script. You can write to Austin Group mailing list and ask for clarification if you want.
zbendefy 2y ago
wouldn't targeting webassembly provide something similiar? how would this compare to targeting WASM and running a runtime like wasmer?
- optymizer 2y ago
  WebAssembly is bytecode that runs on a virtual machine. Cosmopolitan is a libc replacement. Your program is still compiled to real CPU instructions, which execute much faster.
tiffanyh 2y ago
I assumed OpenBSD introducing pinning syscalls would prevent Cosmo from working. I guess not. https://news.ycombinator.com/item?id=38579913 https://news.ycombinator.com/item?id=38579913
geokon 2y ago
It'd be fun/great if someone made Graal Native work with it. Then things could come full circle and I could compile my Clojure/Java binaries to run everywhere (but now without a VM)
- sgammon 2y ago
  work is ongoing
  - geokon 2y ago
    I'm quite fuzzy on the technical details - but do you think it'd ever lead to JavaFX running everywhere? It was a while before JFX worked with Graal Native I'd like to someday be able to easily distribute my little GUI apps (for scientific applications) in a compile-once kind of way. Asking users to install a JVM runtime has turned out to be unworkable (everyone installs the Java 8 runtime b/c that's what comes up first on Google and then the apps silently crash/don't-work and users are confused)
    - throwaway2037 2y ago
      > everyone installs the Java 8 runtime b/c that's what comes up first on Google and then the apps silently crash/don't-work and users are confused Is it not possible to add a tiny check from main()? If version is less than required, then show a (dreaded) JMessageBox or print message to STDOUT/STDERR where users can safely download the correct JDK.
      - geokon 2y ago
        No idea. I'm guessing the process doesn't even launch (I don't remember if I checked this..). Do you have an example with a check ? I'd love to add it. You'd probably need a very basic Swing error popup
      - sgammon 2y ago
        No, it’s not possible, because bytecode compiled at a higher level cannot be run by JDK8. It can’t even parse the class file. Most projects are compiling at JDK11 or JDK17 now at least, if not JDK21, which is the latest LTS. the latest release is JDK22. so if you’re running 8, you won’t be able to run much modern Java at all.
superkuh 2y ago
Any idea if there will ever be a fix for the majority of linux distributions that cannot run cosmopolitan C applications without the bin fmt fix?
- deleted 2y ago
  [deleted]
meisel 2y ago
Although this project is undoubtedly very cool, and maybe simplifies build processes by having a single binary, is there any other reason to use it? How does it compare in terms of performance, static linkability, standards conformance, etc. with musl and glibc? I’m curious because I’m picking a libc at the moment for my project.
- doctorpangloss 2y ago
  > How does it compare in terms of performance I tried rsync for Windows from its "cosmos" pile of binaries (https://github.com/jart/cosmopolitan/releases/download/3.3.1/cosmos-3.3.1.zip https://github.com/jart/cosmopolitan/releases/download/3.3.1...), and it was too slow to be usable compared to WSL rsync. For many other utilities I use Busybox for Windows (https://github.com/rmyorston/busybox-w32 https://github.com/rmyorston/busybox-w32) which is well maintained, fast, and its maintainer is also very responsive to bug reports.
  - jart 2y ago
    You're basically just benchmarking the WIN32 filesystem versus a Linux VM in that case. The Windows file system is famously slow. It'd make more sense to compare a Cosmo binary running on WIN32 with a WIN32 native program. For example, Emacs has a WIN32 port where they wrote all the Windows polyfills on their own, and because they aren't experts on Windows, the official GNU Emacs Windows releases go a lot slower on Windows than if you just compile Emacs from source on Linux using Cosmo and scp the the binary over to run on Windows. See https://justine.lol/cosmo3/ https://justine.lol/cosmo3/
- tracker1 2y ago
  I don't know enough to compare to musl, but I suspect being ISC licensed (similar to musl's MIT license) allows for static linking without upstream impact such as GPL. Would be curious regarding features and performance comparisons with musl, which seems to come up a little short compared to say gnu libc.
  - saghm 2y ago
stephc_int13 2y ago
Is there a plan to distribute the cosmopolitan libc without all the APE stuff? For now it seems to be more like a framework+toolchain than a standalone library.
- deleted 2y ago
  [deleted]
aidenn0 2y ago
I've been following it for almost 4 years now and I still do not know how serious of a project Cosmopolitain/αpε is.
- TimTheTinker 2y ago
  The author is certainly very serious about writing highly distributable production software that works and will continue to work far into the future. It's already being used for serious things - Mozilla's llamafile and redbean are two examples that come to mind.
tarruda 2y ago
I wonder if it is possible to have a 100% portable qemu build with this. Would be funny to have a usb stick with qemu + VM that runs on any OS.
- jart 2y ago
  blink is almost as good as qemu-user and it works great compiled with cosmocc. https://github.com/jart/blink https://github.com/jart/blink
modeless 2y ago
The commit message for the first commit there says "Actually Portable Executable now supports Android". I assume this means that the bare executables can run on Android kernels, not that there's any support for installing APEs as Android apps. But it seems possible for that to eventually work! Is that a goal of the project?
fusslo 2y ago
Never heard of Cosmopolitan before, but can I just say I really appreciate that the FIRST paragraph is a description of WHAT Cosmopolitan is and does. There's way too many readme's that just jump in and assume the reader already is familiar with the project. it's a pet peeve of mine
- deleted 2y ago
  [deleted]
deleted 2y ago
[deleted]
hahaitsfunny 2y ago
[dead]
kortex 2y ago
Looking forward toward somebody hooking together Python in APE [0], something like pex [1]/shiv[2]/pyinstaller[3], and the pants build system [4] to have a toolchain which spits out single-file python executables with baked-in venv and portable across mainstream OSes with native (or close enough) performance. 0 - https://news.ycombinator.com/item?id=40040342 https://news.ycombinator.com/item?id=40040342 1 - https://docs.pex-tool.org/ https://docs.pex-tool.org/ 2 - https://shiv.readthedocs.io/en/latest/ https://shiv.readthedocs.io/en/latest/ 3 - https://pyinstaller.org/en/stable/ https://pyinstaller.org/en/stable/ 4 - https://www.pantsbuild.org/ https://www.pantsbuild.org/
- WaxProlix 2y ago
  You can probably generate C code from Python now with Nuitka and pump that into this Cosmopolitan tool, today, to get that? https://nuitka.net/ https://nuitka.net/
sangnoir 2y ago
I gave up trying to run cgo-compiled Go on an old appliance[1] the version of libc was simply too old. Has anyone ever successfully built cosmopolitan-flavored cgo binaries? I see Cosmopolitan libc supports Linux 2.6, so I'm hopeful. 1. Linux 2.x on little-endian MIPS.
throwaway2037 2y ago
From here: https://github.com/jart/cosmopolitan/releases/tag/3.5.0 https://github.com/jart/cosmopolitan/releases/tag/3.5.0 Justine wrote: > We've made a lot of progress reinventing the C++ STL. What is the motivation behind this? Try to reduce (compiled) binary size? Naively, I am surprised that the C++ STL from Clang isn't OK to use, including the license. Or is this a clean room impl thing? EDIT To be clear: I mean no disrespect with this question. This is an amazing project. EDIT 2 Ok, I just spotted this commit message: https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3df6b1f4a426f8ee89d3d20ac https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3... > We now have a C++ red-black tree implementation that implements standard template library compatible APIs while compiling 10x faster than libcxx. Can someone explain to an average C++ programmer like me how they make it compile 10x faster !?
- simjnd 2y ago
  The rationale is explained in this README [1], but it basically boils down to transitive dependencies and the amount of checks the compiler has to do. From linked README: "If we `#include <string>` for the LLVM libcxx header containing the `std::string` class, then the compiler needs to consider 4,800 `#include` lines. Most of them are edges it'll discount since they've already been included but the sum total of files that get included is 694!" [1]: https://github.com/jart/cosmopolitan/blob/master/ctl/README.md https://github.com/jart/cosmopolitan/blob/master/ctl/README....
bdahz 2y ago
How does cosmopolitan support GPU-based applications?
nuc1e0n 2y ago
Can anyone point me to examples of this being used as a dependancy of other projects?

Cosmopolitan v3.5

366 points by sgammon 2y ago

WantonQuantum 2y ago

I assume the executables won't run on ARM platforms? So modern Macs are out, etc.

nabla9 2y ago
It works on AMD64 and ARM64 both. It makes two architecture fat binary, but removes all code not needed. Resulting binary is typically only 30-50% larger than 1-architecture binary.
- mkl 2y ago
  More details of supported platforms at https://justine.lol/cosmo3/ https://justine.lol/cosmo3/ and https://github.com/jart/cosmopolitan/blob/master/tool/cosmocc/README.md https://github.com/jart/cosmopolitan/blob/master/tool/cosmoc..., both linked in the release notes.
lukemerrick 2y ago
Looks like there is both an ARM and x86 version according to the docs. Probably need two different binaries, but you still get cross-OS for each architecture.
- eesmith 2y ago
  It's a single binary, cross-OS, cross-arch. Go to https://cosmo.zip/pub/cosmos/bin/ https://cosmo.zip/pub/cosmos/bin/ and download an executable, like https://cosmo.zip/pub/cosmos/bin/basename https://cosmo.zip/pub/cosmos/bin/basename . % curl -O https://cosmo.zip/pub/cosmos/bin/basename % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 663k 100 663k 0 0 440k 0 0:00:01 0:00:01 --:--:-- 441k % chmod +x basename On macOS with an M1: % arch arm64 % sha256sum basename 2e4cf8378b679dd0c2cdcc043bb1f21eaf464d3c3b636ed101fbc5e3230eb5fb basename % file ./basename ./basename: DOS/MBR boot sector; partition 1 : ID=0x7f, active, start-CHS (0x0,0,1), end-CHS (0x3ff,255,63), startsector 0, 4294967295 sectors % ./basename /dev/null.txt null.txt On FreeBSD with an amd64: % uname -sm FreeBSD amd64 % sha256sum basename 2e4cf8378b679dd0c2cdcc043bb1f21eaf464d3c3b636ed101fbc5e3230eb5fb basename % file basename basename: DOS/MBR boot sector; partition 1 : ID=0x7f, active, start-CHS (0x0,0,1), end-CHS (0x3ff,255,63), startsector 0, 4294967295 sectors % ./basename /dev/null.pdf null.pdf
  - alganet 2y ago
    Kind of. It can be read as a single binary, some supported systems will do that. In others, the executable is first parsed as a shell script. There's definitely more to it than a single binary. MZqFpD=' ø @ ' <<'justine0e5c7z' È ²@ë ëëHƒì1Ò½ ëéE ü‡>à¿ p1ÉŽÁúŽ×‰Ìûè ^îr ¸ PP 1ÿ¹ ó¤‡ÒÿêŽ ŽÙ¹ ¸P ŽÀ1À1ÿóª€ú@tè °1É0ö¿Pèo ŒÆƒÆ ŽÆOuóê ' SR´Ís1ÀÍrF¸¹ ¶ » ŽÃ1ÛÍr3´Ír-ˆÏ€ç?€áÀÐÁÐÁ†Í1öŽÆ¾‡÷¥¥¥¥¥¤“«®‘«’«Xª’[ÃZ€ò€1ÀÍr÷ë¡PQ†ÍÐÉÐÉÁ1Û°´ÍYXrþÀ:$v °þÆ:6)v0öAÃP1ÀÍXëÌ‰þ¬„Àt » ´ÍëòÃW¿¹&èèÿ_èäÿ¿Á&èÞÿóëü¹ ¾ …Àt QV—¾Ð&è ^YâîÃ‰ú…ÒtRV1É±Ê¬^ €îZ¬îBIyúÃ € ÿÿÿ ÿÿÿÿ Uª justine0e5c7z #'" o=$(command -v "$0") [ x"$1" != x--assimilate ] && type ape >/dev/null 2>&1 && exec ape "$o" "$@" t="${TMPDIR:-${HOME:-.}}/.ape-1.10" [ x"$1" != x--assimilate ] && [ -x "$t" ] && exec "$t" "$o" "$@" m=$(uname -m 2>/dev/null) || m=x86_64 if [ ! -d /Applications ]; then if [ x"$1" = x--assimilate ]; then if [ "$m" = x86_64 ] || [ "$m" = amd64 ]; then exec 7<> "$o" || exit 121 printf '\177ELF\2\1\1\11\0\0\0\0\0\0\0\0\2\0>\0\1\0\0\0vE@\0\0\0\0\0\510\013\000\000\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\100\0\70\0\006\000\0\0\0\0\0\0' >&7 exec 7<&- fi exit
    - bux93 2y ago
      Reminds me of a cool tool I once used, uudecode.com, which was a DOS binary that only used 7-bit characters and could decode uuencoded (base64 predecessor) files. Was useful for getting attachments through e-mail in the face of all kinds of filters.
      - eesmith 2y ago
        https://news.ycombinator.com/item?id=16312317 https://news.ycombinator.com/item?id=16312317 - "A C89 compiler that produces executables that are also valid ASCII text files" http://tom7.org/abc/ http://tom7.org/abc/ The paper for SIGBOVIK 2017 is both a text file and an executable.
        metadat 2y ago
        Really near. Also discussed 11 months ago: https://news.ycombinator.com/item?id=36821985 https://news.ycombinator.com/item?id=36821985 (15 comments)
d4rti 2y ago
Works for sure: $ file ./bin/python ./bin/python: DOS/MBR boot sector; partition 1 : ID=0x7f, active, start-CHS (0x0,0,1), end-CHS (0x3ff,255,63), startsector 0, 4294967295 sectors $ ./bin/python Python 3.11.4 (heads/pypack1:6eea485, Jan 24 2024, 10:14:24) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.processor() 'arm'
- 4gotunameagain 2y ago
  Out of curiosity, why are you running python packed with cosmopolitan ?
  - simonw 2y ago
    Presumably to demonstrate that it works on Mac ARM platforms, in response to the question about that.
    - 4gotunameagain 2y ago
      If was satisfied with my own presumption I wouldn't have asked ;)
      - simonw 2y ago
        That Python app is a popular demo for Cosmopolitan. It's what I would have chosen for that demo, too! It's handy because it outputs a little bit of information about the current architecture on the first line when you start the shell. There's some interesting background on it in this issue: https://github.com/jart/cosmopolitan/issues/141 https://github.com/jart/cosmopolitan/issues/141
- smcameron 2y ago
  I forget how I did this, but: $ file hello.com hello.com: αcτµαlly pδrταblε εxεcµταblε
Aissen 2y ago
It depends. It does not work on CentOS 8 / Rocky 8 for example: https://github.com/jart/cosmopolitan/issues/1151 https://github.com/jart/cosmopolitan/issues/1151
- jart 2y ago
  Is that an embedded device with a small address space? APE used to require a 47bit address space. The release I published today fixes that. Maybe give it another try?
  - Aissen 2y ago
    No, it's a server with 256GB of RAM, but indeed the kernel config has CONFIG_ARM64_VA_BITS_48=y (and 52 for userspace). I have since moved away from this kernel, I'll see if I find the time to boot it again to test the latest memory map manager. Edit: I tried booting it, the issue is still present, I updated the github issue.

speedgoose 2y ago

The repository also contains the source code for redbean, which is a neat web server with lua and plenty of features. https://redbean.dev/ https://redbean.dev/ https://github.com/jart/cosmopolitan/tree/master/tool/net https://github.com/jart/cosmopolitan/tree/master/tool/net

sgt_bilko 2y ago
The Lua fullmoon web framework is built around redbean. https://github.com/pkulchenko/fullmoon https://github.com/pkulchenko/fullmoon
- morbicer 2y ago
  Wow this is awesome, thanks for the tip
ncruces 2y ago
Having written a portable SQLite VFS from scratch, I'm a bit scared of how they blindly translate fcntl locks into LockFileEx in a forking web server. The opportunity for data corruption just freaks me out.
- deleted 2y ago
  [deleted]

m1keil 2y ago

Cosmo looks so cool. Are there any big projects out there that use it for distribution?

PhilippGille 2y ago
Llamafile: https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile

mkl 2y ago

Related re Windows on ARM https://news.ycombinator.com/item?id=40772475 https://news.ycombinator.com/item?id=40772475.

trustno2 2y ago

I'm always somewhat scared - not sure if rationally or irrationally - about cosmopolitan. It's a cool hack but I somehow feel like it should not work.

hiAndrewQuinn 2y ago
I'm somewhat of the same mind, but I'm fairly sure a study of Operating Systems: Three Easy Steps would get me over the hump. There's no actual reason to suspect this thing would like, sidestep normal process management or memory virtualization or something and run amok... I think.
alganet 2y ago
It should work. We should have portable executables across many operating systems and architectures. The other expectation is scary.
- cpach 2y ago
  There’s also Java, Python, Ruby, NodeJS etc etc.
- 3836293648 2y ago
  Working across unices is reasonable. Adding Windows is less so
  - cpach 2y ago
    It’s quite rare to ship multi-Unix binaries, isn’t it? Cosmopolitan is a very cool hack indeed, but IMHO it’s not likely to gain widespread usage.
  - flohofwoe 2y ago
    A thin POSIX layer and ELF loader shouldn't be too much of a problem for Microsoft to implement if they wanted to do so (WinNT actually did have a POSIX personality at some point, but I don't think that's still supported). I'd also like to see a builtin WASM runtime in all operating systems.

auggierose 2y ago

Is there a UI library that goes with that?

DrNosferatu 2y ago
A Web Interface for Cosmopolitan Apps would be a great bridge for an UI.
- auggierose 2y ago
  I am looking for an Electron replacement for writing my apps. Could cosmopolitan be the basis of something like that, or is that not really its purpose?
  - brabel 2y ago
    Lua is perfect for Redbean because it's tiny and made for embedding, which is this exactly use case. Something like TypeScript would require a transpiler to JS and a JS runtime, which as far as I know, even the smallest one would be much bigger than Lua. And Lua has well defined semantics to call C and be called by C, which may be hard to do with JS/TS.
    - DrNosferatu 2y ago
      Any examples and good tutorials on how to best go about it?
      - brabel 2y ago
        Best go about using Redbean? The project is well documented: https://redbean.dev/ https://redbean.dev/ Just go through it, it answers all questions you may come up with IMO. Or you mean about embedding Lua? The official docs are great too: https://www.lua.org/pil/24.html https://www.lua.org/pil/24.html If you prefer a tutorial, there's plenty as Lua has been around for a long time. Search on Youtube for example and there's lots of videos. The top hit on DDG for me was https://lucasklassmann.com/blog/2019-02-02-embedding-lua-in-c/ https://lucasklassmann.com/blog/2019-02-02-embedding-lua-in-... which seems pretty straightforward to follow.

pixelpoet 2y ago

I checked two commits and they both had exactly the same number of chars per line, e.g. https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3df6b1f4a426f8ee89d3d20ac https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3... I enjoy doing this too sometimes and don't find it too difficult, but damn...

whenlambo 2y ago
https://www.youtube.com/watch?v=Y65FRxE7uMc https://www.youtube.com/watch?v=Y65FRxE7uMc
- lioeters 2y ago
  I have found my people.
- cgh 2y ago
  Amazing, thank you for this.
- pixelpoet 2y ago
  I thought I was alone!! wipes tears of joy
- bdjsiqoocwk 2y ago
  Well. Lorem Ipsum vs Donald Knuth. I think they both suck. I have a different hero. Linus Torvalds. He IS good AND actually delivers. I enjoyed the video. However, both the author and Donald Keith strike as two people who are mainly motivated by aesthetics (which is supremely ironic as they almost define themselves as being the opposite of Loren Ipsum). I like people who deliver results. I think the author mostly likes the appearance of someone who pays attention to detail. He says that in a movie if it's obvious that the coffee cups are empty, he notices and bothers him. 3 minutes later in his video he has a guy using his laptop at the beach where glare would make it impossible to see anything on the screen.
lproven 2y ago
Back in the pre-GUI era, I did this with all my posts and comments online. All fully-justified, both margins, without any extra whitespace: just careful choice of words, and occasional (but always normal standard) abbreviations. It was fun, once practised it barely slowed me down, and when people noticed it blew their minds. Then along came GUIs and proportional fonts and made it all invisible. :'(

Nevin1901 2y ago

The "actually portable executable header", fun to read through: https://raw.githubusercontent.com/jart/cosmopolitan/1.0/ape/ape.S https://raw.githubusercontent.com/jart/cosmopolitan/1.0/ape/...

dirkc 2y ago
I just scrolled for the images, I don't know about reading assembly for fun :)

j16sdiz 2y ago

in release note: > ... , it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format ... Did POSIX really _approve_ this? if yes, when?

xiaq 2y ago
Look for “POSIX” on https://justine.lol/cosmo3/ https://justine.lol/cosmo3/
- oguz-ismail 2y ago
  POSIX now says: >The input file can be of any type, but the initial portion of the file intended to be parsed according to the shell grammar [...] shall not contain the NUL character. But the initial portion of, https://cosmo.zip/pub/cosmos/bin/cat https://cosmo.zip/pub/cosmos/bin/cat for example, does contain the NUL character.
  - jart 2y ago
    Cosmo author here. I'm familiar with the new rules. They were first implemented in https://github.com/freebsd/freebsd-src/commit/e0f5c1387df23c8c4811f5b24a7ef6ecac51a71a https://github.com/freebsd/freebsd-src/commit/e0f5c1387df23c... which makes a clear intentional allowance for what APE is doing. If you look at the hex dump: main jart@luna:~/llamafile$ hexdump -C cat | head 00000000 4d 5a 71 46 70 44 3d 27 0a 0a 00 10 00 f8 00 00 |MZqFpD='........| 00000010 00 00 00 00 00 01 00 08 40 00 00 00 00 00 00 00 |........@.......| You'll notice there's no NUL characters on the first line, and that the subsequent NULs are escaped by a single-quoted string, which is legal. The rules used to be more restrictive but they relaxed the requirements specifically so I could work on APE. Jilles Tjoelker is one of the heroes who made that possible.
    - oguz-ismail 2y ago
      [T]he initial portion doesn't mean the first line, it means the script part of a file consisting of a shell script and a binary payload, separated by `exit', `exec whatever', etc. A good example is the Oracle Developer Studio installation script. You can write to Austin Group mailing list and ask for clarification if you want.

zbendefy 2y ago

wouldn't targeting webassembly provide something similiar? how would this compare to targeting WASM and running a runtime like wasmer?

optymizer 2y ago
WebAssembly is bytecode that runs on a virtual machine. Cosmopolitan is a libc replacement. Your program is still compiled to real CPU instructions, which execute much faster.

tiffanyh 2y ago

I assumed OpenBSD introducing pinning syscalls would prevent Cosmo from working. I guess not. https://news.ycombinator.com/item?id=38579913 https://news.ycombinator.com/item?id=38579913

geokon 2y ago

It'd be fun/great if someone made Graal Native work with it. Then things could come full circle and I could compile my Clojure/Java binaries to run everywhere (but now without a VM)

sgammon 2y ago
work is ongoing
- geokon 2y ago
  I'm quite fuzzy on the technical details - but do you think it'd ever lead to JavaFX running everywhere? It was a while before JFX worked with Graal Native I'd like to someday be able to easily distribute my little GUI apps (for scientific applications) in a compile-once kind of way. Asking users to install a JVM runtime has turned out to be unworkable (everyone installs the Java 8 runtime b/c that's what comes up first on Google and then the apps silently crash/don't-work and users are confused)
  - throwaway2037 2y ago
    > everyone installs the Java 8 runtime b/c that's what comes up first on Google and then the apps silently crash/don't-work and users are confused Is it not possible to add a tiny check from main()? If version is less than required, then show a (dreaded) JMessageBox or print message to STDOUT/STDERR where users can safely download the correct JDK.
    - geokon 2y ago
      No idea. I'm guessing the process doesn't even launch (I don't remember if I checked this..). Do you have an example with a check ? I'd love to add it. You'd probably need a very basic Swing error popup
    - sgammon 2y ago
      No, it’s not possible, because bytecode compiled at a higher level cannot be run by JDK8. It can’t even parse the class file. Most projects are compiling at JDK11 or JDK17 now at least, if not JDK21, which is the latest LTS. the latest release is JDK22. so if you’re running 8, you won’t be able to run much modern Java at all.

superkuh 2y ago

Any idea if there will ever be a fix for the majority of linux distributions that cannot run cosmopolitan C applications without the bin fmt fix?

deleted 2y ago
[deleted]

meisel 2y ago

Although this project is undoubtedly very cool, and maybe simplifies build processes by having a single binary, is there any other reason to use it? How does it compare in terms of performance, static linkability, standards conformance, etc. with musl and glibc? I’m curious because I’m picking a libc at the moment for my project.

doctorpangloss 2y ago
> How does it compare in terms of performance I tried rsync for Windows from its "cosmos" pile of binaries (https://github.com/jart/cosmopolitan/releases/download/3.3.1/cosmos-3.3.1.zip https://github.com/jart/cosmopolitan/releases/download/3.3.1...), and it was too slow to be usable compared to WSL rsync. For many other utilities I use Busybox for Windows (https://github.com/rmyorston/busybox-w32 https://github.com/rmyorston/busybox-w32) which is well maintained, fast, and its maintainer is also very responsive to bug reports.
- jart 2y ago
  You're basically just benchmarking the WIN32 filesystem versus a Linux VM in that case. The Windows file system is famously slow. It'd make more sense to compare a Cosmo binary running on WIN32 with a WIN32 native program. For example, Emacs has a WIN32 port where they wrote all the Windows polyfills on their own, and because they aren't experts on Windows, the official GNU Emacs Windows releases go a lot slower on Windows than if you just compile Emacs from source on Linux using Cosmo and scp the the binary over to run on Windows. See https://justine.lol/cosmo3/ https://justine.lol/cosmo3/
tracker1 2y ago
I don't know enough to compare to musl, but I suspect being ISC licensed (similar to musl's MIT license) allows for static linking without upstream impact such as GPL. Would be curious regarding features and performance comparisons with musl, which seems to come up a little short compared to say gnu libc.
- saghm 2y ago

stephc_int13 2y ago

Is there a plan to distribute the cosmopolitan libc without all the APE stuff? For now it seems to be more like a framework+toolchain than a standalone library.

deleted 2y ago
[deleted]

aidenn0 2y ago

I've been following it for almost 4 years now and I still do not know how serious of a project Cosmopolitain/αpε is.

TimTheTinker 2y ago
The author is certainly very serious about writing highly distributable production software that works and will continue to work far into the future. It's already being used for serious things - Mozilla's llamafile and redbean are two examples that come to mind.

tarruda 2y ago

I wonder if it is possible to have a 100% portable qemu build with this. Would be funny to have a usb stick with qemu + VM that runs on any OS.

jart 2y ago
blink is almost as good as qemu-user and it works great compiled with cosmocc. https://github.com/jart/blink https://github.com/jart/blink

modeless 2y ago

The commit message for the first commit there says "Actually Portable Executable now supports Android". I assume this means that the bare executables can run on Android kernels, not that there's any support for installing APEs as Android apps. But it seems possible for that to eventually work! Is that a goal of the project?

fusslo 2y ago

Never heard of Cosmopolitan before, but can I just say I really appreciate that the FIRST paragraph is a description of WHAT Cosmopolitan is and does. There's way too many readme's that just jump in and assume the reader already is familiar with the project. it's a pet peeve of mine

deleted 2y ago
[deleted]

deleted 2y ago

[deleted]

hahaitsfunny 2y ago

[dead]

kortex 2y ago

Looking forward toward somebody hooking together Python in APE [0], something like pex [1]/shiv[2]/pyinstaller[3], and the pants build system [4] to have a toolchain which spits out single-file python executables with baked-in venv and portable across mainstream OSes with native (or close enough) performance. 0 - https://news.ycombinator.com/item?id=40040342 https://news.ycombinator.com/item?id=40040342 1 - https://docs.pex-tool.org/ https://docs.pex-tool.org/ 2 - https://shiv.readthedocs.io/en/latest/ https://shiv.readthedocs.io/en/latest/ 3 - https://pyinstaller.org/en/stable/ https://pyinstaller.org/en/stable/ 4 - https://www.pantsbuild.org/ https://www.pantsbuild.org/

WaxProlix 2y ago
You can probably generate C code from Python now with Nuitka and pump that into this Cosmopolitan tool, today, to get that? https://nuitka.net/ https://nuitka.net/

sangnoir 2y ago

I gave up trying to run cgo-compiled Go on an old appliance[1] the version of libc was simply too old. Has anyone ever successfully built cosmopolitan-flavored cgo binaries? I see Cosmopolitan libc supports Linux 2.6, so I'm hopeful. 1. Linux 2.x on little-endian MIPS.

throwaway2037 2y ago

From here: https://github.com/jart/cosmopolitan/releases/tag/3.5.0 https://github.com/jart/cosmopolitan/releases/tag/3.5.0 Justine wrote: > We've made a lot of progress reinventing the C++ STL. What is the motivation behind this? Try to reduce (compiled) binary size? Naively, I am surprised that the C++ STL from Clang isn't OK to use, including the license. Or is this a clean room impl thing? EDIT To be clear: I mean no disrespect with this question. This is an amazing project. EDIT 2 Ok, I just spotted this commit message: https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3df6b1f4a426f8ee89d3d20ac https://github.com/jart/cosmopolitan/commit/c4c812c15445f5c3... > We now have a C++ red-black tree implementation that implements standard template library compatible APIs while compiling 10x faster than libcxx. Can someone explain to an average C++ programmer like me how they make it compile 10x faster !?

simjnd 2y ago
The rationale is explained in this README [1], but it basically boils down to transitive dependencies and the amount of checks the compiler has to do. From linked README: "If we `#include <string>` for the LLVM libcxx header containing the `std::string` class, then the compiler needs to consider 4,800 `#include` lines. Most of them are edges it'll discount since they've already been included but the sum total of files that get included is 694!" [1]: https://github.com/jart/cosmopolitan/blob/master/ctl/README.md https://github.com/jart/cosmopolitan/blob/master/ctl/README....

bdahz 2y ago

How does cosmopolitan support GPU-based applications?

nuc1e0n 2y ago

Can anyone point me to examples of this being used as a dependancy of other projects?