5 ms·

Cosmopolitan v3.5.8

103 points by sgammon 2y ago

oguz-ismail 2y ago
The README for this project still says "POSIX-approved" but there is nothing in POSIX that allows mixing commands and binary data in a shell script. It does allow appending binary data to the end of a shell script* as long as it is guarded from being parsed as shell code by an exit, exec, etc. but that's it. * https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sh.html#tag_20_110_07 https://pubs.opengroup.org/onlinepubs/9799919799/utilities/s...
- yawpitch 2y ago
  Definitely not my area of either pedantry or expertise, but if I read this correctly: > The input file can be of any type, but the initial portion of the file intended to be parsed according to the shell grammar […] shall consist of characters and shall not contain the NUL character. Then POSIX doesn’t proscribe against appending binary data at all, it just requires that binary data appears after whatever portion of the file is intended to be parsed according to the grammar… there’s nothing stating that intention can’t apply to a zero characters. > The shell shall not enforce any line length limits. That seems to unambiguously allow a zero-length initial portion. > If the input file consists solely of zero or more blank lines and comments, sh shall exit with a zero exit status. And so does that, while also making an enormous loophole for any input file that is not zero or more blank lines. In other words it seems to me on a quick reading that a decent rules-lawyer could drive a planet through that part of the spec, as written.
- imiric 2y ago
  According to Justine herself[1], POSIX specifically changed their rules to allow this type of functionality[2]. The paragraph you linked to was part of it, but also this paragraph under RATIONALE[3]: > Earlier versions of this standard required that input files to the shell be text files except that line lengths were unlimited. However, that was overly restrictive in relation to the fact that shells can parse a script without a trailing newline, and in relation to a common practice of concatenating a shell script ending with an exit or exec $command with a binary data payload to form a single-file self-extracting archive. So Cosmopolitan indeed seems to be POSIX-approved as far as this behavior goes. [1]: https://justine.lol/cosmo3/ https://justine.lol/cosmo3/ [2]: https://austingroupbugs.net/view.php?id=1250 https://austingroupbugs.net/view.php?id=1250 [3]: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sh.html#tag_20_110_18 https://pubs.opengroup.org/onlinepubs/9799919799/utilities/s...
  - oguz-ismail 2y ago
    [flagged]
    - Y_Y 2y ago
      I think this isn't an entirely worthless comment, but you start off by insisting that Justine Tunney is referred to as "himself". This is either a silly mistake, or a misguided political statement. Either way I don't think it adds to the discussion, and it's certainly going to distract people from your more interesting point that mixing binary data throughout the shell script is not permitted by the standard.
      - sgammon 2y ago
        They made this comment as a correction. It is intentional. When somebody shows you who they are, believe them.
        AlexeyBelov 2y ago
        And it's always the fresh accounts.
      - jart 2y ago
        It's not an interesting point, it's a willful misinterpretation of the intent of the standard made by someone who wasn't there. The worst thing about bigots is they usually hide it and grasp for other more socially acceptable reasons to make you miserable when you're someone like me. I'm astonished you would still continue to take them seriously after they've revealed their intent and motivation.
        Y_Y 2y ago
        I apologise, my interpretation was naïve. Not being familiar with the intent of the person I was responding to (directly or by experience) and not knowing the how to correctly interpret the POSIX standard, I was trying to politely re-focus the thread. I'm lucky enough not to have to spend much time thinking about identity issues or being the target of bigotry and I suppose that was reflected in my previous comment. Maybe it's telling that you're here making fantastic contributions for FOSS and people who are trying to bring you down are only making invalid claims about standards compliance. I'm a big fan of yours and I'm really sorry that I contributed to that incident.
- Cloudef 2y ago
  POSIX is one of those standards that don't really define anything at all. Microsoft POSIX subsystem is one of the nicest examples of something that lets you tick the POSIX checkbox but does almost nothing. One of the recent posix annoyances I stumbled upon was the clock_gettime where every platform has slightly different idea how the different CLOCKs are supposed to be interpreted :) Personally I don't find the APE part of cosmopolitan that interesting, but their own POSIX API <https://justine.lol/cosmopolitan/functions.html https://justine.lol/cosmopolitan/functions.html> which also covers windows is interesting for sure. Zig's std also offers something similar, but perhaps not as good coverage. Getting fork, mmap, and poll work on windows is not small feat.
DidYaWipe 2y ago
[flagged]
schmorptron 2y ago
For those who don't want to click through to the main github description: "Cosmopolitan Libc makes C a build-anywhere run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. Instead, it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS on AMD64 and ARM64 with the best possible performance. "
- DidYaWipe 2y ago
  That should've been in the title: "Cosmopolitan Libc makes C a build-anywhere run-anywhere language" Let the furious downvoting begin.
- amelius 2y ago
  If this means the end of libc version errors, then count me in. One question, is C++ supported?
  - jart 2y ago
    Absolutely. Just download and extract cosmocc-3.5.8.zip and use its cosmoc++ command instead of g++. By default it'll use LLVM's libcxx STL although we've been working on our own versions of classes like std::set (called ctl::set) that are better. For example, just using ctl::vector alone might help your C++ code compile noticeably faster. https://github.com/jart/cosmopolitan/tree/master/ctl https://github.com/jart/cosmopolitan/tree/master/ctl
- tarruda 2y ago
  > runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS on AMD64 and ARM64 with the best possible performance Is it possible that a single binary will run on both AMD64 and ARM64?
  - bheadmaster 2y ago
    As long as the subset of compatible instructions exist, it is likely possible to program a binary version of If you speak english, go to page 7. Si hablas español, ve a la página 25. Note that the instructions don't have to be completely equivalent - all that's required is for some set of bytes to be arranged such that the same pattern will be interpreted by AMD64 as jump to X and by ARM64 as jump to Y. Afterwards, it's all about segmenting the instructions in blocks, one for each OS/architecture.
relistan 2y ago
Lots of focus in other comments on pedantry. This looks like a cool project.
- deleted 2y ago
  [deleted]
masjl 2y ago
[flagged]
deleted 2y ago
[deleted]

103 points by sgammon 2y ago

The README for this project still says "POSIX-approved" but there is nothing in POSIX that allows mixing commands and binary data in a shell script. It does allow appending binary data to the end of a shell script* as long as it is guarded from being parsed as shell code by an exit, exec, etc. but that's it. * https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sh.html#tag_20_110_07 https://pubs.opengroup.org/onlinepubs/9799919799/utilities/s...

yawpitch 2y ago
Definitely not my area of either pedantry or expertise, but if I read this correctly: > The input file can be of any type, but the initial portion of the file intended to be parsed according to the shell grammar […] shall consist of characters and shall not contain the NUL character. Then POSIX doesn’t proscribe against appending binary data at all, it just requires that binary data appears after whatever portion of the file is intended to be parsed according to the grammar… there’s nothing stating that intention can’t apply to a zero characters. > The shell shall not enforce any line length limits. That seems to unambiguously allow a zero-length initial portion. > If the input file consists solely of zero or more blank lines and comments, sh shall exit with a zero exit status. And so does that, while also making an enormous loophole for any input file that is not zero or more blank lines. In other words it seems to me on a quick reading that a decent rules-lawyer could drive a planet through that part of the spec, as written.
imiric 2y ago
According to Justine herself[1], POSIX specifically changed their rules to allow this type of functionality[2]. The paragraph you linked to was part of it, but also this paragraph under RATIONALE[3]: > Earlier versions of this standard required that input files to the shell be text files except that line lengths were unlimited. However, that was overly restrictive in relation to the fact that shells can parse a script without a trailing newline, and in relation to a common practice of concatenating a shell script ending with an exit or exec $command with a binary data payload to form a single-file self-extracting archive. So Cosmopolitan indeed seems to be POSIX-approved as far as this behavior goes. [1]: https://justine.lol/cosmo3/ https://justine.lol/cosmo3/ [2]: https://austingroupbugs.net/view.php?id=1250 https://austingroupbugs.net/view.php?id=1250 [3]: https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sh.html#tag_20_110_18 https://pubs.opengroup.org/onlinepubs/9799919799/utilities/s...
- oguz-ismail 2y ago
  [flagged]
  - Y_Y 2y ago
    I think this isn't an entirely worthless comment, but you start off by insisting that Justine Tunney is referred to as "himself". This is either a silly mistake, or a misguided political statement. Either way I don't think it adds to the discussion, and it's certainly going to distract people from your more interesting point that mixing binary data throughout the shell script is not permitted by the standard.
    - sgammon 2y ago
      They made this comment as a correction. It is intentional. When somebody shows you who they are, believe them.
      - AlexeyBelov 2y ago
        And it's always the fresh accounts.
    - jart 2y ago
      It's not an interesting point, it's a willful misinterpretation of the intent of the standard made by someone who wasn't there. The worst thing about bigots is they usually hide it and grasp for other more socially acceptable reasons to make you miserable when you're someone like me. I'm astonished you would still continue to take them seriously after they've revealed their intent and motivation.
      - Y_Y 2y ago
        I apologise, my interpretation was naïve. Not being familiar with the intent of the person I was responding to (directly or by experience) and not knowing the how to correctly interpret the POSIX standard, I was trying to politely re-focus the thread. I'm lucky enough not to have to spend much time thinking about identity issues or being the target of bigotry and I suppose that was reflected in my previous comment. Maybe it's telling that you're here making fantastic contributions for FOSS and people who are trying to bring you down are only making invalid claims about standards compliance. I'm a big fan of yours and I'm really sorry that I contributed to that incident.
Cloudef 2y ago
POSIX is one of those standards that don't really define anything at all. Microsoft POSIX subsystem is one of the nicest examples of something that lets you tick the POSIX checkbox but does almost nothing. One of the recent posix annoyances I stumbled upon was the clock_gettime where every platform has slightly different idea how the different CLOCKs are supposed to be interpreted :) Personally I don't find the APE part of cosmopolitan that interesting, but their own POSIX API <https://justine.lol/cosmopolitan/functions.html https://justine.lol/cosmopolitan/functions.html> which also covers windows is interesting for sure. Zig's std also offers something similar, but perhaps not as good coverage. Getting fork, mmap, and poll work on windows is not small feat.

DidYaWipe 2y ago

[flagged]

schmorptron 2y ago

For those who don't want to click through to the main github description: "Cosmopolitan Libc makes C a build-anywhere run-anywhere language, like Java, except it doesn't need an interpreter or virtual machine. Instead, it reconfigures stock GCC and Clang to output a POSIX-approved polyglot format that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS on AMD64 and ARM64 with the best possible performance. "

DidYaWipe 2y ago
That should've been in the title: "Cosmopolitan Libc makes C a build-anywhere run-anywhere language" Let the furious downvoting begin.
amelius 2y ago
If this means the end of libc version errors, then count me in. One question, is C++ supported?
- jart 2y ago
  Absolutely. Just download and extract cosmocc-3.5.8.zip and use its cosmoc++ command instead of g++. By default it'll use LLVM's libcxx STL although we've been working on our own versions of classes like std::set (called ctl::set) that are better. For example, just using ctl::vector alone might help your C++ code compile noticeably faster. https://github.com/jart/cosmopolitan/tree/master/ctl https://github.com/jart/cosmopolitan/tree/master/ctl
tarruda 2y ago
> runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS on AMD64 and ARM64 with the best possible performance Is it possible that a single binary will run on both AMD64 and ARM64?
- bheadmaster 2y ago
  As long as the subset of compatible instructions exist, it is likely possible to program a binary version of If you speak english, go to page 7. Si hablas español, ve a la página 25. Note that the instructions don't have to be completely equivalent - all that's required is for some set of bytes to be arranged such that the same pattern will be interpreted by AMD64 as jump to X and by ARM64 as jump to Y. Afterwards, it's all about segmenting the instructions in blocks, one for each OS/architecture.

relistan 2y ago

Lots of focus in other comments on pedantry. This looks like a cool project.

deleted 2y ago
[deleted]

masjl 2y ago

[flagged]

deleted 2y ago

[deleted]