5 ms·

Seems impossible on the face of it. Linux binaries have to start with 0x7F ELF and Windows binaries have to start with MZ or ZM so I don't get how it could wor

by ape4 6y ago

Seems impossible on the face of it. Linux binaries have to start with 0x7F ELF and Windows binaries have to start with MZ or ZM so I don't get how it could work with both.

icedchai 6y ago
Based on the generated code, you may need to broaden the definition of "binary" to include anything that is executable (shell script.)
- chungy 6y ago
  It's not really a shell script though, it just looks like one enough to the point that a bourne shell can modify it to be directly executable. I'd probably be most worried about the bourne shell part. It seems like it'll be easy for this to break on any "non-standard" shell that Linux might execute.
  - icedchai 6y ago
    This is true, but the shell is executing it "as if" it were a script, right? Until the binary part takes over...
  - jart 6y ago
    The behavior that Cosmopolitan needs is required by POSIX because UNIX Sixth Edition didn't have shebang lines. It's normally an implementation detail of execve() in C libraries and the behavior is implemented by nearly all shells too. So far the only issue has been that some shells, like zsh, have a binary safety check where it refuses to run scripts that have embedded NUL characters. I worked with the FreeBSD team to get POSIX revised to clarify the correct behavior w.r.t. these restrictions. FreeBSD's /bin/sh has already been patched and zsh is pending upstream. https://github.com/jart/zsh/commit/94a4bc14bb2e415ec3d10cf716512bd3e0d99f48 https://github.com/jart/zsh/commit/94a4bc14bb2e415ec3d10cf71...
    - ashkankiani 6y ago
      I appreciate the thoroughness with which you are pursuing this. The amount of work you're doing outside of the technical aspect is definitely something I can admit that I would've never pursued. It's encouraging to see.
- fao_ 6y ago
  /bin/sh execution is supported at the kernel level iirc? Like, the minimum requirement for a linux distribution is /bin/sh, the rest of the system doesn't have to be there necessarily.
  - deleted 6y ago
    [deleted]
  - deleted 6y ago
    [deleted]
  - 10000truths 6y ago
    No, the Linux kernel only requires an init executable to run. It otherwise makes no assumptions about the programs available in userland. Pretty much any sane/usable Linux distribution will package a shell, though.
    - saagarjha 6y ago
      It does, however, hardcode /bin/sh as a init of last resort: https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f3d58a21b01fdb14e749/init/main.c#L1476 https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f...
      - fao_ 6y ago
        Exactly what I was talking about, thank you
  - saagarjha 6y ago
    The behavior your are likely thinking of (execvp on a file without a shebang) is implemented in libc.
    - fao_ 6y ago
      No. As someone else mentioned, the kernel hardcodes /bin/sh as a last resort: https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f3d58a21b01fdb14e749/init/main.c#L1476 https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f...
lpapez 6y ago
As explained by the author https://justine.lol/ape.html https://justine.lol/ape.html
- Dylan16807 6y ago
  In short: The start of the file is MZ but also a shell script. If you run it on linux it replaces its header with the ELF header, then execs itself.
im3w1l 6y ago
And the reason Windows requires MZ is so that the program is also a valid dos program (that prints an error and exits). According to wikipedia MZ were picked to be the initials of one of the leading dos developers. Turtles all the way down.

by ape4 6y ago

Seems impossible on the face of it. Linux binaries have to start with 0x7F ELF and Windows binaries have to start with MZ or ZM so I don't get how it could work with both.

icedchai 6y ago

Based on the generated code, you may need to broaden the definition of "binary" to include anything that is executable (shell script.)

chungy 6y ago
It's not really a shell script though, it just looks like one enough to the point that a bourne shell can modify it to be directly executable. I'd probably be most worried about the bourne shell part. It seems like it'll be easy for this to break on any "non-standard" shell that Linux might execute.
- icedchai 6y ago
  This is true, but the shell is executing it "as if" it were a script, right? Until the binary part takes over...
- jart 6y ago
  The behavior that Cosmopolitan needs is required by POSIX because UNIX Sixth Edition didn't have shebang lines. It's normally an implementation detail of execve() in C libraries and the behavior is implemented by nearly all shells too. So far the only issue has been that some shells, like zsh, have a binary safety check where it refuses to run scripts that have embedded NUL characters. I worked with the FreeBSD team to get POSIX revised to clarify the correct behavior w.r.t. these restrictions. FreeBSD's /bin/sh has already been patched and zsh is pending upstream. https://github.com/jart/zsh/commit/94a4bc14bb2e415ec3d10cf716512bd3e0d99f48 https://github.com/jart/zsh/commit/94a4bc14bb2e415ec3d10cf71...
  - ashkankiani 6y ago
    I appreciate the thoroughness with which you are pursuing this. The amount of work you're doing outside of the technical aspect is definitely something I can admit that I would've never pursued. It's encouraging to see.
fao_ 6y ago
/bin/sh execution is supported at the kernel level iirc? Like, the minimum requirement for a linux distribution is /bin/sh, the rest of the system doesn't have to be there necessarily.
- deleted 6y ago
  [deleted]
- deleted 6y ago
  [deleted]
- 10000truths 6y ago
  No, the Linux kernel only requires an init executable to run. It otherwise makes no assumptions about the programs available in userland. Pretty much any sane/usable Linux distribution will package a shell, though.
  - saagarjha 6y ago
    It does, however, hardcode /bin/sh as a init of last resort: https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f3d58a21b01fdb14e749/init/main.c#L1476 https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f...
    - fao_ 6y ago
      Exactly what I was talking about, thank you
- saagarjha 6y ago
  The behavior your are likely thinking of (execvp on a file without a shebang) is implemented in libc.
  - fao_ 6y ago
    No. As someone else mentioned, the kernel hardcodes /bin/sh as a last resort: https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f3d58a21b01fdb14e749/init/main.c#L1476 https://github.com/torvalds/linux/blob/dcc0b49040c70ad827a7f...

lpapez 6y ago

As explained by the author https://justine.lol/ape.html https://justine.lol/ape.html

Dylan16807 6y ago
In short: The start of the file is MZ but also a shell script. If you run it on linux it replaces its header with the ELF header, then execs itself.

im3w1l 6y ago

And the reason Windows requires MZ is so that the program is also a valid dos program (that prints an error and exits). According to wikipedia MZ were picked to be the initials of one of the leading dos developers. Turtles all the way down.