6 ms·

The author appears to have conflated "Linux" with the "kernel" - for example, "When calling a Linux service the function number goes in X16 rather than X8.", "X

The author appears to have conflated "Linux" with the "kernel" - for example, "When calling a Linux service the function number goes in X16 rather than X8.", "X16 - linux function number", "Call linux to output the string", etc. in the macOS assembly code. Linux refers specifically to the operating system kernel that's used on Linux systems; the macOS kernel is called Mach. Technically, the non-negative-numbered system calls on macOS are those derived from BSD; macOS additionally has another (negative-numbered) set of system calls for its Mach microkernel.

athrun 6y ago
Yes I think he's getting some of terminology wrong. Saying that "Linux [is] based on Unix" is not really accurate either. But ultimately this is just nitpicking. It's great that he's sharing what he learned.
- derefr 6y ago
  > Saying that "Linux [is] based on Unix" is not really accurate either. The context of the author's statement was syscall ABIs. And Linux's original (x86) syscall ABI is based on [a snapshot of] the syscall ABI from a Unix (4.2BSD, I think.) Loosely based, mind you — 4.2BSD wasn't targeting a 32-bit architecture, let alone x86, so the registers et al weren't the same. But the syscall numbers match up, and the number and order of registers used have direct parallels. Compare and contrast: • FreeBSD ABI (direct descendant of 4.2BSD): https://github.com/freebsd/freebsd-src/blob/master/sys/kern/syscalls.master https://github.com/freebsd/freebsd-src/blob/master/sys/kern/... • Linux x86 ABI: https://chromium.googlesource.com/chromiumos/docs/+/master/constants/syscalls.md#x86-32_bit https://chromium.googlesource.com/chromiumos/docs/+/master/c... Everything lines up until you get to the OS-proprietary stuff.
  - spijdar 6y ago
    So, a few things. 4.2BSD was targeting VAX, which is most assuredly a 32 bit machine. Further, the common denominator you're seeing goes back much, much further than BSD itself. Behold! "Version 1" of UNIX, written in just 14 files of PDP-11 assembly. And if you look at u1.s, you'll see the sysent routine which handles syscalls. The numbering should be familiar :-) https://github.com/dspinellis/unix-history-repo/blob/Research-V1-Snapshot-Development/u1.s#L35 https://github.com/dspinellis/unix-history-repo/blob/Researc... If you go to 4.2BSD, you'll see many of these syscalls labeled as "old", e.g. "old pause", "old wait", "old break". https://github.com/dspinellis/unix-history-repo/blob/BSD-4_2-Snapshot-Development/usr/src/sys/sys/syscalls.c#L6 https://github.com/dspinellis/unix-history-repo/blob/BSD-4_2... You can go a little further back to the original PDP7 assembly, but I don't believe the "kernel" actually ran on a separate "process" at all, and the userland simply linked into kernel symbols and called "read", "readdir" etc directly, hence why even later unix documentation tends to call these "routines" or "library calls".
    - derefr 6y ago
      > Further, the common denominator you're seeing goes back much, much further than BSD itself. Interesting, I didn’t know that! There’s definitely an essay to be written here about the evolution of this “Unix system-call convention” over the decades, going into how this table of calls survived each transition and port mostly-unscathed. (Given that we’re throwing actual source syscall tables back and forth as proofs, there’s definitely some narrativizing to be done here, The Old New Thing-style.) I assume the syscalls weren’t retained in ports with the goal of “binary compatibility”, given that these descendant Unix ports were on different architectures that couldn’t literally exec(2) binaries from their ancestor. Guesses: • Toolchain compatibility? • Some shared cross-compiling assembler/linker that nevertheless had hard-coded syscalls? • (The perhaps never-achieved-in-practice goal of) emulation-assisted descendant cross-compatibility, ala z/OS? • The existence of hybrid/transitional minicomputer generations, that had application processors for both the old and new architectures (or application processors that could execute on both ISAs!), such that at least some of the systems being ported to could exec(2) the ancestor’s binaries straight from tape? • Or just the expectation that, despite Unix and C being so intertwined, there were still enough people writing ASM for these machines — using a non-macro assembler — who had developed reflex-memory for the existing syscall numbers, that it would be a bad idea to change the table out from under them? > the userland simply linked into kernel symbols and called "read", "readdir" etc directly So basically, the original Unix was a DOS, rather than a kernel/supervisor. Was that just because the PDP7 didn’t have virtual memory management, or was it a conscious design decision that was later reversed?
      - spijdar 6y ago
        Well first of all I was wrong -- the PDP7 did have syscalls, I'm just bad at reading PDP7 assembly and missed the dispatcher. Curiously, it looks like the sequence is entirely different, although there could be some magic that makes the order different than it appears at first glance. https://github.com/DoctorWkt/pdp7-unix/blob/master/src/sys/s1.s#L137 https://github.com/DoctorWkt/pdp7-unix/blob/master/src/sys/s... It's all just guessing, but I figure the explanation is much simpler -- for PDP11 UNIX, they just kept using the same syscalls up till V7 / 2BSD, and there should have been a sort of "rolling release" binary compatibility. For the VAX, the first port (32v) probably just retained the original numbering since there was no reason to deviate from it, which colored 3BSD and 4BSD, hence {Net,Free,Open}BSD and Darwin and friends. Worth pointing out that several versions of Linux have rather different syscall tables. 32 bit ARM and x86 are more-or-less matches, with ARM differing on a few early syscalls, while 64 bit ARM and amd64 differing quite dramatically. The old ABI for 32bit MIPS also matches, but both the n32 and n64 ABIs use slightly variant syscall tables. PowerPC 32/64 bit is also a close match, although it has some impedance (I think it matches closer to AIX by design) At the end of the day, I think the similarity is mostly a mixture of coincidence, system developers being influenced by their bootstrap system's syscall tables, and no real reason to change them up. No reason to not change them, either, since it's pretty trivial to use different dispatch tables for different types of processes, like how the BSD's handle other-OS compat.
- Blikkentrekker 6y ago
  The term is so often conflated because it seems as though what is wanted be a technical definition of a concept that is not technical, but social. “Linux” the kernel is technical, but that other thing for which “Linux” is often used has no technical barriers to it, and is entirely a social thing — something is “Linux” when it declares itself part of the tribe, and is so accepted. SteamOS as a marketing strategy always used the phrase “Linux and SteamOS” and was thus successful in moving itself outside of the tribe, for example. There is no technical definition of “an operating system” and there are no technical reasons for what “operating systems" are and aren't “Linux”; it is purely based on social cohesion.
userbinator 6y ago
Nothing a little s/Linux/macOS/ won't fix... but I sure hope the book he is promoting is less confusing than that. To his credit, he does say "in MacOS it's X, in Linux it's Y". (I'm a long-time x86 programmer, who has done a very little of ARM but finds it boring since you can't beat a compiler as easily as you can on x86... and yet instinctively felt unease at moving 0 into a register. ;-)
- nneonneo 6y ago
  Yeah, there's a bunch of stuff in AArch64 that takes getting used to - for example, fixed-length instructions (everything is a 32-bit instruction!), tons of sanely-named registers (X0~X30, W0~W30), shorter literals (all literals need to fit in a 32-bit instruction), and so much more. Plus, it's a total break from AArch32 - I'm pretty comfortable with 32-bit ARM code, and yet 64-bit ARM code still looks pretty unusual to me.
  - lovelyviking 6y ago
    > I'm pretty comfortable with 32-bit ARM Could you give a direction to a good place for learning it ? Where is better to start? I wish to write hello world without linker or assembler for linux.
    - saagarjha 6y ago
      Try https://azeria-labs.com/writing-arm-assembly-part-1/ https://azeria-labs.com/writing-arm-assembly-part-1/.
    - madog 6y ago
      I found Azeria Labs [0] to be a pretty good introduction for learning ARM assembly basics. Although the main focus is ARM exploitation, it's still a good primer [0]: https://azeria-labs.com/writing-arm-assembly-part-1/ https://azeria-labs.com/writing-arm-assembly-part-1/
- lovelyviking 6y ago
  What do you mean by “beat the compiler” ? like running code without it nor linker ? Looks like I have a mood to try that now
bananabreakfast 6y ago
Small nitpick, but the macOS kernel is actually called XNU which is a hybrid of Mach and FreeBSD.
- reaperducer 6y ago
  the macOS kernel is actually called XNU Since you seem knowledgeable about these things, how does one pronounce "XNU?" Is it "ex-en-you" or "zznoo" or "she-no" or something else?
  - glhaynes 6y ago
    Fairly certain I've heard it as "ex-noo". But it could just be that I've said it in my head that way forever!
  - joshspankit 6y ago
    ZA-na-doo
deleted 6y ago
[deleted]
coldtea 6y ago
>The author appears to have conflated "Linux" with the "kernel" I don't think he conflated it (in his mind). In the post he's quite clear which is which. He probably just reused an ARM Linux example code (probably from his book), made some changes required for macOS ARM, but forgot to change the name of the kernel.
raxxorrax 6y ago
Not yet given the M1 or MacOS a spin, but the syscall adresses should be available under /sys/syscall.h according to an answer on stack exchange. Cannot find a table on the net and syscall adresses could be subject to changes.
- saagarjha 6y ago
  https://opensource.apple.com/source/xnu/xnu-7195.50.7.100.1/bsd/kern/syscalls.master.auto.html https://opensource.apple.com/source/xnu/xnu-7195.50.7.100.1/...

athrun 6y ago

Yes I think he's getting some of terminology wrong. Saying that "Linux [is] based on Unix" is not really accurate either. But ultimately this is just nitpicking. It's great that he's sharing what he learned.

derefr 6y ago
> Saying that "Linux [is] based on Unix" is not really accurate either. The context of the author's statement was syscall ABIs. And Linux's original (x86) syscall ABI is based on [a snapshot of] the syscall ABI from a Unix (4.2BSD, I think.) Loosely based, mind you — 4.2BSD wasn't targeting a 32-bit architecture, let alone x86, so the registers et al weren't the same. But the syscall numbers match up, and the number and order of registers used have direct parallels. Compare and contrast: • FreeBSD ABI (direct descendant of 4.2BSD): https://github.com/freebsd/freebsd-src/blob/master/sys/kern/syscalls.master https://github.com/freebsd/freebsd-src/blob/master/sys/kern/... • Linux x86 ABI: https://chromium.googlesource.com/chromiumos/docs/+/master/constants/syscalls.md#x86-32_bit https://chromium.googlesource.com/chromiumos/docs/+/master/c... Everything lines up until you get to the OS-proprietary stuff.
- spijdar 6y ago
  So, a few things. 4.2BSD was targeting VAX, which is most assuredly a 32 bit machine. Further, the common denominator you're seeing goes back much, much further than BSD itself. Behold! "Version 1" of UNIX, written in just 14 files of PDP-11 assembly. And if you look at u1.s, you'll see the sysent routine which handles syscalls. The numbering should be familiar :-) https://github.com/dspinellis/unix-history-repo/blob/Research-V1-Snapshot-Development/u1.s#L35 https://github.com/dspinellis/unix-history-repo/blob/Researc... If you go to 4.2BSD, you'll see many of these syscalls labeled as "old", e.g. "old pause", "old wait", "old break". https://github.com/dspinellis/unix-history-repo/blob/BSD-4_2-Snapshot-Development/usr/src/sys/sys/syscalls.c#L6 https://github.com/dspinellis/unix-history-repo/blob/BSD-4_2... You can go a little further back to the original PDP7 assembly, but I don't believe the "kernel" actually ran on a separate "process" at all, and the userland simply linked into kernel symbols and called "read", "readdir" etc directly, hence why even later unix documentation tends to call these "routines" or "library calls".
  - derefr 6y ago
    > Further, the common denominator you're seeing goes back much, much further than BSD itself. Interesting, I didn’t know that! There’s definitely an essay to be written here about the evolution of this “Unix system-call convention” over the decades, going into how this table of calls survived each transition and port mostly-unscathed. (Given that we’re throwing actual source syscall tables back and forth as proofs, there’s definitely some narrativizing to be done here, The Old New Thing-style.) I assume the syscalls weren’t retained in ports with the goal of “binary compatibility”, given that these descendant Unix ports were on different architectures that couldn’t literally exec(2) binaries from their ancestor. Guesses: • Toolchain compatibility? • Some shared cross-compiling assembler/linker that nevertheless had hard-coded syscalls? • (The perhaps never-achieved-in-practice goal of) emulation-assisted descendant cross-compatibility, ala z/OS? • The existence of hybrid/transitional minicomputer generations, that had application processors for both the old and new architectures (or application processors that could execute on both ISAs!), such that at least some of the systems being ported to could exec(2) the ancestor’s binaries straight from tape? • Or just the expectation that, despite Unix and C being so intertwined, there were still enough people writing ASM for these machines — using a non-macro assembler — who had developed reflex-memory for the existing syscall numbers, that it would be a bad idea to change the table out from under them? > the userland simply linked into kernel symbols and called "read", "readdir" etc directly So basically, the original Unix was a DOS, rather than a kernel/supervisor. Was that just because the PDP7 didn’t have virtual memory management, or was it a conscious design decision that was later reversed?
    - spijdar 6y ago
      Well first of all I was wrong -- the PDP7 did have syscalls, I'm just bad at reading PDP7 assembly and missed the dispatcher. Curiously, it looks like the sequence is entirely different, although there could be some magic that makes the order different than it appears at first glance. https://github.com/DoctorWkt/pdp7-unix/blob/master/src/sys/s1.s#L137 https://github.com/DoctorWkt/pdp7-unix/blob/master/src/sys/s... It's all just guessing, but I figure the explanation is much simpler -- for PDP11 UNIX, they just kept using the same syscalls up till V7 / 2BSD, and there should have been a sort of "rolling release" binary compatibility. For the VAX, the first port (32v) probably just retained the original numbering since there was no reason to deviate from it, which colored 3BSD and 4BSD, hence {Net,Free,Open}BSD and Darwin and friends. Worth pointing out that several versions of Linux have rather different syscall tables. 32 bit ARM and x86 are more-or-less matches, with ARM differing on a few early syscalls, while 64 bit ARM and amd64 differing quite dramatically. The old ABI for 32bit MIPS also matches, but both the n32 and n64 ABIs use slightly variant syscall tables. PowerPC 32/64 bit is also a close match, although it has some impedance (I think it matches closer to AIX by design) At the end of the day, I think the similarity is mostly a mixture of coincidence, system developers being influenced by their bootstrap system's syscall tables, and no real reason to change them up. No reason to not change them, either, since it's pretty trivial to use different dispatch tables for different types of processes, like how the BSD's handle other-OS compat.
Blikkentrekker 6y ago
The term is so often conflated because it seems as though what is wanted be a technical definition of a concept that is not technical, but social. “Linux” the kernel is technical, but that other thing for which “Linux” is often used has no technical barriers to it, and is entirely a social thing — something is “Linux” when it declares itself part of the tribe, and is so accepted. SteamOS as a marketing strategy always used the phrase “Linux and SteamOS” and was thus successful in moving itself outside of the tribe, for example. There is no technical definition of “an operating system” and there are no technical reasons for what “operating systems" are and aren't “Linux”; it is purely based on social cohesion.

userbinator 6y ago

Nothing a little s/Linux/macOS/ won't fix... but I sure hope the book he is promoting is less confusing than that. To his credit, he does say "in MacOS it's X, in Linux it's Y". (I'm a long-time x86 programmer, who has done a very little of ARM but finds it boring since you can't beat a compiler as easily as you can on x86... and yet instinctively felt unease at moving 0 into a register. ;-)

nneonneo 6y ago
Yeah, there's a bunch of stuff in AArch64 that takes getting used to - for example, fixed-length instructions (everything is a 32-bit instruction!), tons of sanely-named registers (X0~X30, W0~W30), shorter literals (all literals need to fit in a 32-bit instruction), and so much more. Plus, it's a total break from AArch32 - I'm pretty comfortable with 32-bit ARM code, and yet 64-bit ARM code still looks pretty unusual to me.
- lovelyviking 6y ago
  > I'm pretty comfortable with 32-bit ARM Could you give a direction to a good place for learning it ? Where is better to start? I wish to write hello world without linker or assembler for linux.
  - saagarjha 6y ago
    Try https://azeria-labs.com/writing-arm-assembly-part-1/ https://azeria-labs.com/writing-arm-assembly-part-1/.
  - madog 6y ago
    I found Azeria Labs [0] to be a pretty good introduction for learning ARM assembly basics. Although the main focus is ARM exploitation, it's still a good primer [0]: https://azeria-labs.com/writing-arm-assembly-part-1/ https://azeria-labs.com/writing-arm-assembly-part-1/
lovelyviking 6y ago
What do you mean by “beat the compiler” ? like running code without it nor linker ? Looks like I have a mood to try that now

bananabreakfast 6y ago

Small nitpick, but the macOS kernel is actually called XNU which is a hybrid of Mach and FreeBSD.

reaperducer 6y ago
the macOS kernel is actually called XNU Since you seem knowledgeable about these things, how does one pronounce "XNU?" Is it "ex-en-you" or "zznoo" or "she-no" or something else?
- glhaynes 6y ago
  Fairly certain I've heard it as "ex-noo". But it could just be that I've said it in my head that way forever!
- joshspankit 6y ago
  ZA-na-doo

deleted 6y ago

[deleted]

coldtea 6y ago

>The author appears to have conflated "Linux" with the "kernel" I don't think he conflated it (in his mind). In the post he's quite clear which is which. He probably just reused an ARM Linux example code (probably from his book), made some changes required for macOS ARM, but forgot to change the name of the kernel.

raxxorrax 6y ago

Not yet given the M1 or MacOS a spin, but the syscall adresses should be available under /sys/syscall.h according to an answer on stack exchange. Cannot find a table on the net and syscall adresses could be subject to changes.

saagarjha 6y ago
https://opensource.apple.com/source/xnu/xnu-7195.50.7.100.1/bsd/kern/syscalls.master.auto.html https://opensource.apple.com/source/xnu/xnu-7195.50.7.100.1/...