4 ms·

The Unix API is more than just system calls or Posix (2018)

77 points by goranmoomin 4y ago

zik 4y ago
I can see his point but it seems to rely on redefining what the term "API" means. You could as well say "The Windows API isn't just .NET - it's Win32 and a whole lot of other things". Which is kind of true, but we don't usually call all of those things an API. We call some of those individual parts APIs, just like we do with UNIX. And other parts we'd call filesystem conventions and so on. I think he's arguing that you need to know all of those things to program with the OS and he's right, but that's still not how we conventionally use the word "API". It's more of a "standard" and the POSIX standard actually does define a lot of this.
- jart 4y ago
  The API is the tip of the topological order of interfaces provided by the platform. For example, .NET would NOT be Windows' API because .NET depends on WIN32 (or even more appropriately, NTDLL) which makes .NET important but non-essential. It doesn't make sense to define "the API" as including all the layers of turtles that wrap the canonical one. Just like we wouldn't say that the UNIX userspace utilities like cp/mv/ls are "THE unix API" because those tools all depend on the system calls. On UNIX the system call interface is THE api for the UNIX operating system, and it IS what people commonly refer to as the C API. It's just a more formal definition of the C API and it is defined by the Open Group. It's abstract and language agnostic because it's provided by well-understood and widely encoded binary interfaces which have been defined to nearly exactly model the abstract so-called C definition. Therefore when we say "C API" we aren't necessarily including things like all the manipulations possible in an implementation intended only for a single language like C, e.g. glibc header files or a thunk that irons out some weird piece of legacy assembly technical debt like 386BSD's affinity for the carry flag. Because unless kernels are willing to implement the same degree of SYSCALL instruction fascism that Microsoft implements with NT then no UNIX vendor really has the right to make that claim that the ordinals they copied from the System V codebase. So people who think that "system Libc" and "system API" are the same thing are generally missing the forest for the trees. Dynamic shared objects are stupid and they aren't systems. A libc is just a tool for interfacing with real system apis.
Karellen 4y ago
> As people have found out in the past, the real Unix API is the whole collection of behaviors and environments that Unix programs assume. It isn't just POSIX library calls; it's also the shell and standard utilities and files that are in known locations and standard capabilities and various other things. But... "the shell and standard utilities and files that are in known locations and standard capabilities and various other things" are also specified by POSIX. (Historically, they were the "POSIX.2" standard, and the libc/syscall library functions were in the "POSIX.1" standard, but since 1998 when POSIX became the ratification of Single Unix Specification version 2-and-later, they're all just in "POSIX".) So... yes? > Unless you intend for your program to be narrowly and specifically portable to POSIX or an even more minimal standard, it is not a bug for it to rely on portions of the broader, de facto Unix API. It's not even necessarily a bug to rely on APIs that are only there on some Unixes (for example Linux and FreeBSD), Why would any of those things be considered a "bug"? > for example, can you implement a compatible and fully capable Bourne shell using only public Unix kernel APIs, or at most public C library APIs? What other APIs does the author think that Bourne shell writers used? I feel like I'm missing some broader context for this post.
- fulafel 4y ago
  It's a bit ambiguous, people often mean the 90s POSIX versions when they say POSIX, even though later some rebranding/republishing of SUS happened under the POSIX name. SUS is the less ambiguous term to use I think when talking about the newer stuff.
- faho 4y ago
  >But... "the shell and standard utilities and files that are in known locations and standard capabilities and various other things" are also specified by POSIX. Unfortunately, it doesn't specify those known locations. It explicitly declines specifying that /bin/sh exists. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s... says: > Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH, ensuring that the returned pathname is an absolute pathname and not a shell built-in. and it doesn't specify the location of getconf either, so you have a chicken-and-egg problem where you need the standard $PATH to find getconf to get the standard $PATH.
kazinator 4y ago
> It isn't just POSIX library calls; it's also the shell and standard utilities and files that are in known locations and standard capabilities and various other things. POSIX has standardized the shell and utilities since before 1990. Of course, not all of them. As far as locations go, you have to rely on PATH; systems do not agree on what exactly is in /bin, /usr/bin and /sbin. > A 'Unix' without a useful $HOME environment variable and /tmp may be specification compliant (I haven't checked POSIX) but it's not useful, in that many programs that people want generally won't run on it. Hits for the HOME variable in POSIX: https://pubs.opengroup.org/cgi/susv4search.cgi?KEYWORDS=HOME&SUBSTRING=substring&CONTEXT= https://pubs.opengroup.org/cgi/susv4search.cgi?KEYWORDS=HOME... Under the Shell Command Language it is documented that HOME is "[t]he pathname of the user's home directory. The contents of HOME are used in tilde expansion (see Tilde Expansion)." /tmp is a de facto standard. POSIX encourages applications to check for the existence of a variable called TMPDIR, but doesn't require implementations to provide that. There are functions which abstract creating a temporary file or directory. Some of them need a template, some don't. It is possible to write a program which uses /tmp for temporary files, yet doesn't refer to "/tmp".
- knorker 4y ago
  Interesting that this person would write a post about how much is supposedly not in POSIX, but at the same time saying "I haven't checked POSIX". Rings of "did you know that 'gullible' is not in the dictionary? It's true, look for yourself!".
sylware 4y ago
If we could finally semantically split the signals as those which can be asynchronoulsy delivered and those which sort of cannot (segfault)... that without extra and bonus kludge.
kodah 4y ago
This is kind of an odd take because POSIX is purposed to make application development across Linux and Unix easier by providing reliable behavior. Saying, "my OS is better than the high level standard because it does more" makes me go, "Okay". Also, I'm not sure the author has ever actually read POSIX?

77 points by goranmoomin 4y ago

zik 4y ago

I can see his point but it seems to rely on redefining what the term "API" means. You could as well say "The Windows API isn't just .NET - it's Win32 and a whole lot of other things". Which is kind of true, but we don't usually call all of those things an API. We call some of those individual parts APIs, just like we do with UNIX. And other parts we'd call filesystem conventions and so on. I think he's arguing that you need to know all of those things to program with the OS and he's right, but that's still not how we conventionally use the word "API". It's more of a "standard" and the POSIX standard actually does define a lot of this.

jart 4y ago
The API is the tip of the topological order of interfaces provided by the platform. For example, .NET would NOT be Windows' API because .NET depends on WIN32 (or even more appropriately, NTDLL) which makes .NET important but non-essential. It doesn't make sense to define "the API" as including all the layers of turtles that wrap the canonical one. Just like we wouldn't say that the UNIX userspace utilities like cp/mv/ls are "THE unix API" because those tools all depend on the system calls. On UNIX the system call interface is THE api for the UNIX operating system, and it IS what people commonly refer to as the C API. It's just a more formal definition of the C API and it is defined by the Open Group. It's abstract and language agnostic because it's provided by well-understood and widely encoded binary interfaces which have been defined to nearly exactly model the abstract so-called C definition. Therefore when we say "C API" we aren't necessarily including things like all the manipulations possible in an implementation intended only for a single language like C, e.g. glibc header files or a thunk that irons out some weird piece of legacy assembly technical debt like 386BSD's affinity for the carry flag. Because unless kernels are willing to implement the same degree of SYSCALL instruction fascism that Microsoft implements with NT then no UNIX vendor really has the right to make that claim that the ordinals they copied from the System V codebase. So people who think that "system Libc" and "system API" are the same thing are generally missing the forest for the trees. Dynamic shared objects are stupid and they aren't systems. A libc is just a tool for interfacing with real system apis.

Karellen 4y ago

> As people have found out in the past, the real Unix API is the whole collection of behaviors and environments that Unix programs assume. It isn't just POSIX library calls; it's also the shell and standard utilities and files that are in known locations and standard capabilities and various other things. But... "the shell and standard utilities and files that are in known locations and standard capabilities and various other things" are also specified by POSIX. (Historically, they were the "POSIX.2" standard, and the libc/syscall library functions were in the "POSIX.1" standard, but since 1998 when POSIX became the ratification of Single Unix Specification version 2-and-later, they're all just in "POSIX".) So... yes? > Unless you intend for your program to be narrowly and specifically portable to POSIX or an even more minimal standard, it is not a bug for it to rely on portions of the broader, de facto Unix API. It's not even necessarily a bug to rely on APIs that are only there on some Unixes (for example Linux and FreeBSD), Why would any of those things be considered a "bug"? > for example, can you implement a compatible and fully capable Bourne shell using only public Unix kernel APIs, or at most public C library APIs? What other APIs does the author think that Bourne shell writers used? I feel like I'm missing some broader context for this post.

fulafel 4y ago
It's a bit ambiguous, people often mean the 90s POSIX versions when they say POSIX, even though later some rebranding/republishing of SUS happened under the POSIX name. SUS is the less ambiguous term to use I think when talking about the newer stuff.
faho 4y ago
>But... "the shell and standard utilities and files that are in known locations and standard capabilities and various other things" are also specified by POSIX. Unfortunately, it doesn't specify those known locations. It explicitly declines specifying that /bin/sh exists. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s... says: > Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH, ensuring that the returned pathname is an absolute pathname and not a shell built-in. and it doesn't specify the location of getconf either, so you have a chicken-and-egg problem where you need the standard $PATH to find getconf to get the standard $PATH.

kazinator 4y ago

> It isn't just POSIX library calls; it's also the shell and standard utilities and files that are in known locations and standard capabilities and various other things. POSIX has standardized the shell and utilities since before 1990. Of course, not all of them. As far as locations go, you have to rely on PATH; systems do not agree on what exactly is in /bin, /usr/bin and /sbin. > A 'Unix' without a useful $HOME environment variable and /tmp may be specification compliant (I haven't checked POSIX) but it's not useful, in that many programs that people want generally won't run on it. Hits for the HOME variable in POSIX: https://pubs.opengroup.org/cgi/susv4search.cgi?KEYWORDS=HOME&SUBSTRING=substring&CONTEXT= https://pubs.opengroup.org/cgi/susv4search.cgi?KEYWORDS=HOME... Under the Shell Command Language it is documented that HOME is "[t]he pathname of the user's home directory. The contents of HOME are used in tilde expansion (see Tilde Expansion)." /tmp is a de facto standard. POSIX encourages applications to check for the existence of a variable called TMPDIR, but doesn't require implementations to provide that. There are functions which abstract creating a temporary file or directory. Some of them need a template, some don't. It is possible to write a program which uses /tmp for temporary files, yet doesn't refer to "/tmp".

knorker 4y ago
Interesting that this person would write a post about how much is supposedly not in POSIX, but at the same time saying "I haven't checked POSIX". Rings of "did you know that 'gullible' is not in the dictionary? It's true, look for yourself!".

sylware 4y ago

If we could finally semantically split the signals as those which can be asynchronoulsy delivered and those which sort of cannot (segfault)... that without extra and bonus kludge.

kodah 4y ago

This is kind of an odd take because POSIX is purposed to make application development across Linux and Unix easier by providing reliable behavior. Saying, "my OS is better than the high level standard because it does more" makes me go, "Okay". Also, I'm not sure the author has ever actually read POSIX?