5 ms·

The explanation for why it's useful is at the bottom of the page, but it wasn't necessary before glibc got behind and before Linux started adding things that gl

The explanation for why it's useful is at the bottom of the page, but it wasn't necessary before glibc got behind and before Linux started adding things that glibc would probably never implement*. Generally, on UNIX-like systems, the C library isn't just the C library but also exposes the UNIX/POSIX API which is just a set of wrappers around calling into the kernel (done with software interrupt 0x80 on x86 Linux.) * Remember that glibc is part of the GNU project and is meant to be portable beyond Linux, especially to Hurd. It might not make sense for them to implement interfaces that wouldn't work or would require significant work to emulate on Hurd, the BSDs and Solaris.

Joker_vD 5y ago
Well, yeah, merging kernel calls into the runtime of one specifically favoured programming language is a silly idea, why not have the kernel calls in a separate, cleanly defined language-agnostic OS-specific library? "It just the way the things has come to work historically"?
- guerrilla 5y ago
  Well, first of all the UNIX/POSIX API is a C API, so that question doesn't make much sense. On top of that, there have been no competitive systems programming languages until recently (and certainly none that weren't written in C), so that wouldn't have been worth it. Also, it wasn't really an idea per se: C is just the native language and lingua franca of UNIX and that's just how UNIX developed organically.
  - GoblinSlayer 5y ago
    Why would it matter what system programming languages are there? Unix can't run other languages?
    - koprulusector 5y ago
      The kernel is written in C. Ok, why does that matter? Because its interfaces and architecture are tightly intertwined with C. Remember that C was partially developed to use in building Unix. So, kernel was written in C and therefore uses C data structures, pointers, and follows C conventions for using them (for example, kernel might return a struct of pointers). This applies to interfaces and structures, making syscalls (depending on which syscall and what is needed), checking return values or error codes. You don’t NEED this stuff exactly to run on Unix. What and how much you need, or even just convenience, are how it boils down. If you want to use eBPF to trace things from raw network packets, syscalls, file systems, you can do some in Python but you’ll need some degree of C. You want to run a Django or flask app in just Python? Good to go. [This article on Linux ABI](https://en.wikipedia.org/wiki/Application_binary_interface https://en.wikipedia.org/wiki/Application_binary_interface) might help explain better/in more detail. Sorry if this isn’t the best explanation, I’m on mobile and up too late.
      - GoblinSlayer 5y ago
        Design of unix was influenced more by PDP-7 than by C. What unix does, it does PDP-7 stuff, not C stuff. C itself isn't limited to POSIX.
    - johannes1234321 5y ago
      Unix can run other languages. But for answering your question deeper, we have to find an understanding what boundary we're talking about. The kernel syscall interface? Things like Linux's vdso? POSIX library (libc)? The more we go on in that sequence the more C we get into the interfaces and many of those APIs are essential for writing well-behaving non-trivial programs. The closer we stay to the kernel's syscall interface the less C you get, but still the syscall interface and C are related. Strings are often NULL-terminated. Arguemtns are passed (via registers) from left to right, not like in Pascal convention right to left. (First arg always in EBX, second in EXC etc.) A different language will need some interoperability with C conventions for being a good citizen on Unix. A Pascal operating system for instance would use Pascal strings (historically what tis now called ShortString) where length is encoded in the first byte.
      - Joker_vD 5y ago
        > A Pascal operating system for instance would use Pascal strings Only if the designers would make that decision, and they are not in any way obliged to make such decision! Nobody stops them from using e.g. a moral equivalent of "RECORD size: CARDINAL; str_ptr: ^CHAR END" for passing strings in their system calls (which would probably coincide with using "size_t size, char *str_ptr" in the C prototype at the binary level), or something else entirely. Is the idea of an OS not exposing its implementation language in its public API that foreign so I have to express it three times in this thread and every time be replied with "No, what are you talking about, of course the OS would use reuse the structures of the runtime of the language that it is written in in its public syscall API"? Gee, I wonder how the syscall calling convention would look if I wrote an OS entirely in assembly.
        johannes1234321 5y ago
        It's not what the implementation uses, but "which conventions does the call interface assume" you can use Pascal to create an operating system with a system interface that benefits Fortran. (I know too little about Fortran to describe what that would look like) With C and Unix the issue of course is that both were developed around the same time by the same people for working together.
  - hnlmorg 5y ago
    > On top of that, there have been no competing systems programming languages until very recently There have been plenty. What has changed isn't the lack of competing languages, it's the discontent with C and C++. The Rust hype has coincided with the anti-C and anti-C++ hype and thus the "rewrite it in Rust" movement was born. But Rust wasn't even the first memory safe systems programming language, let alone first alternative to C (and neither was C the first systems programming language itself).
    - guerrilla 5y ago
      I meant competitive and edited my post now to reflect that and I stand by it. I can only speak for myself with certainty but the benefits never outweighed the costs until more recent languages. If there were really alternatives then more things would have been written in them instead of in C or moving to higher-level languages.
      - hnlmorg 5y ago
        > I meant competitive and edited my post now to reflect that and I stand by it. I can only speak for myself with certainty but the benefits never outweighed the costs until more recent languages but if there were really alternatives then more things would have been written in them instead of in C or moving to higher-level languages. You're now conflating "popular" with "better". What makes a language successful isn't always the same as what makes it better (eg there are countless examples of languages failing to gain significant traction (like Zig, Nim, OCaml, D, ) over other languages that had a large company backing it (like Rust did). Moreover, what makes a language "better" isn't going to be the same for everyone. Some crave simplicity so that the entire language can be memorised (like SPARK) whereas others crave complexity where they can chose the subset of features to support (like C++). Some like C-style braces, some like ALGOL style keywords (like Ada), some prefer S-Expressions (like LISP) and some people even prefer to go direct to assembly. We've seen operating systems built in Fortran, Pascal, LISP, Oberon and others so it's not even as if I'm making a theoretical point here.
        guerrilla 5y ago
  - Joker_vD 5y ago
    I would argue that your post is a nice post-rationalization, and is not very accurate historically. The mans in the first versions of UNIX used to show how to do syscalls both in Fortran and C, and only later they've dropped the Fortran because it was too much hassle; and since it was all done way before Ulrich Drepper invented shared libraries, we ended with system calls buried in the C runtime. Also, people wrote complete working OSes in Pascal exactly around the time C was invented, so the part about "no competing systems programming languages" is not true either.
    - guerrilla 5y ago
      > and is not very accurate historically. The mans in the first versions of UNIX used to show how to do syscalls both in Fortran and C I don't know what early versions and manuals you mean because that wasn't in v6 or v7. Was there Fortran on UNIX before F77 was introduced in v7?
      - Joker_vD 5y ago
        Sorry, I was wrong; for some reason I thought it was Fortran but of course it was PDP-11 assembly. It stayed there up until v7, and was removed in v8. That subtracts from my argument.
- hnlmorg 5y ago
  Because every programming language will have their own opinions on how to link to external libraries and what those external libraries should look like (eg Java libraries are ZIP archives with a .jar extension, JIT languages will have their libraries as raw source code). But since C is the de facto standard for OS libraries, most languages end up supporting calls to C libraries. Thus C libraries do become the language-agnostic library.
  - GoblinSlayer 5y ago
    Most languages can call C libraries because they can call any libraries, not because C libraries are language agnostic.
    - couchand 5y ago
      That doesn't ring true to me. My FFI experience is somewhat limited (JS to C, JS to C++ via C wrappers, Rust to C, Rust to C++ via C wrappers) but from what I've seen you have to speak C to make FFI.
      - GoblinSlayer 5y ago
        Few languages truly speak full C (e.g. through clang plugin), they support ABI, but there are libraries like libpng and gtk with API disconnected from ABI, for them FFI doesn't resemble heavily macro encumbered C API.
    - matheusmoreira 5y ago
      Higher level languages such as C++ have extremely complex binary interfaces. Only C++ code dares to link against C++ code. Even then there are problems between different compilers and even different versions of the same compiler. New standard version comes out, it requires that standard classes have certain performance characteristics... Suddenly compilers are breaking binary compatibility in order to restablish conformance. So in order to have usable libraries people will wrap their C++ implementation in C interfaces. Rust faces the same problem. All those nice abstractions result in a complex ABI that will never touched by anything outside the Rust ecosystem. There is no escaping the extern "C". Some languages are virtualized. Java, C#, Ruby, Python, Javascript all have virtual machines. There's no way to interface directly with code running in those languages. We must interface with the virtual machine instead: import it into our process and ask it to call the functions we want to use. C binary interfaces are not language agnostic but they are quite simple. The result of compiling a C library is symbols pointing to functions with simple calling conventions. As a result, virtually any language can look up these symbols and call them.
- deleted 5y ago
  [deleted]

Joker_vD 5y ago

Well, yeah, merging kernel calls into the runtime of one specifically favoured programming language is a silly idea, why not have the kernel calls in a separate, cleanly defined language-agnostic OS-specific library? "It just the way the things has come to work historically"?

guerrilla 5y ago
Well, first of all the UNIX/POSIX API is a C API, so that question doesn't make much sense. On top of that, there have been no competitive systems programming languages until recently (and certainly none that weren't written in C), so that wouldn't have been worth it. Also, it wasn't really an idea per se: C is just the native language and lingua franca of UNIX and that's just how UNIX developed organically.
- GoblinSlayer 5y ago
  Why would it matter what system programming languages are there? Unix can't run other languages?
  - koprulusector 5y ago
    The kernel is written in C. Ok, why does that matter? Because its interfaces and architecture are tightly intertwined with C. Remember that C was partially developed to use in building Unix. So, kernel was written in C and therefore uses C data structures, pointers, and follows C conventions for using them (for example, kernel might return a struct of pointers). This applies to interfaces and structures, making syscalls (depending on which syscall and what is needed), checking return values or error codes. You don’t NEED this stuff exactly to run on Unix. What and how much you need, or even just convenience, are how it boils down. If you want to use eBPF to trace things from raw network packets, syscalls, file systems, you can do some in Python but you’ll need some degree of C. You want to run a Django or flask app in just Python? Good to go. [This article on Linux ABI](https://en.wikipedia.org/wiki/Application_binary_interface https://en.wikipedia.org/wiki/Application_binary_interface) might help explain better/in more detail. Sorry if this isn’t the best explanation, I’m on mobile and up too late.
    - GoblinSlayer 5y ago
      Design of unix was influenced more by PDP-7 than by C. What unix does, it does PDP-7 stuff, not C stuff. C itself isn't limited to POSIX.
  - johannes1234321 5y ago
    Unix can run other languages. But for answering your question deeper, we have to find an understanding what boundary we're talking about. The kernel syscall interface? Things like Linux's vdso? POSIX library (libc)? The more we go on in that sequence the more C we get into the interfaces and many of those APIs are essential for writing well-behaving non-trivial programs. The closer we stay to the kernel's syscall interface the less C you get, but still the syscall interface and C are related. Strings are often NULL-terminated. Arguemtns are passed (via registers) from left to right, not like in Pascal convention right to left. (First arg always in EBX, second in EXC etc.) A different language will need some interoperability with C conventions for being a good citizen on Unix. A Pascal operating system for instance would use Pascal strings (historically what tis now called ShortString) where length is encoded in the first byte.
    - Joker_vD 5y ago
      > A Pascal operating system for instance would use Pascal strings Only if the designers would make that decision, and they are not in any way obliged to make such decision! Nobody stops them from using e.g. a moral equivalent of "RECORD size: CARDINAL; str_ptr: ^CHAR END" for passing strings in their system calls (which would probably coincide with using "size_t size, char *str_ptr" in the C prototype at the binary level), or something else entirely. Is the idea of an OS not exposing its implementation language in its public API that foreign so I have to express it three times in this thread and every time be replied with "No, what are you talking about, of course the OS would use reuse the structures of the runtime of the language that it is written in in its public syscall API"? Gee, I wonder how the syscall calling convention would look if I wrote an OS entirely in assembly.
      - johannes1234321 5y ago
        It's not what the implementation uses, but "which conventions does the call interface assume" you can use Pascal to create an operating system with a system interface that benefits Fortran. (I know too little about Fortran to describe what that would look like) With C and Unix the issue of course is that both were developed around the same time by the same people for working together.
- hnlmorg 5y ago
  > On top of that, there have been no competing systems programming languages until very recently There have been plenty. What has changed isn't the lack of competing languages, it's the discontent with C and C++. The Rust hype has coincided with the anti-C and anti-C++ hype and thus the "rewrite it in Rust" movement was born. But Rust wasn't even the first memory safe systems programming language, let alone first alternative to C (and neither was C the first systems programming language itself).
  - guerrilla 5y ago
    I meant competitive and edited my post now to reflect that and I stand by it. I can only speak for myself with certainty but the benefits never outweighed the costs until more recent languages. If there were really alternatives then more things would have been written in them instead of in C or moving to higher-level languages.
    - hnlmorg 5y ago
      > I meant competitive and edited my post now to reflect that and I stand by it. I can only speak for myself with certainty but the benefits never outweighed the costs until more recent languages but if there were really alternatives then more things would have been written in them instead of in C or moving to higher-level languages. You're now conflating "popular" with "better". What makes a language successful isn't always the same as what makes it better (eg there are countless examples of languages failing to gain significant traction (like Zig, Nim, OCaml, D, ) over other languages that had a large company backing it (like Rust did). Moreover, what makes a language "better" isn't going to be the same for everyone. Some crave simplicity so that the entire language can be memorised (like SPARK) whereas others crave complexity where they can chose the subset of features to support (like C++). Some like C-style braces, some like ALGOL style keywords (like Ada), some prefer S-Expressions (like LISP) and some people even prefer to go direct to assembly. We've seen operating systems built in Fortran, Pascal, LISP, Oberon and others so it's not even as if I'm making a theoretical point here.
      - guerrilla 5y ago
- Joker_vD 5y ago
  I would argue that your post is a nice post-rationalization, and is not very accurate historically. The mans in the first versions of UNIX used to show how to do syscalls both in Fortran and C, and only later they've dropped the Fortran because it was too much hassle; and since it was all done way before Ulrich Drepper invented shared libraries, we ended with system calls buried in the C runtime. Also, people wrote complete working OSes in Pascal exactly around the time C was invented, so the part about "no competing systems programming languages" is not true either.
  - guerrilla 5y ago
    > and is not very accurate historically. The mans in the first versions of UNIX used to show how to do syscalls both in Fortran and C I don't know what early versions and manuals you mean because that wasn't in v6 or v7. Was there Fortran on UNIX before F77 was introduced in v7?
    - Joker_vD 5y ago
      Sorry, I was wrong; for some reason I thought it was Fortran but of course it was PDP-11 assembly. It stayed there up until v7, and was removed in v8. That subtracts from my argument.
hnlmorg 5y ago
Because every programming language will have their own opinions on how to link to external libraries and what those external libraries should look like (eg Java libraries are ZIP archives with a .jar extension, JIT languages will have their libraries as raw source code). But since C is the de facto standard for OS libraries, most languages end up supporting calls to C libraries. Thus C libraries do become the language-agnostic library.
- GoblinSlayer 5y ago
  Most languages can call C libraries because they can call any libraries, not because C libraries are language agnostic.
  - couchand 5y ago
    That doesn't ring true to me. My FFI experience is somewhat limited (JS to C, JS to C++ via C wrappers, Rust to C, Rust to C++ via C wrappers) but from what I've seen you have to speak C to make FFI.
    - GoblinSlayer 5y ago
      Few languages truly speak full C (e.g. through clang plugin), they support ABI, but there are libraries like libpng and gtk with API disconnected from ABI, for them FFI doesn't resemble heavily macro encumbered C API.
  - matheusmoreira 5y ago
    Higher level languages such as C++ have extremely complex binary interfaces. Only C++ code dares to link against C++ code. Even then there are problems between different compilers and even different versions of the same compiler. New standard version comes out, it requires that standard classes have certain performance characteristics... Suddenly compilers are breaking binary compatibility in order to restablish conformance. So in order to have usable libraries people will wrap their C++ implementation in C interfaces. Rust faces the same problem. All those nice abstractions result in a complex ABI that will never touched by anything outside the Rust ecosystem. There is no escaping the extern "C". Some languages are virtualized. Java, C#, Ruby, Python, Javascript all have virtual machines. There's no way to interface directly with code running in those languages. We must interface with the virtual machine instead: import it into our process and ask it to call the functions we want to use. C binary interfaces are not language agnostic but they are quite simple. The result of compiling a C library is symbols pointing to functions with simple calling conventions. As a result, virtually any language can look up these symbols and call them.
deleted 5y ago
[deleted]