4 ms·

There are no technical reasons. The issue is that the kernel and glibc are not one coherent whole because they were always developed by two unrelated groups. gl

There are no technical reasons. The issue is that the kernel and glibc are not one coherent whole because they were always developed by two unrelated groups. glibc wasn't even designed for Linux, but for the GNU kernel. The end result is that the users of GNU/Linux will always draw the short end of the stick.

jancsika 8y ago
> The issue is that the kernel and glibc are not one coherent whole because they were always developed by two unrelated groups. But AFAICT the glibc dogma is based on the premise that it would be impossible for a large, complex project to have backward compatibility without making regular changes to the extant interfaces that it provides. Given that premise glibc devs seem to have some process for figuring out what "correctness" means for time=now and then noodle around with their interface to reflect that correctness in the next version of the lib. Thus symbol versioning is employed. At the same time, Linux is a large, complex project with backward compatibility which does significantly less noodling around with the extant interface. AFAICT the process consists mainly of a) devs breaking the extant interface for correctness, b) a user submitting a bug, and c) the lead dev surrounding the declarative sentence "We don't break userspace" with imperative sentences containing curses and then rejecting the change. I've read where Linus and others have tried to defend their choice and argued that the glibc dev process is worse. Regardless of the persuasiveness of that argument, I've read it and am familiar with it. I am not familiar with the glibc argument as to why they require regular interface changes, nor an acknowledgement that a closely connected large complex project gets by without that. I don't see anything on the glibc FAQ about it-- only a question about symbol versioning where the answer assumes that the interface must change.
- cesarb 8y ago
  > At the same time, Linux is a large, complex project with backward compatibility which does significantly less noodling around with the extant interface. Take a look at the system call table: #define __NR_oldstat 18 #define __NR_oldfstat 28 #define __NR_oldolduname 59 #define __NR_oldlstat 84 #define __NR_olduname 109 [...] #define __NR_dup3 330 #define __NR_pipe2 331 #define __NR_preadv2 378 #define __NR_pwritev2 379 (And that's before the impending Y2038 changes to the API.) The main difference is that, instead of defining a new symbol, a new system call number is defined. The effect is similar: a program using the new "stat" system call (106) won't work on an older kernel which doesn't have it, while on the opposite direction it still works (new kernels still understand the old system call). One thing the kernel developers do nowadays to reduce the API churn is to add a flags argument to every new system call (for instance, the "dup3" above is the same as "dup2", but with a flags argument). Even then, if you try to use a flag which the current kernel doesn't know, it won't work (the kernel developers learned the hard way that you can't ignore unknown flags, since programs will pass them and then break on newer kernels). And that's without considering the "escape hatches" of ioctl() and fcntl(), or the virtual filesystems like /proc and /sys, which are also part of the Linux kernel API. So yes, the Linux kernel does see regular interface changes.
  - jancsika 8y ago
    Still trying to get my bearings, so bear with me... Here are two different types of backward compatibility: 1. will old binary work with the new version? 2. will old code build and run correctly with the new version? So when I talk about extant interface changes, I'm speculating that old code that leverages the public Linux interface is more likely to work and work correctly vs. old code that leverages the public glibc interface. For example: suppose foodev built a Linux driver for a very popular piece of hardware in 2003, abandoned it, and in 2018 there are problems getting it to run correctly. Are those problems more likely due to Linux public interface churn or glibc public interface churn?
    - cesarb 8y ago
      For a driver, there would be problems getting it to run correctly in 2004 already, since the internal Linux kernel API used by drivers is not stable and changes very rapidly. Unless the driver has been upstreamed, because developers updating the kernel internal APIs also update all the in-tree drivers at the same time. About leveraging one interface or the other: when you use the glibc interface, you are also using the Linux interface behind it, so a change in either can affect your program. On the other hand, if you are using the Linux interface directly instead of going through the C library, chances are you are doing something unusual, which increases the risk of it breaking by accident. And there are some things which exist only on the glibc interface, like nameserver lookups (getaddrinfo), user database lookups (getpwnam/getpwuid), and many more.
      - Annatar 8y ago
        "For a driver, there would be problems getting it to run correctly in 2004 already," This is unthinkable on Solaris / illumos kernels because of DDI / DDK interfaces: I can take a driver from 1993 for Solaris 2.5.1 and modload(1M) it into the latest nightly illumos build and I'm guaranteed that it will work.
- Annatar 8y ago
  But AFAICT the glibc dogma is based on the premise that it would be impossible for a large, complex project to have backward compatibility without making regular changes to the extant interfaces that it provides. Yeah well tell that to the engineers of HP-UX, IRIX, and Solaris, because all of those managed to produce libc's which were backward compatible. Sun Microsystems even legally warranted Solaris and therefore libc, they were that paranoid about backwards-compatibility. That's not the issue. The issue is that glibc is developed by people who are not and never were system engineers and instead of learning from the masters, asking them how to do it correctly, or sticking with BSD when its situation was dire, they just decided to re-invent the wheel. One does not simply re-invent glibc from first principles, especially so if one does not have the requisite insights and experience, which they didn't and they still don't, and most likely if they haven't by now, never will. GNU developers are a lost cause. Just look at how long it took them to "discover" versioned interfaces with linker map files, something Solaris system engineers have been using since the early '90's of the past century, and everything becomes crystal clear, if one knows the Red flags. That's one Red flag right there, "late in phase and unlikely to ever catch up".

jancsika 8y ago

> The issue is that the kernel and glibc are not one coherent whole because they were always developed by two unrelated groups. But AFAICT the glibc dogma is based on the premise that it would be impossible for a large, complex project to have backward compatibility without making regular changes to the extant interfaces that it provides. Given that premise glibc devs seem to have some process for figuring out what "correctness" means for time=now and then noodle around with their interface to reflect that correctness in the next version of the lib. Thus symbol versioning is employed. At the same time, Linux is a large, complex project with backward compatibility which does significantly less noodling around with the extant interface. AFAICT the process consists mainly of a) devs breaking the extant interface for correctness, b) a user submitting a bug, and c) the lead dev surrounding the declarative sentence "We don't break userspace" with imperative sentences containing curses and then rejecting the change. I've read where Linus and others have tried to defend their choice and argued that the glibc dev process is worse. Regardless of the persuasiveness of that argument, I've read it and am familiar with it. I am not familiar with the glibc argument as to why they require regular interface changes, nor an acknowledgement that a closely connected large complex project gets by without that. I don't see anything on the glibc FAQ about it-- only a question about symbol versioning where the answer assumes that the interface must change.

cesarb 8y ago
> At the same time, Linux is a large, complex project with backward compatibility which does significantly less noodling around with the extant interface. Take a look at the system call table: #define __NR_oldstat 18 #define __NR_oldfstat 28 #define __NR_oldolduname 59 #define __NR_oldlstat 84 #define __NR_olduname 109 [...] #define __NR_dup3 330 #define __NR_pipe2 331 #define __NR_preadv2 378 #define __NR_pwritev2 379 (And that's before the impending Y2038 changes to the API.) The main difference is that, instead of defining a new symbol, a new system call number is defined. The effect is similar: a program using the new "stat" system call (106) won't work on an older kernel which doesn't have it, while on the opposite direction it still works (new kernels still understand the old system call). One thing the kernel developers do nowadays to reduce the API churn is to add a flags argument to every new system call (for instance, the "dup3" above is the same as "dup2", but with a flags argument). Even then, if you try to use a flag which the current kernel doesn't know, it won't work (the kernel developers learned the hard way that you can't ignore unknown flags, since programs will pass them and then break on newer kernels). And that's without considering the "escape hatches" of ioctl() and fcntl(), or the virtual filesystems like /proc and /sys, which are also part of the Linux kernel API. So yes, the Linux kernel does see regular interface changes.
- jancsika 8y ago
  Still trying to get my bearings, so bear with me... Here are two different types of backward compatibility: 1. will old binary work with the new version? 2. will old code build and run correctly with the new version? So when I talk about extant interface changes, I'm speculating that old code that leverages the public Linux interface is more likely to work and work correctly vs. old code that leverages the public glibc interface. For example: suppose foodev built a Linux driver for a very popular piece of hardware in 2003, abandoned it, and in 2018 there are problems getting it to run correctly. Are those problems more likely due to Linux public interface churn or glibc public interface churn?
  - cesarb 8y ago
    For a driver, there would be problems getting it to run correctly in 2004 already, since the internal Linux kernel API used by drivers is not stable and changes very rapidly. Unless the driver has been upstreamed, because developers updating the kernel internal APIs also update all the in-tree drivers at the same time. About leveraging one interface or the other: when you use the glibc interface, you are also using the Linux interface behind it, so a change in either can affect your program. On the other hand, if you are using the Linux interface directly instead of going through the C library, chances are you are doing something unusual, which increases the risk of it breaking by accident. And there are some things which exist only on the glibc interface, like nameserver lookups (getaddrinfo), user database lookups (getpwnam/getpwuid), and many more.
    - Annatar 8y ago
      "For a driver, there would be problems getting it to run correctly in 2004 already," This is unthinkable on Solaris / illumos kernels because of DDI / DDK interfaces: I can take a driver from 1993 for Solaris 2.5.1 and modload(1M) it into the latest nightly illumos build and I'm guaranteed that it will work.
Annatar 8y ago
But AFAICT the glibc dogma is based on the premise that it would be impossible for a large, complex project to have backward compatibility without making regular changes to the extant interfaces that it provides. Yeah well tell that to the engineers of HP-UX, IRIX, and Solaris, because all of those managed to produce libc's which were backward compatible. Sun Microsystems even legally warranted Solaris and therefore libc, they were that paranoid about backwards-compatibility. That's not the issue. The issue is that glibc is developed by people who are not and never were system engineers and instead of learning from the masters, asking them how to do it correctly, or sticking with BSD when its situation was dire, they just decided to re-invent the wheel. One does not simply re-invent glibc from first principles, especially so if one does not have the requisite insights and experience, which they didn't and they still don't, and most likely if they haven't by now, never will. GNU developers are a lost cause. Just look at how long it took them to "discover" versioned interfaces with linker map files, something Solaris system engineers have been using since the early '90's of the past century, and everything becomes crystal clear, if one knows the Red flags. That's one Red flag right there, "late in phase and unlikely to ever catch up".