7 ms·

This might get me some angry replies, but does the feature set of Linux ever end? Will there ever be a day where someone says "you know what, Linux isn't built

This might get me some angry replies, but does the feature set of Linux ever end? Will there ever be a day where someone says "you know what, Linux isn't built for this and your code adds too much complexity, let's not". It really just seems like Linux has become a dumping ground for anyone to introduce whatever they feel like as long as it passes code review, with nobody looking at the big picture.

trillic 6y ago
So what? This code (and most drivers, modules, additional fucntionality, etc) will only be compiled in if necessary and will only actually be run if you use the feature.
- c7DJTLrn 6y ago
  My concern is not about what my flavor of kernel includes. It's about the size of the codebase, and whether the limited number of maintainers can keep up with the countless features being introduced.
  - coolreader18 6y ago
    I mean I think in terms of maintainer resources Linux is one of the projects with the least to worry about, there are always companies who are willing to have a team work on stuff.
    - Ericson2314 6y ago
      Those megacorps also accumulate endless complexity internally, as does the industry as a whole. It's a problem larger than the discipline of reviewers of C code could ever hope to fix. Even the larger world outside the field is somewhat aware of the the looming complexity problem: https://www.theatlantic.com/technology/archive/2017/09/saving-the-world-from-code/540393/ https://www.theatlantic.com/technology/archive/2017/09/savin... Basically on our current trajectory we will end up with something like the biosphere that can do stuff but too complex to fully understand --- but also probably well more fragile than that?
      - kaba0 6y ago
        That’s what testing is for. We can’t really reason globally about much smaller codebases either Software is complex.
  - cbmuser 6y ago
    The kernel is regularly shedding off unmaintained code. So the codebase is actually not growing as much as it could.
    - c7DJTLrn 6y ago
      The code they're shedding off is drivers. They won't ever get rid of old syscalls because Linus refuses to break userspace, ever.
      - caf 6y ago
        Frequently the old syscall just ends up as a stub that calls into the new implementation though.
pram 6y ago
That’s the beauty of open source! Just fork your own Linux and then refuse any further patches or PRs. True freedom.
dooglius 6y ago
No, because the Linux kernel is designed explicitly as a monolithic kernel, so as long as there is anything anyone wants to do that is "OS-like", it expects to own that.
gpderetta 6y ago
this patch both cleanup existing complexity and add functionality needed for a significant use case.
thedracle 6y ago
I mean, the futex2 proposal has been around for over a year. A lot of the history and discussion can be found here: https://lkml.org/lkml/2021/4/27/1208 https://lkml.org/lkml/2021/4/27/1208 I personally feel like the kernel maintainers have been very conservative over the years regarding the introduction of new system calls, but then again I've maybe gotten used to the trauma of being familiar with windows internals.
fullstop 6y ago
The BSDs are always an option.
- zokula 6y ago
  The BSDs were an option they are becoming more moribund and less used every year.
  - trasz 6y ago
    [citation needed]
bombcar 6y ago
Linus has done exactly that a few times; though usually he defines what would be needed to be merged. Some of the changes aren't compatible with the kernel direction as a whole and so they're a separate branch. Given enough time (and if the change is major AND ends up being widely used) it may get merged in. Those who want a specific kernel with a defined feature set often will look to one of the BSDs as a base (as often they also have reduced hardware requirements, think routers).
yarg 6y ago
As a relative layman, this all seemed reasonable and desirable - and the fact that he integrated it with pthreads seems particularly cool to me. NUMA awareness especially stood out as highly significant. The new supported sizes seems nice and shiny - if well utilised, it could improve performance and capacity in general, capabilities in the low-end and scaling in the high-end.
lizknope 6y ago
Linus Torvalds looks at the big picture all the time. That is basically his job as he doesn't write a lot of low level code anymore. Do you have an example of something that you consider part of the "dumping ground?"
marcodiego 6y ago
Getting a new syscall on linux is hard. You'll often need user-space users before it gets accepted; that alone is hard. The linux syscall set is very small considered the breadth of cases it is used today. Compared to nt, linux syscall set is very small and well thought out.
- formerly_proven 6y ago
  Windows has around 500 syscalls, Linux around 300. Both have several escape-hatches for arbitrary driver-userspace communication, which makes the number of syscalls pretty irrelevant. (E.g. Linux has ioctl, netlink, procfs, sysfs, ...)
  - Ericson2314 6y ago
    > Both have several escape-hatches for arbitrary driver-userspace communication, which makes the number of syscalls pretty irrelevant. (E.g. Linux has ioctl, netlink, procfs, sysfs, ...) Yes, we need a better metric for accounting for those things, which will paint a much more sobering picture of the complexity and growth of complexity of the userland <-> kernel interface.
  - monocasa 6y ago
    Well, there's also win32k.sys which provides an additional 1000 or so syscalls.
  - marcodiego 6y ago
    Compare this: https://linuxhint.com/list_of_linux_syscalls/ https://linuxhint.com/list_of_linux_syscalls/ with https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/ and https://j00ru.vexillium.org/syscalls/win32k/64/ https://j00ru.vexillium.org/syscalls/win32k/64/
- GoblinSlayer 6y ago
  Linux syscall set is very small and posix. No matter what you think, the first version is always a prototype.
rkangel 6y ago
> with nobody looking at the big picture As I understand it, this is exactly what Linus does. He's the final arbiter of what makes it in, and has a history of setting high level direction for how stuff should be (usually prefixed with "I'm rejecting this because").
anoncake 6y ago
What purpose is a general-purpose kernel not built for?
vondur 6y ago
I'm assuming if you build the kernel yourself, you may be able to strip out features you don't want.
- broodbucket 6y ago
  Yep, there's loads and loads of config options. Linux isn't a big immutable blob, there's a reason it gets used in both tiny devices in supercomputers, you can build your kernel however you like. It's not a microkernel but it does a pretty good job of being flexible for a monolithic kernel. If you're curious, clone the kernel source and run `make menuconfig` and you can play with it yourself
Ericson2314 6y ago
This shouldn't be getting down-votes. Consider the slogan "composition not configuration", Linux is squarely the latter. All the hardware, all the different use-cases, that are just added to what's mostly a monolith. You can disable them, but not really pull things a part. Now, Linux also has some of the most stringent code review in the industry, and they are well aware of issues of maintainability, but fundamentally C is not a sufficiently expressive language for them to architect the code base in a compositional manner. What really worries me is that as the totality of the userland users an increasingly large interface to the kernel, it will very hard to ever replace linux withsomething else when we do have something more compositional. I am therefore interested, and somewhat working on, stuff that hopefully would end up with a multi-kernel NixOS (Think Debian GNU/kFreeBSD) to try to get some kernel competition going again, and incentivize some userland portability. Note that I don't think portability <===> just use old system calls. I am no Unix purist. I would really like a kernel that does add new interfaces but also dispenses with the old cruft; think Rust vs C++.
- fabianhjr 6y ago
  > Consider the slogan "composition not configuration", Linux is squarely the latter. All the hardware, all the different use-cases, that are just added to what's mostly a monolith. You can disable them, but not really pull things a part. Linux is a Monolithic Kernel, sounds like you are more interested in Microkernels. https://www.wikiwand.com/en/Microkernel https://www.wikiwand.com/en/Microkernel One microkernel alternative could be Redox-OS: https://www.wikiwand.com/en/Redox_(operating_system) https://www.wikiwand.com/en/Redox_(operating_system)
  - bruckie 6y ago
    You could write a monolithic kernel using composition for your code structure. It would just be composition at build time rather than at runtime. It seems pretty hard to write a microkernel without using composition, though, since the runtime requirements kind of dictate your code structure.
- oalae5niMiel7qu 6y ago
  > but fundamentally C is not a sufficiently expressive language for them to architect the code base in a compositional manner. Plan 9 would like to have a word with you.
DSMan195276 6y ago
Features get rejected all the time, even some pretty substantial ones. That said I think there's two things to keep in mind: 1. If someone is willing to take the time to develop and submit something to the kernel, it probably has some redeeming factors to it. The original idea or code might not get merged, but something similar to meet those needs likely can be if it's really something the kernel can't do. 2. The modularity of the Linux Kernel build system means that adding new features doesn't have to be that big of a deal, since anybody who doesn't want them can just choose not to compile them in. It's not always that simple, but for a lot of the added features it is.
joppy 6y ago
I don’t think this is an achievable goal for a kernel, since it has strictly more permissions than user code. There are some parts of user code that simply cannot be made faster by changing how they’re written: no amount of rewriting in Rust/C/C++ will help if the relevant kernel primitives are not there. The kernel must move with the times, and incorporate new features as they are (tastefully) decided upon.
penagwin 6y ago
There are pieces of linux - say the networking - that already don't meet many people's use cases. People seem to like BSD for certain networking stuff due to differences in the respective networking stacks.
- zokula 6y ago
  > People seem to like BSD for certain networking stuff due to differences in the respective networking stacks. Is that why *BSD has lower market share for servers and other networking equipment than it did 15 years ago?
  - trasz 6y ago
    It doesn’t, really - 15 or 10 years ago was pretty much the low point, by then everyone who wanted to jump ship did so.
slaymaker1907 6y ago
I think that will be the day somebody comes and creates a new FOSS kernel which gets rid of mostly unused legacy code. The problem with this sort of stuff is that it is hard to know what abstractions will actually be useful before said abstractions are available. I don't think anyone in their right mind would design the IndexedDB APIs the way they are with the background of modern JS. It would certainly be promise based and I think there would be a better distinction between an actual error with IndexedDB vs just an aborted transaction. It's also extremely confusing about how long you can have transactions ongoing. I suspect they would replace the nonsensical autocommit behavior of transactions currently with some sort of timeout mechanism. There is way too much asynchronous code out there these days to rely on the event loop for transactions committing. Linux is honestly pretty clean as far as outdated abstractions go, but Windows is definitely very guilty of keeping around old outdated abstractions. Having execution policies for PowerShell while batch scripts get off scotch free is really strange. These restrictions would only make sense if Windows didn't let users run VBScript, Batch, etc. Can anyone tell me how running a PowerShell script is inherently more dangerous than VBScript or Batch scripts?
stjohnswarts 6y ago
That's a fair criticism but the kernel is very modular and the legacy stuff that no one will maintain gets cut all the time, so I think it's all a fair tradeoff. I'm not angry, just pragmatic because if you don't evolve you die, look at how much market share BSD has lost over the past 15 years
- __d 6y ago
  I think it's hard to blame BSD's (collective) loss of market share on their reluctance to add system calls, or even their pace of evolution more generally. Are there some specific evolutions that you think Linux adopted, and eg. FreeBSD didn't, that were/are crucial to Linux' now overwhelming dominance?

trillic 6y ago

So what? This code (and most drivers, modules, additional fucntionality, etc) will only be compiled in if necessary and will only actually be run if you use the feature.

c7DJTLrn 6y ago
My concern is not about what my flavor of kernel includes. It's about the size of the codebase, and whether the limited number of maintainers can keep up with the countless features being introduced.
- coolreader18 6y ago
  I mean I think in terms of maintainer resources Linux is one of the projects with the least to worry about, there are always companies who are willing to have a team work on stuff.
  - Ericson2314 6y ago
    Those megacorps also accumulate endless complexity internally, as does the industry as a whole. It's a problem larger than the discipline of reviewers of C code could ever hope to fix. Even the larger world outside the field is somewhat aware of the the looming complexity problem: https://www.theatlantic.com/technology/archive/2017/09/saving-the-world-from-code/540393/ https://www.theatlantic.com/technology/archive/2017/09/savin... Basically on our current trajectory we will end up with something like the biosphere that can do stuff but too complex to fully understand --- but also probably well more fragile than that?
    - kaba0 6y ago
      That’s what testing is for. We can’t really reason globally about much smaller codebases either Software is complex.
- cbmuser 6y ago
  The kernel is regularly shedding off unmaintained code. So the codebase is actually not growing as much as it could.
  - c7DJTLrn 6y ago
    The code they're shedding off is drivers. They won't ever get rid of old syscalls because Linus refuses to break userspace, ever.
    - caf 6y ago
      Frequently the old syscall just ends up as a stub that calls into the new implementation though.

pram 6y ago

That’s the beauty of open source! Just fork your own Linux and then refuse any further patches or PRs. True freedom.

dooglius 6y ago

No, because the Linux kernel is designed explicitly as a monolithic kernel, so as long as there is anything anyone wants to do that is "OS-like", it expects to own that.

gpderetta 6y ago

this patch both cleanup existing complexity and add functionality needed for a significant use case.

thedracle 6y ago

I mean, the futex2 proposal has been around for over a year. A lot of the history and discussion can be found here: https://lkml.org/lkml/2021/4/27/1208 https://lkml.org/lkml/2021/4/27/1208 I personally feel like the kernel maintainers have been very conservative over the years regarding the introduction of new system calls, but then again I've maybe gotten used to the trauma of being familiar with windows internals.

fullstop 6y ago

The BSDs are always an option.

zokula 6y ago
The BSDs were an option they are becoming more moribund and less used every year.
- trasz 6y ago
  [citation needed]

bombcar 6y ago

Linus has done exactly that a few times; though usually he defines what would be needed to be merged. Some of the changes aren't compatible with the kernel direction as a whole and so they're a separate branch. Given enough time (and if the change is major AND ends up being widely used) it may get merged in. Those who want a specific kernel with a defined feature set often will look to one of the BSDs as a base (as often they also have reduced hardware requirements, think routers).

yarg 6y ago

As a relative layman, this all seemed reasonable and desirable - and the fact that he integrated it with pthreads seems particularly cool to me. NUMA awareness especially stood out as highly significant. The new supported sizes seems nice and shiny - if well utilised, it could improve performance and capacity in general, capabilities in the low-end and scaling in the high-end.

lizknope 6y ago

Linus Torvalds looks at the big picture all the time. That is basically his job as he doesn't write a lot of low level code anymore. Do you have an example of something that you consider part of the "dumping ground?"

marcodiego 6y ago

Getting a new syscall on linux is hard. You'll often need user-space users before it gets accepted; that alone is hard. The linux syscall set is very small considered the breadth of cases it is used today. Compared to nt, linux syscall set is very small and well thought out.

formerly_proven 6y ago
Windows has around 500 syscalls, Linux around 300. Both have several escape-hatches for arbitrary driver-userspace communication, which makes the number of syscalls pretty irrelevant. (E.g. Linux has ioctl, netlink, procfs, sysfs, ...)
- Ericson2314 6y ago
  > Both have several escape-hatches for arbitrary driver-userspace communication, which makes the number of syscalls pretty irrelevant. (E.g. Linux has ioctl, netlink, procfs, sysfs, ...) Yes, we need a better metric for accounting for those things, which will paint a much more sobering picture of the complexity and growth of complexity of the userland <-> kernel interface.
- monocasa 6y ago
  Well, there's also win32k.sys which provides an additional 1000 or so syscalls.
- marcodiego 6y ago
  Compare this: https://linuxhint.com/list_of_linux_syscalls/ https://linuxhint.com/list_of_linux_syscalls/ with https://j00ru.vexillium.org/syscalls/nt/64/ https://j00ru.vexillium.org/syscalls/nt/64/ and https://j00ru.vexillium.org/syscalls/win32k/64/ https://j00ru.vexillium.org/syscalls/win32k/64/
GoblinSlayer 6y ago
Linux syscall set is very small and posix. No matter what you think, the first version is always a prototype.

rkangel 6y ago

> with nobody looking at the big picture As I understand it, this is exactly what Linus does. He's the final arbiter of what makes it in, and has a history of setting high level direction for how stuff should be (usually prefixed with "I'm rejecting this because").

anoncake 6y ago

What purpose is a general-purpose kernel not built for?

vondur 6y ago

I'm assuming if you build the kernel yourself, you may be able to strip out features you don't want.

broodbucket 6y ago
Yep, there's loads and loads of config options. Linux isn't a big immutable blob, there's a reason it gets used in both tiny devices in supercomputers, you can build your kernel however you like. It's not a microkernel but it does a pretty good job of being flexible for a monolithic kernel. If you're curious, clone the kernel source and run `make menuconfig` and you can play with it yourself

Ericson2314 6y ago

This shouldn't be getting down-votes. Consider the slogan "composition not configuration", Linux is squarely the latter. All the hardware, all the different use-cases, that are just added to what's mostly a monolith. You can disable them, but not really pull things a part. Now, Linux also has some of the most stringent code review in the industry, and they are well aware of issues of maintainability, but fundamentally C is not a sufficiently expressive language for them to architect the code base in a compositional manner. What really worries me is that as the totality of the userland users an increasingly large interface to the kernel, it will very hard to ever replace linux withsomething else when we do have something more compositional. I am therefore interested, and somewhat working on, stuff that hopefully would end up with a multi-kernel NixOS (Think Debian GNU/kFreeBSD) to try to get some kernel competition going again, and incentivize some userland portability. Note that I don't think portability <===> just use old system calls. I am no Unix purist. I would really like a kernel that does add new interfaces but also dispenses with the old cruft; think Rust vs C++.

fabianhjr 6y ago
> Consider the slogan "composition not configuration", Linux is squarely the latter. All the hardware, all the different use-cases, that are just added to what's mostly a monolith. You can disable them, but not really pull things a part. Linux is a Monolithic Kernel, sounds like you are more interested in Microkernels. https://www.wikiwand.com/en/Microkernel https://www.wikiwand.com/en/Microkernel One microkernel alternative could be Redox-OS: https://www.wikiwand.com/en/Redox_(operating_system) https://www.wikiwand.com/en/Redox_(operating_system)
- bruckie 6y ago
  You could write a monolithic kernel using composition for your code structure. It would just be composition at build time rather than at runtime. It seems pretty hard to write a microkernel without using composition, though, since the runtime requirements kind of dictate your code structure.
oalae5niMiel7qu 6y ago
> but fundamentally C is not a sufficiently expressive language for them to architect the code base in a compositional manner. Plan 9 would like to have a word with you.

DSMan195276 6y ago

Features get rejected all the time, even some pretty substantial ones. That said I think there's two things to keep in mind: 1. If someone is willing to take the time to develop and submit something to the kernel, it probably has some redeeming factors to it. The original idea or code might not get merged, but something similar to meet those needs likely can be if it's really something the kernel can't do. 2. The modularity of the Linux Kernel build system means that adding new features doesn't have to be that big of a deal, since anybody who doesn't want them can just choose not to compile them in. It's not always that simple, but for a lot of the added features it is.

joppy 6y ago

I don’t think this is an achievable goal for a kernel, since it has strictly more permissions than user code. There are some parts of user code that simply cannot be made faster by changing how they’re written: no amount of rewriting in Rust/C/C++ will help if the relevant kernel primitives are not there. The kernel must move with the times, and incorporate new features as they are (tastefully) decided upon.

penagwin 6y ago

There are pieces of linux - say the networking - that already don't meet many people's use cases. People seem to like BSD for certain networking stuff due to differences in the respective networking stacks.

zokula 6y ago
> People seem to like BSD for certain networking stuff due to differences in the respective networking stacks. Is that why *BSD has lower market share for servers and other networking equipment than it did 15 years ago?
- trasz 6y ago
  It doesn’t, really - 15 or 10 years ago was pretty much the low point, by then everyone who wanted to jump ship did so.

slaymaker1907 6y ago

I think that will be the day somebody comes and creates a new FOSS kernel which gets rid of mostly unused legacy code. The problem with this sort of stuff is that it is hard to know what abstractions will actually be useful before said abstractions are available. I don't think anyone in their right mind would design the IndexedDB APIs the way they are with the background of modern JS. It would certainly be promise based and I think there would be a better distinction between an actual error with IndexedDB vs just an aborted transaction. It's also extremely confusing about how long you can have transactions ongoing. I suspect they would replace the nonsensical autocommit behavior of transactions currently with some sort of timeout mechanism. There is way too much asynchronous code out there these days to rely on the event loop for transactions committing. Linux is honestly pretty clean as far as outdated abstractions go, but Windows is definitely very guilty of keeping around old outdated abstractions. Having execution policies for PowerShell while batch scripts get off scotch free is really strange. These restrictions would only make sense if Windows didn't let users run VBScript, Batch, etc. Can anyone tell me how running a PowerShell script is inherently more dangerous than VBScript or Batch scripts?

stjohnswarts 6y ago

That's a fair criticism but the kernel is very modular and the legacy stuff that no one will maintain gets cut all the time, so I think it's all a fair tradeoff. I'm not angry, just pragmatic because if you don't evolve you die, look at how much market share BSD has lost over the past 15 years

__d 6y ago
I think it's hard to blame BSD's (collective) loss of market share on their reluctance to add system calls, or even their pace of evolution more generally. Are there some specific evolutions that you think Linux adopted, and eg. FreeBSD didn't, that were/are crucial to Linux' now overwhelming dominance?