8 ms·

splice() and the ghost of set_fs()

210 points by chmaynard 4y ago

encryptluks2 4y ago
Systemd had a regression recently that has broken booting a secondary encrypted LUKS device.
- teddyh 4y ago
  What does “booting a secondary […] device” mean? Either you boot from a device, and that makes it the primary device, or you boot off another device, in which case this is now the primary device. Or did you mean unlocking an encrypted device after unlocking and booting on the primary device? What does systemd have to do with that? Unless you want to type in the password manually every boot, you store the password (for the secondary device) on the primary device, add the password file to /etc/crypttab and be done with it, right?
manchmalscott 4y ago
The only time I’ve had a kernel update break things was a weird Proxmox bug where if you booted with the (at the time) latest kernel, it would fail to start the first VM, and nothing you did from the UI or the command line could touch it without timing out. Rebooting to the previous kernel release and it just started working again with no other changes. That definitely broke my assumption that kernel updates were well vetted for regressions.
- bombcar 4y ago
  I suspect kernel updates aren't vetted that well in general; the "don't break user space" is a ruling from Linus about what not to do - usually with regards to the user space API (don't remove or change things that already exist). And this article is an example of where the decision was made to break user space.
  - josefx 4y ago
    Is it even a hard break? It seems to be more of a short lived regression while patches for filesystem drivers are still coming in to restore support and I don't think Linux ever guaranteed a stable driver API.
- simlevesque 4y ago
  > kernel updates were well vetted for regressions. That seems like the kind of things that is easier said than done.
- caslon 4y ago
  > The only time I’ve had a kernel update break things > That definitely broke my assumption that kernel updates were well vetted for regressions. Shouldn't that be evidence for the contrary? One break in a decade or two of use isn't so bad. It's actually pretty good.
- 867-5309 4y ago
  I've just had that with the most recent Proxmox update and also had no option but to regress
  - Fnoord 4y ago
    I had the issue as well but I managed to fix it by I changing my boot parameters (exact change differs per bootloader and hardware). Now I happily boot with .15 series (up from .13). I expect some more users of Proxmox given Broadcom taking over VMware (e.g. I'd like to merge away from ESXi to Proxmox as I don't trust Broadcom). Hopefully it does the product and community well.
- bityard 4y ago
  "Don't break userspace" is more like a _goal_ than a law. The kernel is an extremely complex piece of code, the developers can't be asked to test every kernel release against every piece of userspace software on every hardware configuration. That's one of the reasons that new code and significant changes require a bunch of mailing list discussion, reviews, and signing-off. Also, Proxmox ships with its own supported kernel, it shouldn't be a big surprise that you ran into issues while straying off the beaten path.
  - londons_explore 4y ago
    > The kernel is an extremely complex piece of code, With no regression tests, nearly no unit tests, and lots of bits of supported hardware that nobody on the Dev team uses day to day... It's a miracle that quality is as high as it is.
    - bonzini 4y ago
      There are plenty of tests, just not in Linus's tree. KVM has not one but three suites of unit and integration tests, for example, but only 60ish tests are in the kernel's tools/testing/selftests directory.
- yjftsjthsd-h 4y ago
  > That definitely broke my assumption that kernel updates were well vetted for regressions. I would argue that it's also a failure by the distro; unless there was something special about your exact setup that would have made the bug not show up in testing, I would argue that proxmox should have been testing updates to catch that kind of problem before users noticed.
  - fomine3 4y ago
    Proxmox VE provides Enterprise and No-Subscription repository. Possibly GP uses latter repo that's for testing (and free!). Maybe this? https://forum.proxmox.com/threads/latest-update-5-13-19-4-pve-broke-my-qemu-pcie-sharing-works-with-5-13-19-3.104252/ https://forum.proxmox.com/threads/latest-update-5-13-19-4-pv...
    - yjftsjthsd-h 4y ago
      Ah, so free users hitting bugs is the test suite:)
i13e 4y ago
Not sure if here is the place for this but here goes. Personally I've been dealing with a regression of sorts on my laptop: it no longer suspends without being quickly woken up again on the main kernel release (works fine on LTS). Does anyone know if and where I could file a bug report?
- mathstuf 4y ago
  I noticed it recently too, but disconnecting my Bluetooth headset first allowed it to suspend properly (it seems like the wifi chip kept it awake before making like a yo-yo back and forth).
  - zonotope 4y ago
    Me 3 and it's maddening. I've posted this [1] on the Arch Linux forums but haven't found a fix. I've seen it mentioned around the web that it's a known bug, but I have yet to find a bug report 1: https://bbs.archlinux.org/viewtopic.php?id=276064 https://bbs.archlinux.org/viewtopic.php?id=276064
    - csdvrx 4y ago
      It's an open question whether Bluetooth should prevent sleep, or let sleep go through. On a well done "modern suspend/suspend to idle", I use disconnected modern sleep + Windows Media Player on Windows 11 to listen to songs on my bluetooth headphones for a cost of about 1% of the battery per hour (as measured and plotted with powercfg) which can come down to about half of it, 0.5%/h when not using Bluetooth. I wouldn't want Bluetooth to prevent sleep (a 1% drop per hour is better than not sleeping!) I also wouldn't want sleep to prevent me from using my Bluetooth headset (the difference between 0.5% and 1% is significant, but it doesn't matter much in practice if my computer can be usable in the morning) This is one of the rare examples of "modern suspend" delivering on its promises, and blowing the good old ACPI S3 away: I never saw a drop of battery <5% on S3 suspend-to-ram unless it also involved S4 in a hybrid sleep of "ACPI S3 suspend-to-ram then S4 suspend-to-disk after a while or when I run out of power whichever comes first"!
sc68cal 4y ago
I think that leaving splice() as it was, is worse than breaking the "NEVER BREAK USERSPACE" rule, in this specific instance. It should not become a regular occurence, but I don't see a better way. They tried to stick to it, things happened, but I am optimistic that they will fix it and try better the next time.
- rwj 4y ago
  If rule #1 is don't break userspace, we might need a rule #0, which would be don't support insecure behaviour. In the end, people might need to balance competing concerns...
mc4ndr3 4y ago
with_resource(mutex file handle etc etc *o, func callback) is a safer pattern, for this reason. Thank you, Python (with_resource is a generic example. Search for such capabilities in your library.) Computers exist to automate. Let the computer remember to close resources upon scope end. Go's defer is also helpful in this regard.
- viraptor 4y ago
  That was my first thought - would they leave the set_fs solution in place if C had a native solution for resource blocks / RAII+drop.
  - ars 4y ago
    It's not that hard to do in C. Make a macro that takes a function, it runs: set_fs(KERN) *function_pointer() set_fs(USER) There did it before, so they knew about that option, and if they didn't do it, they must have had a reason.
    - simias 4y ago
      It's true, unfortunately the lack of lambdas in C makes this pattern cumbersome. You could achieve it with a macro instead, I suppose.
  - shaded-enmity 4y ago
    A simple policy that both set_fs() calls need to happen within the same function body with corresponding CI test based on AST/DWARF inspection would have also prevented it. Do you really want to rely on stack unwinding/destructors for security sensitive code when stack is usually the first thing that gets controlled by the attacker? Exception handling (SEH) on Windows is an exploitation vector of it's own.
    - viraptor 4y ago
      I'm talking about the general idea not specific implementation. Having something happen at function/block exit doesn't mean a runtime configurable behaviour. If you don't have exceptions, it's pretty easy to statically compile that behaviour and guarantee it rather than rely on checks.
theteapot 4y ago
Shouldn't they have left it for 6.0?
- mhitza 4y ago
  Kernel doesn't adhere to a SemVer policy
  - theteapot 4y ago
    Maybe they should. Maybe, they, should.
    - SAI_Peregrinus 4y ago
      Naw. Kernel version number increases. Can't be more than 255 in any field, except the EXTRAVERSION which is a string. Kernel version numbering is exposed in the userspace ABI, so changing it to have meaning would be a breaking change for no substantial benefit. This[1] LWN article is a good overview of the history. Effectively the major gets incremented when the minor gets to 20 and Linus runs out of fingers & toes to keep track. [1] https://lwn.net/Articles/871989/ https://lwn.net/Articles/871989/
      - theteapot 4y ago
        > Can't be more than 255 in any field, except the EXTRAVERSION which is a string. Kernel version numbering is exposed in the userspace ABI, so changing it to have meaning would be a breaking change. Sounds like they should leave this change for 6.0 then.
        SAI_Peregrinus 4y ago
        Why? The kernel version number has no meaning, other than to provide an ordering of releases.
        theteapot 4y ago
deleted 4y ago
[deleted]
mark_undoio 4y ago
Our software[1] got broken by a kernel change a few years ago. The experience was quite interesting. We were making use of `/proc/PID/pagemap`, which is a kernel-generated file that previously would show the physical addresses of all the pages in a process's address space. Unfortunately, with the Rowhammer exploit, exposing this information - even for one's own processes - to unprivileged users went from being harmless to a security risk. The first we saw of the change was when newer kernels started reporting zeros for all physical addresses, unless we ran as root. We raised this the LKML, explaining that we'd been relying on this feature to implement a somewhat esoteric optimisation. Linus replied very helpfully - the security fix trumped userspace compatibility but he could see a secure way of getting us the information we really needed, given the technique we'd described. He invited us to submit a kernel patch and gave a few hints about potential gotchas. I did work one up but the kernel community actually jumped on it as an opportunity to do more cleanup, so I ended up just signing off on the patch they produced. It was all a remarkably smooth and efficient process. [1] Time travel debugging - http://undo.io http://undo.io
- khuey 4y ago
  Heh, as someone who maintains another exotic debugger[1] my experience has been that the kernel development process is kind of a pain. We've had good experiences with people fixing regressions once they're discovered but getting new features into the kernel has been difficult. I think it took 10 revisions for me to get cpuid faulting into the kernel, including multiple review cycles where I was first told "change X to Y" and then in a subsequent cycle told "change Y to X". [1] https://rr-project.org/ https://rr-project.org/
- Havoc 4y ago
  > the security fix trumped userspace compatibility TIL All stories about kernels describe it as an absolute that userspace is never broken, but that actually makes sense
  - Quekid5 4y ago
    Yeah, there are rare exceptions to The Rule.
teraflop 4y ago
Even if you don't want to directly support splice() for every possible filesystem or file descriptor, I don't understand why it couldn't be "emulated" in those cases, by having the kernel just pretend it was given a series of read()/write() calls. That might not be as efficient, but surely it would be better than just breaking completely.
- londons_explore 4y ago
  Perhaps because some of the semantics of a spliced pipe aren't exactly the same as read() followed by write()?
- derefr 4y ago
  The point of splice(2) is to be a fast-path; it's not in POSIX, so software that wants to guarantee portability has to write the slow path too, and only use splice(2) if it's present. Such a blind/naive compatibility shim, would likely be much slower than whatever hand-written slow path the developer has in place. If the whole point of a call is to be a fast/efficient version of something else, then the call is breaking its semantics if it isn't more fast/efficient than the alternative. Because of this, it's better to just make autoconf et al detect splice(2) as absent for the given use-case (and fall back to the hand-rolled portable slow path), rather than relying on a naive kernel shim.
  - bonzini 4y ago
    You cannot in general use autoconf to detect runtime behavior.
    - rootw0rm 4y ago
      I don't think they're suggesting runtime use, but rather building software after the splice(2) change. That's how I read it at least.
      - bonzini 4y ago
        It would break if compiled on old kernel and run on new kernel. When CI is containerized the kernel might not even have anything to do with the distro that you're building on.
loeg 4y ago
> But it is true that this type of episode makes the kernel's "no regressions" rule look a bit more like just a guideline. It does not take too many of those to create breakage to the project's reputation that is hard to splice back together.
lamontcg 4y ago
Don't like the lecturing tone of this article. This seems entirely successful to me and retired crappy and insecure behavior. The idea that you can never break anything means that some historical mistakes can never get fixed. The someone complains about how the language / operating system is a bolted on mess of nonsense and people go chasing after the next hot thing which has the benefit of just being newer and being able to get away with breaking changes because people on the bleeding edge put up with it. The linux kernel is probably striking the right balance here, and I doubt that this one breakage is a sign of the decay and downfall of the Empire.
ranger207 4y ago
As an aside, if you enjoy this article please consider subscribing to LWN at [0]. Normally articles are only available for free a week after they're posted; one of the perks of subscribing is being able to send a link like the one posted here so non-subscribers can view it before the week has passed. LWN's main source of revenue as I understand it is subscriptions, and the work they do is definitely worth it [0] https://lwn.net/subscribe/Info https://lwn.net/subscribe/Info
- rtpg 4y ago
  yep, LWN is a great resource, highly recommend for any developer (even/especially those who aren't linux people, they have lots of articles regarding other programming language development and the like). I used to complain that they didn't have recurring subscriptions, but I believe they have fixed that! Just sign up yearly, at the worst you are a patron to good journalism that is not really happening elsewhere. Just reading an article or two a week will give you loads of insight into what is going on in the tools you have everyday
raggi 4y ago
This could have interesting implications for Go programs. Go optimizes io.Copy down to splice if dst and src both support the necessary bits. Unlike C code where you could easily grep this out, this happens transparently, and could even by influenced by user input in various cases. The tail could be quite long for bugs that show up from dropping support. I'm sure some dynlangs may have similar optimizations. edit: checked the current implementation in go, and it's only doing that for unix and tcp sockets, so it'll be fine.
kccqzy 4y ago
The "don't break user space" guideline should just be called "break user space, and if someone complains, fix it afterwards."
freebuju 4y ago
Not too long ago I was constantly facing this kernel bug [0] that would kill audio whenever system memory would be under moderately high load. It's fixed now but it's kind of insane that the kernel would mess around with components as critical as sound for desktop users. [0]https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2272 https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/22...

210 points by chmaynard 4y ago

encryptluks2 4y ago

Systemd had a regression recently that has broken booting a secondary encrypted LUKS device.

teddyh 4y ago
What does “booting a secondary […] device” mean? Either you boot from a device, and that makes it the primary device, or you boot off another device, in which case this is now the primary device. Or did you mean unlocking an encrypted device after unlocking and booting on the primary device? What does systemd have to do with that? Unless you want to type in the password manually every boot, you store the password (for the secondary device) on the primary device, add the password file to /etc/crypttab and be done with it, right?

manchmalscott 4y ago

The only time I’ve had a kernel update break things was a weird Proxmox bug where if you booted with the (at the time) latest kernel, it would fail to start the first VM, and nothing you did from the UI or the command line could touch it without timing out. Rebooting to the previous kernel release and it just started working again with no other changes. That definitely broke my assumption that kernel updates were well vetted for regressions.

bombcar 4y ago
I suspect kernel updates aren't vetted that well in general; the "don't break user space" is a ruling from Linus about what not to do - usually with regards to the user space API (don't remove or change things that already exist). And this article is an example of where the decision was made to break user space.
- josefx 4y ago
  Is it even a hard break? It seems to be more of a short lived regression while patches for filesystem drivers are still coming in to restore support and I don't think Linux ever guaranteed a stable driver API.
simlevesque 4y ago
> kernel updates were well vetted for regressions. That seems like the kind of things that is easier said than done.
caslon 4y ago
> The only time I’ve had a kernel update break things > That definitely broke my assumption that kernel updates were well vetted for regressions. Shouldn't that be evidence for the contrary? One break in a decade or two of use isn't so bad. It's actually pretty good.
867-5309 4y ago
I've just had that with the most recent Proxmox update and also had no option but to regress
- Fnoord 4y ago
  I had the issue as well but I managed to fix it by I changing my boot parameters (exact change differs per bootloader and hardware). Now I happily boot with .15 series (up from .13). I expect some more users of Proxmox given Broadcom taking over VMware (e.g. I'd like to merge away from ESXi to Proxmox as I don't trust Broadcom). Hopefully it does the product and community well.
bityard 4y ago
"Don't break userspace" is more like a _goal_ than a law. The kernel is an extremely complex piece of code, the developers can't be asked to test every kernel release against every piece of userspace software on every hardware configuration. That's one of the reasons that new code and significant changes require a bunch of mailing list discussion, reviews, and signing-off. Also, Proxmox ships with its own supported kernel, it shouldn't be a big surprise that you ran into issues while straying off the beaten path.
- londons_explore 4y ago
  > The kernel is an extremely complex piece of code, With no regression tests, nearly no unit tests, and lots of bits of supported hardware that nobody on the Dev team uses day to day... It's a miracle that quality is as high as it is.
  - bonzini 4y ago
    There are plenty of tests, just not in Linus's tree. KVM has not one but three suites of unit and integration tests, for example, but only 60ish tests are in the kernel's tools/testing/selftests directory.
yjftsjthsd-h 4y ago
> That definitely broke my assumption that kernel updates were well vetted for regressions. I would argue that it's also a failure by the distro; unless there was something special about your exact setup that would have made the bug not show up in testing, I would argue that proxmox should have been testing updates to catch that kind of problem before users noticed.
- fomine3 4y ago
  Proxmox VE provides Enterprise and No-Subscription repository. Possibly GP uses latter repo that's for testing (and free!). Maybe this? https://forum.proxmox.com/threads/latest-update-5-13-19-4-pve-broke-my-qemu-pcie-sharing-works-with-5-13-19-3.104252/ https://forum.proxmox.com/threads/latest-update-5-13-19-4-pv...
  - yjftsjthsd-h 4y ago
    Ah, so free users hitting bugs is the test suite:)

i13e 4y ago

Not sure if here is the place for this but here goes. Personally I've been dealing with a regression of sorts on my laptop: it no longer suspends without being quickly woken up again on the main kernel release (works fine on LTS). Does anyone know if and where I could file a bug report?

mathstuf 4y ago
I noticed it recently too, but disconnecting my Bluetooth headset first allowed it to suspend properly (it seems like the wifi chip kept it awake before making like a yo-yo back and forth).
- zonotope 4y ago
  Me 3 and it's maddening. I've posted this [1] on the Arch Linux forums but haven't found a fix. I've seen it mentioned around the web that it's a known bug, but I have yet to find a bug report 1: https://bbs.archlinux.org/viewtopic.php?id=276064 https://bbs.archlinux.org/viewtopic.php?id=276064
  - csdvrx 4y ago
    It's an open question whether Bluetooth should prevent sleep, or let sleep go through. On a well done "modern suspend/suspend to idle", I use disconnected modern sleep + Windows Media Player on Windows 11 to listen to songs on my bluetooth headphones for a cost of about 1% of the battery per hour (as measured and plotted with powercfg) which can come down to about half of it, 0.5%/h when not using Bluetooth. I wouldn't want Bluetooth to prevent sleep (a 1% drop per hour is better than not sleeping!) I also wouldn't want sleep to prevent me from using my Bluetooth headset (the difference between 0.5% and 1% is significant, but it doesn't matter much in practice if my computer can be usable in the morning) This is one of the rare examples of "modern suspend" delivering on its promises, and blowing the good old ACPI S3 away: I never saw a drop of battery <5% on S3 suspend-to-ram unless it also involved S4 in a hybrid sleep of "ACPI S3 suspend-to-ram then S4 suspend-to-disk after a while or when I run out of power whichever comes first"!

sc68cal 4y ago

I think that leaving splice() as it was, is worse than breaking the "NEVER BREAK USERSPACE" rule, in this specific instance. It should not become a regular occurence, but I don't see a better way. They tried to stick to it, things happened, but I am optimistic that they will fix it and try better the next time.

rwj 4y ago
If rule #1 is don't break userspace, we might need a rule #0, which would be don't support insecure behaviour. In the end, people might need to balance competing concerns...

mc4ndr3 4y ago

with_resource(mutex file handle etc etc *o, func callback) is a safer pattern, for this reason. Thank you, Python (with_resource is a generic example. Search for such capabilities in your library.) Computers exist to automate. Let the computer remember to close resources upon scope end. Go's defer is also helpful in this regard.

viraptor 4y ago
That was my first thought - would they leave the set_fs solution in place if C had a native solution for resource blocks / RAII+drop.
- ars 4y ago
  It's not that hard to do in C. Make a macro that takes a function, it runs: set_fs(KERN) *function_pointer() set_fs(USER) There did it before, so they knew about that option, and if they didn't do it, they must have had a reason.
  - simias 4y ago
    It's true, unfortunately the lack of lambdas in C makes this pattern cumbersome. You could achieve it with a macro instead, I suppose.
- shaded-enmity 4y ago
  A simple policy that both set_fs() calls need to happen within the same function body with corresponding CI test based on AST/DWARF inspection would have also prevented it. Do you really want to rely on stack unwinding/destructors for security sensitive code when stack is usually the first thing that gets controlled by the attacker? Exception handling (SEH) on Windows is an exploitation vector of it's own.
  - viraptor 4y ago
    I'm talking about the general idea not specific implementation. Having something happen at function/block exit doesn't mean a runtime configurable behaviour. If you don't have exceptions, it's pretty easy to statically compile that behaviour and guarantee it rather than rely on checks.

theteapot 4y ago

Shouldn't they have left it for 6.0?

mhitza 4y ago
Kernel doesn't adhere to a SemVer policy
- theteapot 4y ago
  Maybe they should. Maybe, they, should.
  - SAI_Peregrinus 4y ago
    Naw. Kernel version number increases. Can't be more than 255 in any field, except the EXTRAVERSION which is a string. Kernel version numbering is exposed in the userspace ABI, so changing it to have meaning would be a breaking change for no substantial benefit. This[1] LWN article is a good overview of the history. Effectively the major gets incremented when the minor gets to 20 and Linus runs out of fingers & toes to keep track. [1] https://lwn.net/Articles/871989/ https://lwn.net/Articles/871989/
    - theteapot 4y ago
      > Can't be more than 255 in any field, except the EXTRAVERSION which is a string. Kernel version numbering is exposed in the userspace ABI, so changing it to have meaning would be a breaking change. Sounds like they should leave this change for 6.0 then.
      - SAI_Peregrinus 4y ago
        Why? The kernel version number has no meaning, other than to provide an ordering of releases.
        theteapot 4y ago

deleted 4y ago

[deleted]

mark_undoio 4y ago

Our software[1] got broken by a kernel change a few years ago. The experience was quite interesting. We were making use of `/proc/PID/pagemap`, which is a kernel-generated file that previously would show the physical addresses of all the pages in a process's address space. Unfortunately, with the Rowhammer exploit, exposing this information - even for one's own processes - to unprivileged users went from being harmless to a security risk. The first we saw of the change was when newer kernels started reporting zeros for all physical addresses, unless we ran as root. We raised this the LKML, explaining that we'd been relying on this feature to implement a somewhat esoteric optimisation. Linus replied very helpfully - the security fix trumped userspace compatibility but he could see a secure way of getting us the information we really needed, given the technique we'd described. He invited us to submit a kernel patch and gave a few hints about potential gotchas. I did work one up but the kernel community actually jumped on it as an opportunity to do more cleanup, so I ended up just signing off on the patch they produced. It was all a remarkably smooth and efficient process. [1] Time travel debugging - http://undo.io http://undo.io

khuey 4y ago
Heh, as someone who maintains another exotic debugger[1] my experience has been that the kernel development process is kind of a pain. We've had good experiences with people fixing regressions once they're discovered but getting new features into the kernel has been difficult. I think it took 10 revisions for me to get cpuid faulting into the kernel, including multiple review cycles where I was first told "change X to Y" and then in a subsequent cycle told "change Y to X". [1] https://rr-project.org/ https://rr-project.org/
Havoc 4y ago
> the security fix trumped userspace compatibility TIL All stories about kernels describe it as an absolute that userspace is never broken, but that actually makes sense
- Quekid5 4y ago
  Yeah, there are rare exceptions to The Rule.

teraflop 4y ago

Even if you don't want to directly support splice() for every possible filesystem or file descriptor, I don't understand why it couldn't be "emulated" in those cases, by having the kernel just pretend it was given a series of read()/write() calls. That might not be as efficient, but surely it would be better than just breaking completely.

londons_explore 4y ago
Perhaps because some of the semantics of a spliced pipe aren't exactly the same as read() followed by write()?
derefr 4y ago
The point of splice(2) is to be a fast-path; it's not in POSIX, so software that wants to guarantee portability has to write the slow path too, and only use splice(2) if it's present. Such a blind/naive compatibility shim, would likely be much slower than whatever hand-written slow path the developer has in place. If the whole point of a call is to be a fast/efficient version of something else, then the call is breaking its semantics if it isn't more fast/efficient than the alternative. Because of this, it's better to just make autoconf et al detect splice(2) as absent for the given use-case (and fall back to the hand-rolled portable slow path), rather than relying on a naive kernel shim.
- bonzini 4y ago
  You cannot in general use autoconf to detect runtime behavior.
  - rootw0rm 4y ago
    I don't think they're suggesting runtime use, but rather building software after the splice(2) change. That's how I read it at least.
    - bonzini 4y ago
      It would break if compiled on old kernel and run on new kernel. When CI is containerized the kernel might not even have anything to do with the distro that you're building on.

loeg 4y ago

> But it is true that this type of episode makes the kernel's "no regressions" rule look a bit more like just a guideline. It does not take too many of those to create breakage to the project's reputation that is hard to splice back together.

lamontcg 4y ago

Don't like the lecturing tone of this article. This seems entirely successful to me and retired crappy and insecure behavior. The idea that you can never break anything means that some historical mistakes can never get fixed. The someone complains about how the language / operating system is a bolted on mess of nonsense and people go chasing after the next hot thing which has the benefit of just being newer and being able to get away with breaking changes because people on the bleeding edge put up with it. The linux kernel is probably striking the right balance here, and I doubt that this one breakage is a sign of the decay and downfall of the Empire.

ranger207 4y ago

As an aside, if you enjoy this article please consider subscribing to LWN at [0]. Normally articles are only available for free a week after they're posted; one of the perks of subscribing is being able to send a link like the one posted here so non-subscribers can view it before the week has passed. LWN's main source of revenue as I understand it is subscriptions, and the work they do is definitely worth it [0] https://lwn.net/subscribe/Info https://lwn.net/subscribe/Info

rtpg 4y ago
yep, LWN is a great resource, highly recommend for any developer (even/especially those who aren't linux people, they have lots of articles regarding other programming language development and the like). I used to complain that they didn't have recurring subscriptions, but I believe they have fixed that! Just sign up yearly, at the worst you are a patron to good journalism that is not really happening elsewhere. Just reading an article or two a week will give you loads of insight into what is going on in the tools you have everyday

raggi 4y ago

This could have interesting implications for Go programs. Go optimizes io.Copy down to splice if dst and src both support the necessary bits. Unlike C code where you could easily grep this out, this happens transparently, and could even by influenced by user input in various cases. The tail could be quite long for bugs that show up from dropping support. I'm sure some dynlangs may have similar optimizations. edit: checked the current implementation in go, and it's only doing that for unix and tcp sockets, so it'll be fine.

kccqzy 4y ago

The "don't break user space" guideline should just be called "break user space, and if someone complains, fix it afterwards."

freebuju 4y ago

Not too long ago I was constantly facing this kernel bug [0] that would kill audio whenever system memory would be under moderately high load. It's fixed now but it's kind of insane that the kernel would mess around with components as critical as sound for desktop users. [0]https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2272 https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/22...