7 ms·

Curious if anyone knows how the kernel is tested?

klodolph 9y ago
…I mean, yes? This question is well documented and simple Google searches will turn up a large number of high-quality results, such as https://stackoverflow.com/questions/3177338/how-is-the-linux-kernel-tested https://stackoverflow.com/questions/3177338/how-is-the-linux....
- xwvvvvwx 9y ago
  Don't be boring. The nice thing about asking on hacker news is you get answers from experts, and you can often get insider knowledge that really isn't available anywhere on google. But thanks for the link, it's a good one ;)
  - klodolph 9y ago
    Please don't accuse me of being a boring person. I'm not sure why you think that is acceptable behavior. The nice thing about doing a minute of preliminary research when you have a question is that the questions you ask afterwards will be more interesting.
simcop2387 9y ago
As best as they can, and by a lot of people running release candidates on test systems. Unfortunately there are basically an infinite (not really, just incredibly impossibly large) number of configurations of software and hardware that can lead to a bug like this not triggering so it's not always possible to catch this ahead of time.
- xwvvvvwx 9y ago
  How often does a bug of this severity slip through the net?
  - katastic 9y ago
    The only one I can find is in 2012 there was a RAID bug that damaged metadata required for RAID access, under specific/rare conditions. http://www.h-online.com/open/news/item/Bug-in-Linux-kernel-can-damage-RAID-arrays-1621164.html http://www.h-online.com/open/news/item/Bug-in-Linux-kernel-c... But if anyone is running critical data, they shouldn't be using a bleeding-edge kernel in the first damn place. It certainly "sucks" for the users that found it, but again, that's the risk you play. And, this bug seems to have been found... a mere week after articles even reported the release of Kernel 4.14: https://bugs.gentoo.org/638206 https://bugs.gentoo.org/638206 http://www.omgubuntu.co.uk/2017/11/linux-kernel-4-14-lts-features http://www.omgubuntu.co.uk/2017/11/linux-kernel-4-14-lts-fea... Which is a pretty amazing turn-around time for "crowd-sourced" discovery of a bug. And according to the bug tracker, it looks like they patched it the same day. So as bad as this is (and I feel for those who lost data, I really do), the solution is simply to not run bleeding-edge kernels on release day. The very fact they have a minor version number 4.14.1 (.1) implies things get released that need fixed, and if you have critical data you should be more patient.
    - raller 9y ago
      > bleeding-edge kernel kernel > risk you play uhm you realize that the kernel was already released upstream right?
      - striking 9y ago
        Yeah, and which distros are shipping it? Using the newest upstream kernel is something you opt into, on non-production hardware. This isn't something that would impact a regular user.
        rdc12 9y ago
        How many people would be using bcache in production fullstop, I'm guessing not many.
      - 0x0 9y ago
        It's been the official position for quite a while that kernel.org releases - even "stable" ones - are to be considered bleeding-edge, and for it to be the responsibility of distros to do the necessary cooking on behalf of users to avoid unleashing something raw on them. Long gone are the days of regular users being trusted (and being able to trust) to just grab the latest linux-2.0.36 tarball from there and upgrade by compiling themselves.
joshbaptiste 9y ago
There was a recent HN post that delved into booting and testing on many machines and fuzzing injections but can't find it at the moment.
osandov 9y ago
Every subsystem is tested differently. Because of the wide variety of subsystems and workflows in the kernel, it's pretty much impossible to have a single test suite you can run and call good. For example, some subsystems have unit tests that run in the kernel at boot time if you compiled the kernel with a specific config option (e.g., Btrfs [1] and ftrace [2]). Additionally, some subsystems have integration tests where we set stuff up and run from userspace. Filesystems use xfstests [3] and the block layer has something similar [4]. For the most part, developers and maintainers run these before sending patches or pull requests. Additionally, Intel operates an automated system [5] which runs several test suites and reports the results to developers. 1: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/btrfs/tests https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... 2: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/trace/trace_selftest.c https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... 3: https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/ https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/ 4: https://github.com/osandov/blktests https://github.com/osandov/blktests 5: https://01.org/lkp/documentation/0-day-test-service https://01.org/lkp/documentation/0-day-test-service

…I mean, yes? This question is well documented and simple Google searches will turn up a large number of high-quality results, such as https://stackoverflow.com/questions/3177338/how-is-the-linux-kernel-tested https://stackoverflow.com/questions/3177338/how-is-the-linux....

xwvvvvwx 9y ago
Don't be boring. The nice thing about asking on hacker news is you get answers from experts, and you can often get insider knowledge that really isn't available anywhere on google. But thanks for the link, it's a good one ;)
- klodolph 9y ago
  Please don't accuse me of being a boring person. I'm not sure why you think that is acceptable behavior. The nice thing about doing a minute of preliminary research when you have a question is that the questions you ask afterwards will be more interesting.

simcop2387 9y ago

As best as they can, and by a lot of people running release candidates on test systems. Unfortunately there are basically an infinite (not really, just incredibly impossibly large) number of configurations of software and hardware that can lead to a bug like this not triggering so it's not always possible to catch this ahead of time.

xwvvvvwx 9y ago
How often does a bug of this severity slip through the net?
- katastic 9y ago
  The only one I can find is in 2012 there was a RAID bug that damaged metadata required for RAID access, under specific/rare conditions. http://www.h-online.com/open/news/item/Bug-in-Linux-kernel-can-damage-RAID-arrays-1621164.html http://www.h-online.com/open/news/item/Bug-in-Linux-kernel-c... But if anyone is running critical data, they shouldn't be using a bleeding-edge kernel in the first damn place. It certainly "sucks" for the users that found it, but again, that's the risk you play. And, this bug seems to have been found... a mere week after articles even reported the release of Kernel 4.14: https://bugs.gentoo.org/638206 https://bugs.gentoo.org/638206 http://www.omgubuntu.co.uk/2017/11/linux-kernel-4-14-lts-features http://www.omgubuntu.co.uk/2017/11/linux-kernel-4-14-lts-fea... Which is a pretty amazing turn-around time for "crowd-sourced" discovery of a bug. And according to the bug tracker, it looks like they patched it the same day. So as bad as this is (and I feel for those who lost data, I really do), the solution is simply to not run bleeding-edge kernels on release day. The very fact they have a minor version number 4.14.1 (.1) implies things get released that need fixed, and if you have critical data you should be more patient.
  - raller 9y ago
    > bleeding-edge kernel kernel > risk you play uhm you realize that the kernel was already released upstream right?
    - striking 9y ago
      Yeah, and which distros are shipping it? Using the newest upstream kernel is something you opt into, on non-production hardware. This isn't something that would impact a regular user.
      - rdc12 9y ago
        How many people would be using bcache in production fullstop, I'm guessing not many.
    - 0x0 9y ago
      It's been the official position for quite a while that kernel.org releases - even "stable" ones - are to be considered bleeding-edge, and for it to be the responsibility of distros to do the necessary cooking on behalf of users to avoid unleashing something raw on them. Long gone are the days of regular users being trusted (and being able to trust) to just grab the latest linux-2.0.36 tarball from there and upgrade by compiling themselves.

joshbaptiste 9y ago

There was a recent HN post that delved into booting and testing on many machines and fuzzing injections but can't find it at the moment.

osandov 9y ago

Every subsystem is tested differently. Because of the wide variety of subsystems and workflows in the kernel, it's pretty much impossible to have a single test suite you can run and call good. For example, some subsystems have unit tests that run in the kernel at boot time if you compiled the kernel with a specific config option (e.g., Btrfs [1] and ftrace [2]). Additionally, some subsystems have integration tests where we set stuff up and run from userspace. Filesystems use xfstests [3] and the block layer has something similar [4]. For the most part, developers and maintainers run these before sending patches or pull requests. Additionally, Intel operates an automated system [5] which runs several test suites and reports the results to developers. 1: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/btrfs/tests https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... 2: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/trace/trace_selftest.c https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... 3: https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/ https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/ 4: https://github.com/osandov/blktests https://github.com/osandov/blktests 5: https://01.org/lkp/documentation/0-day-test-service https://01.org/lkp/documentation/0-day-test-service