2 ms·

> The most common features and configurations will get tested over and over and over well beyond the point of usefulness. Less common features might not get tes

by m45t3r 8y ago

> The most common features and configurations will get tested over and over and over well beyond the point of usefulness. Less common features might not get tested at all. Not that I wouldn't want a featured Unix-like kernel with a comprehensive[1] test suite (and I don't know any major OS that have this, not Windows, macOS or Linux), however I think this is fine. Common setups works fine, uncommon configurations may have some problems, with possible workarounds. Most people will not run mainline kernels anyway, so a workaround is acceptable. [1]: What I mean by comprehensive is basically having tests for every possible configuration, that is basically impossible anyway. Probably the closest thing we can get is formally proven OS, however I don't think we will even have a general purpose OS formally proven.

notacoward 8y ago
> tests for every possible configuration, that is basically impossible anyway Agreed. Expecting 100% across the entire kernel would be totally unreasonable. OTOH, coverage could be better on a lot of components considered individually. Any component as complex as XFS is going to have tons of bugs. I don't mean that as an insult. I was a filesystem developer myself for many years, until quite recently. It's the nature of the beast. The problem is how many of those bugs remain latent. All the common-case bugs are likely to be found and fixed pretty quickly, but that only provides a false sense of security. Without good test coverage, those latent less-common-case bugs start to reappear every time anything changes - as seems to have been the case here. That actually slows the development of new features, so even the MOAR FEECHURS crowd end up getting burned. Good testing is worth it, and users alone don't do good testing.
- m45t3r 8y ago
  I think filesystems in kernel have automated tests, I know xfstests [1] exist, at least. And they exist exactly because filesystems bugs generally are critical: a filesystems bug generally means that someone will lose data. [1]: different from what the name may suggest, xfstests is run in other filesystems too. Here is an example of xfstests ported to ZoL (ZFS on Linux): https://github.com/zfsonlinux/xfstests https://github.com/zfsonlinux/xfstests
  - notacoward 8y ago
    Yes, xfstests exists. I've used it myself and it's actually pretty good for a suite that doesn't include error injection. But part of today's story is that xfstests wasn't being updated to cover the last several features that were added to XFS. The result is exactly the kind of brittleness that is characteristic of poorly tested software. Something else changed, and previously latent bugs started popping out of the rotten woodwork.

by m45t3r 8y ago

notacoward 8y ago

> tests for every possible configuration, that is basically impossible anyway Agreed. Expecting 100% across the entire kernel would be totally unreasonable. OTOH, coverage could be better on a lot of components considered individually. Any component as complex as XFS is going to have tons of bugs. I don't mean that as an insult. I was a filesystem developer myself for many years, until quite recently. It's the nature of the beast. The problem is how many of those bugs remain latent. All the common-case bugs are likely to be found and fixed pretty quickly, but that only provides a false sense of security. Without good test coverage, those latent less-common-case bugs start to reappear every time anything changes - as seems to have been the case here. That actually slows the development of new features, so even the MOAR FEECHURS crowd end up getting burned. Good testing is worth it, and users alone don't do good testing.

m45t3r 8y ago
I think filesystems in kernel have automated tests, I know xfstests [1] exist, at least. And they exist exactly because filesystems bugs generally are critical: a filesystems bug generally means that someone will lose data. [1]: different from what the name may suggest, xfstests is run in other filesystems too. Here is an example of xfstests ported to ZoL (ZFS on Linux): https://github.com/zfsonlinux/xfstests https://github.com/zfsonlinux/xfstests
- notacoward 8y ago
  Yes, xfstests exists. I've used it myself and it's actually pretty good for a suite that doesn't include error injection. But part of today's story is that xfstests wasn't being updated to cover the last several features that were added to XFS. The result is exactly the kind of brittleness that is characteristic of poorly tested software. Something else changed, and previously latent bugs started popping out of the rotten woodwork.