6 ms·
Linux Kernel Defence Map
- EdSchouten 5y agoNotice how “unit testing coverage” is not on this map.
- ylyn 5y agoUnit tests are not the only way, or the best way, to test software.
- Dylan16807 5y agoSure? The note is that it is not a way here.
- egberts 5y agoBecause a kernel is not a user-land?
- zibzab 5y agoI agree with parent that unit testing privileged bare metal code is not always possible, at least not without spending 10x-100x dev time on writing tests. Example: anything that directly accesses MMU or caches.
- rurban 5y agoOn the contrary, simulating MMU or caches are trivial. You can create a memory subsystem mock in a few hours. IO, assembly and interrupts are much harder.
- zibzab 5y agoHow would you create a mock object for something that modifies the memory layout of the unit test while it is running?
- Dylan16807 5y ago> How would you create a mock object for something that modifies the memory layout of the unit test while it is running? There's two ways I can interpret "the unit test". 1) "How would you create a mock object for modifying the memory layout of the code being tested?" That's just a routine that shuffles around some bytes. Or marks parts of the address space to fault so it can simulate the contents. 2) "How would you create a mock object for modifying the memory layout of the code performing the test?" That wouldn't be a test. The code performing the test isn't inside its own simulation.
- rurban 5y agoYou create your mmap's on init. The MMU is just a simple 3way array lookup, as a Unicode table.
- yencabulator 5y agoUnicode?
- Too 5y agoPerhaps there was an /s missing in parent post? Implying that unit testing is often prioritized more by management, over checking security, configuration and tools like this one.
- deleted 5y ago[deleted]
- phendrenad2 5y agoI can just imagine the technical and political difficulties in adding proper unit tests to the kernel now. The C model is "everything just imports and calls whatever it wants". It's great for speed, but for good unit testing you actually want some of that "AbstractFactoryFactory" stuff we make fun of in Java. For instance, take any module of the Linux kernel and try to see what it would take to test it without importing any other kernel code. Then, if you have a plan for that, now you need to stub out the rest of the kernel. Since every module imports dozens of files from other modules, and calls dozens of functions from said modules, it's not very pretty.
- blueflow 5y agoUnit testing is generally not useful for bare-metal code, since it depends more on the actual device behavior than the behavior documented in the datasheet. Creating an accurate mock is, sadly, often just not possible with the documentation available. And with the quality of mocks being bad, you end up testing the datasheets against the devices instead of testing the kernel code against the devices.
- nerdponx 5y agoSerious question: how many CVEs are due to bugs in code that falls into this hard-to-test category?
- rurban 5y agoThat's exactly the mindset why baremetal code is so horribly buggy. Creating a mock is still better than not creating a mock to test against. With the mock you can create simulations, unit tests, coverage tests, static tests, formal verification. Without mock you can just test on actual HW a few selected test scenarios you think of. Emulation: The lack of qemu support for arm chips is mindboggling. avr/atmel is quite good in contrast. renode is far better.
- blueflow 5y ago> Creating a mock is still better than not creating a mock to test against. With the mock you can create simulations, unit tests, coverage tests, static tests, formal verification. You did not understand what i wrote. With hardware, its just damn difficult to get accurate mocks. If the mock is wrong, all tests relying on it are not meaningful. If you ask me, QEMU doesn't even properly support emulating x86. Its just "good enough" to run common operating systems. You could try improving the support, maybe then you will realize how just damn difficult and undocumented this shit is, and you would understand why nobody wants to hear your crappy unit test evangelism.
- rurban 5y agoWell, my company is happy to test baremetal fw on sw, as other better companies do also. Ditching tests is a lousy argument, only coming from people who have no idea how to test difficult targets. Of course you have to patch qemu, gdb, renode and valgrind if something is missing. Eg the recent apple M1. But this is only needed for emulation, and we are talking simulation here.
- GuB-42 5y agoUnit testing coverage is a development practice, not really a defense. Testing, coverage, static analysis, methodologies, etc... are a way to make prevent things from going wrong in the first place, defenses are here to protect you if things go wrong. We need both. The article focuses only on the "defense" part, the development practices could make another article. Note that testing and defenses can go together: test to make sure all paths are taken before shipping, and see if defenses trigger. Fuzzing in particular takes that approach.
- staticassertion 5y agoUnsurprisingly Grsecurity/Pax pretty much dominate this map, especially when you consider how many in-tree mitigations are just cherry picked from grsec. I find the approach taken by this map interesting. I've attempted before to categorize attacks and defenses in terms of graphs, there's interesting opportunities to improve defender's abilities to better understand attack surface, contextualize CVEs (is this exploitable in my env?), etc.
- icare_1er 5y agoI am no kernel-ninja and my experience is limited to a few kprints just to not be a complete noob. My question to the knowledgeable people would be: has the linux kernel become too big and too complex to maintain ? By that i mean, like for an organization, when most of the cost/efforts, are spent on just maintaining/understanding stuff.
- zibzab 5y agoThe core of the kernel is actually pretty small and very well understood and maintained. If your company needs to write drivers for some hardware you produce, there are often specific subsystems to target with have their own maintainers and documentation. Where things get really ugly is when a company decides to cook their own linux platform and they change tons of stuff without always understanding what they are doing. I guess early android would be a good example of that...
- robinsoh 5y ago> The core of the kernel is actually pretty small and very well understood and maintained. I still find that even Linux based OSes like Ubuntu crash at least once a month or so. I've had to boot into initrd and manually do fsck on my laptop once this year. I'm not a software guy but I assume the reason for all this crappiness (windows as well) is because of all the patents on operating system stuff by Microsoft and Google. I'm saying this slightly tongue in cheek beacuse as a display guy I frequently see software people say display technologies suck because of all the patents so I assume that's what they experience in their own industry?
- ungamedplayer 5y agoMaybe you are conflating the gui crashing with the kernel, or maybe it's just shitty hardware you have. I'd be up to my neck with more work if it was generally true.
- modshatereality 5y ago
- david_draco 5y agoMost security features are disabled by default (PAX, grsecurity, SELinux), and when distributions enable them, they are the first to go when admins encounter problems. It is then a chicken-and-egg problem when software fails in secure environments, those failures are not reported/fixed upstream but circumvented, and consequently security cannot be enabled across the board because tools break. I'd be curious if open source software would become more robust if by default the popular CI systems would enable features that do not cause any false positives but cause crashes in buggy software (memory randomization and read guards, compiler flags for detecting stack overflows).
- totony 5y agoThis is because grsec is proprietary and the state of SELinux policies is very poor. Redhat puts a lot of effort into maintaining their SELinux policy and is the only one that doesn't have too much bugs. PaX should be included, but afaik a lot of what PaX has been doing over the years is being/has been included in mainline.