4 ms·

Linux is arguably one of the most important software projects on the planet right now. It is vital in so many aspects of society now, Android, AWS/AZure, IOT,

Linux is arguably one of the most important software projects on the planet right now. It is vital in so many aspects of society now, Android, AWS/AZure, IOT, routers, etc etc. It should be taken as a serious given that bad actors are working hard, with huge budgets to nefariously gain some advantage in the development of Linux. They will have far better engineers, much larger budges, and have been working on it for many years. I would imagine ever intelligence service on the planet would love to have obscure exploits embedded in the kernel. I can easily imagine a scenario where a single exploit was introduced over many months one tiny bit at a time. Each patch a fully working fix for unrelated problem. Fully Legit. But when they are all combined something wonderful happens. (for the nefarious player). This stunt was a few students who were not well funded nor were they uber kernel coders. Yet they had some success (depending on what you call success). I imagine the NSA has a team on this, with some exceedingly well skilled kernel coders working on embedding things, and I am sure they have a second team to find possibly implants by foreign services and so on. Had this been security researchers who managed to get changes into the WindowsNT kernel I think the reaction by the core Linux devs would have been different. (and yes I am 100% there are people spending just as much time doing this for Windows as for Linux, well maybe not). This highlighted a vulnerability, which is frankly quite real and needs to be taken seriously and contemplated over time. The guys were jerks, yes. What they highlighted is real and a lot more serious than what this little stunt did.

zamadatix 6y ago
I think the article raised a good point: "One final lesson that one might be tempted to take is that the kernel is running a terrible risk of malicious patches inserted by actors with rather more skill and resources than the UMN researchers have shown. That could be, but the simple truth of the matter is that regular kernel developers continue to insert bugs at such a rate that there should be little need for malicious actors to add more. " How big/useful a revelation is it that you can show it's easy to break into someone's house while they are sleeping if it's well known the front door won't even stay shut.
- kbenson 6y ago
  It's not that the front door won't stay shut, it's that different portals to the house seem to be locked or unlocked or just plain open sometimes at random intervals. There is benefit to an attacker to having a known ingress available on demand. There's a difference between a bug, a potentially exploitable bug, and an exploited bug. There's also a big difference between what's known and what's not yet known. Just because kernel developers are busy inadvertently introducing bugs doesn't mean that those bugs are all equally useful as an exploit. And that's before even considering that some bugs are easier to identify than others. A sophisticated adversary being able to get an exploitable bug into the kernel that they already have an exploit built for skips the discovery and exploit development phases, while also ensuring that the exploit delivers the type of access desired, rather than depending on luck. This is definitely something we should be thinking about.
ec109685 6y ago
It is interesting that there isn’t more emphasis on requiring automated tests with Kernel changed. It’s clearly impossible to verify that every change is safe, yet test coverage isn’t part of patch acceptance criteria. There’s likely more coverage on a typical website.
- cyphar 6y ago
  I broadly agree that tests should be part of the patch acceptance criteria for core kernel features. That's a no-brainer these days and with kunit it's not even that hard to write kernel unit tests. I will say that there are a fair few test/fuzzing suites for the kernel (xfstests, selftests, kunit, the Intel 0day finder, syzcaller, etc). The last two I would argue are probably more likely to find exploits than unit and integration tests. There also has been a push in recent years to have better test frameworks for the kernel. (The reason I'd like more integration tests is because sometimes new kernel functionality is added without a strong enough specification of what it's behaviour should be -- writing tests tends to cause people to consider those corner cases.) However one issue is that the vast majority of the code in the kernel is for drivers. It seems to me that it'd impractical to even attempt to mock hardware (since drivers also have to work around hardware bugs), so you'd only be able to do fairly rudimentary unit testing for the bulk of the code in the kernel. Modern websites don't have embedded drivers.

zamadatix 6y ago

I think the article raised a good point: "One final lesson that one might be tempted to take is that the kernel is running a terrible risk of malicious patches inserted by actors with rather more skill and resources than the UMN researchers have shown. That could be, but the simple truth of the matter is that regular kernel developers continue to insert bugs at such a rate that there should be little need for malicious actors to add more. " How big/useful a revelation is it that you can show it's easy to break into someone's house while they are sleeping if it's well known the front door won't even stay shut.

kbenson 6y ago
It's not that the front door won't stay shut, it's that different portals to the house seem to be locked or unlocked or just plain open sometimes at random intervals. There is benefit to an attacker to having a known ingress available on demand. There's a difference between a bug, a potentially exploitable bug, and an exploited bug. There's also a big difference between what's known and what's not yet known. Just because kernel developers are busy inadvertently introducing bugs doesn't mean that those bugs are all equally useful as an exploit. And that's before even considering that some bugs are easier to identify than others. A sophisticated adversary being able to get an exploitable bug into the kernel that they already have an exploit built for skips the discovery and exploit development phases, while also ensuring that the exploit delivers the type of access desired, rather than depending on luck. This is definitely something we should be thinking about.

ec109685 6y ago

It is interesting that there isn’t more emphasis on requiring automated tests with Kernel changed. It’s clearly impossible to verify that every change is safe, yet test coverage isn’t part of patch acceptance criteria. There’s likely more coverage on a typical website.

cyphar 6y ago
I broadly agree that tests should be part of the patch acceptance criteria for core kernel features. That's a no-brainer these days and with kunit it's not even that hard to write kernel unit tests. I will say that there are a fair few test/fuzzing suites for the kernel (xfstests, selftests, kunit, the Intel 0day finder, syzcaller, etc). The last two I would argue are probably more likely to find exploits than unit and integration tests. There also has been a push in recent years to have better test frameworks for the kernel. (The reason I'd like more integration tests is because sometimes new kernel functionality is added without a strong enough specification of what it's behaviour should be -- writing tests tends to cause people to consider those corner cases.) However one issue is that the vast majority of the code in the kernel is for drivers. It seems to me that it'd impractical to even attempt to mock hardware (since drivers also have to work around hardware bugs), so you'd only be able to do fairly rudimentary unit testing for the bulk of the code in the kernel. Modern websites don't have embedded drivers.