5 ms·

So.... This is ksplice, then? It sounds hardcore, but this isn't really that complicated is it? Load a module with a patched function and then modify whatever

So.... This is ksplice, then? It sounds hardcore, but this isn't really that complicated is it? Load a module with a patched function and then modify whatever the kernel equiv of the GOT/PLT is, and boom -- hotpatch... Right? Oracle absorbed the kSplice thing though, and I assume the IP around it -- so unless this does something different to that general procedure, then is there a possibility of oracle doing their usual crap here? This is more of a toy than anything though, I recon... In all my years of adopting and trying to unfuck horribly outdated systems (not by choice) -- I've never once really wanted or needed to hotpatch a kernel.... You don't solve your poor architecture decisions by hotpatching running kernels... If your situation is so bad that a kernel update is the only solution, and you can't take a box down to do that without an outage on the platform -- then you need to rethink what you're doing or hire someone who can do that for you..

onli 10y ago
That's a bit critical, isn't it? Just think of servers. You want to reboot them as seldom as possible, to be online. But you also want to have critical security patches (or well, just all security patches) applied as fast as possible. It is already very easy to run unattended upgrades, and a common practice. Maybe not in a high-class startup or big enterprise, but everywhere else. With livepatching the kernel you get the kernel security patches applied immediately, which is very very useful and will make the linux server more secure in general. And even if your security practices are perfect, livepatching is still (potentially) easier and faster and thus a good addition. > if your situation is so bad that a kernel update is the only solution I don't get that part, btw. In every situation you always need to update the kernel when security issues are detected.
- glandium 10y ago
  > I don't get that part, btw. In every situation you always need to update the kernel when security issues are detected. Not the GP, but he means that if you can't take downtime on a server to reboot with an updated kernel, then you have bigger problems than the kernel not being up-to-date. Because instead of a scheduled reboot, the server could crash, lose its disks, lose its power, etc. And if you can't take a scheduled reboot, how do you take those?
  - Xiol 10y ago
    Got RAID and UPS, bruh. Norton on the box. Basically invulnerable.
    - cyberpunk 10y ago
      Raid? Pah! Putting your main tablespaces on ramdrives makes them go like 300x faster!! Redundancy is for cowards, just like unit tests... </s>
- cyberpunk 10y ago
  That's totally correct, and I can't disagree with you.. If you architect things properly, then rebooting any one component shouldn't be a problem.. In such a case though, there is no need for this tool.. In practice, though, things really don't happen all nice like that. I've got loads of internal boxes with holes. Terrible? Yeah, absolutely. We're all ashamed. These aren't a real problem security wise though, if you take a second to think about it... Say I have a box I can't bounce (or rather, won't bounce) to get a patch for some bug -- probably it'll be some DB server in a legacy env, or something so fragile that there's a massive risk in bouncing it, vs the risk of some DOS attack which could only happen from inside the network then usually, we just won't do it... Sure, that's terrible practice and not something anyone should admit, but it's the reality... Environments like this, the attack surface is the JVM and OpenSSH.. Assuming an attacker can get through our CDN, then through our rproxies (which we do seriously patch), after that blindly break out of a JVM on an ASLR'd linux box, THEN look to pwn a kernel hole on the DB server we've not patched, then do you really think patching is going to save you? It's a trade off; and it's all tech debt many teams are working to completely replace -- this stuff is horrible, but sometimes you have to put up with it until you can replace it.. To me it seems the attack surface isn't really the OS anymore -- it's the app. More likely whoever is hitting you is going to rip your DB off via some dodgy post sanitation bug than remotely pwning your internal boxes with kernel exploits -- so this does little to protect anyone..
fragmede 10y ago
Sun tried to do it for Solaris in the 80's, Microsoft tried to do it for Vista, and they both failed, so it might be a little more complicated than "modify whatever the kernel equiv of the GOT/PLT is" (hint: if you don't even know the names of things, would it be possible for the problem to be a bit more involved than you realize?) Remember when Heartbleed came out, and the ensuing panic if you hadn't gotten advanced disclosure? Instead of that, how about if all your Linux servers weren't vulnerable but in exchange, you give money to Oracle. Ksplice now patches userspace as well.
- cyberpunk 10y ago
  Sure the technical details are more involved, I wasn't trying to downplay the cleverness of the tech; but the point stands -- the logic is pretty simple and AFAIK that is how kSplice works it's not black magic or anything.. Some function has an issue, a module is loaded with a replacement, and instead of calling the orig function anymore, all calls go to the one we just loaded -- via some brain surgery... Heartbleed and userspace issues such as that are a much wider topic, but ok.. sure.. I can pay someone for easy access to updates that I can apply without downtime.. I think we're missing the point though.. Tools such as ksplice and livepatch seem to pander to environments which are unable to handle the sort of patch maintenence we've been doing as long as we've had patches available.. I've seen many of these in the wild, and what we needed to do with them wasn't patch them online but sort the infrastructure out to the point where we could handle regular maintenance.. Maybe someone really needs this sort of functionality; I struggle to think of a reason besides poor architecture though.
AdamGibbins 10y ago
> Oracle absorbed the kSplice thing though, and I assume the IP around it -- so unless this does something different to that general procedure, then is there a possibility of oracle doing their usual crap here? The live patching stuff is part of kernel core https://lkml.org/lkml/2014/11/6/387 https://lkml.org/lkml/2014/11/6/387 The Ubuntu proprietary stuff is creating the patches and distributing them.

onli 10y ago

That's a bit critical, isn't it? Just think of servers. You want to reboot them as seldom as possible, to be online. But you also want to have critical security patches (or well, just all security patches) applied as fast as possible. It is already very easy to run unattended upgrades, and a common practice. Maybe not in a high-class startup or big enterprise, but everywhere else. With livepatching the kernel you get the kernel security patches applied immediately, which is very very useful and will make the linux server more secure in general. And even if your security practices are perfect, livepatching is still (potentially) easier and faster and thus a good addition. > if your situation is so bad that a kernel update is the only solution I don't get that part, btw. In every situation you always need to update the kernel when security issues are detected.

glandium 10y ago
> I don't get that part, btw. In every situation you always need to update the kernel when security issues are detected. Not the GP, but he means that if you can't take downtime on a server to reboot with an updated kernel, then you have bigger problems than the kernel not being up-to-date. Because instead of a scheduled reboot, the server could crash, lose its disks, lose its power, etc. And if you can't take a scheduled reboot, how do you take those?
- Xiol 10y ago
  Got RAID and UPS, bruh. Norton on the box. Basically invulnerable.
  - cyberpunk 10y ago
    Raid? Pah! Putting your main tablespaces on ramdrives makes them go like 300x faster!! Redundancy is for cowards, just like unit tests... </s>
cyberpunk 10y ago
That's totally correct, and I can't disagree with you.. If you architect things properly, then rebooting any one component shouldn't be a problem.. In such a case though, there is no need for this tool.. In practice, though, things really don't happen all nice like that. I've got loads of internal boxes with holes. Terrible? Yeah, absolutely. We're all ashamed. These aren't a real problem security wise though, if you take a second to think about it... Say I have a box I can't bounce (or rather, won't bounce) to get a patch for some bug -- probably it'll be some DB server in a legacy env, or something so fragile that there's a massive risk in bouncing it, vs the risk of some DOS attack which could only happen from inside the network then usually, we just won't do it... Sure, that's terrible practice and not something anyone should admit, but it's the reality... Environments like this, the attack surface is the JVM and OpenSSH.. Assuming an attacker can get through our CDN, then through our rproxies (which we do seriously patch), after that blindly break out of a JVM on an ASLR'd linux box, THEN look to pwn a kernel hole on the DB server we've not patched, then do you really think patching is going to save you? It's a trade off; and it's all tech debt many teams are working to completely replace -- this stuff is horrible, but sometimes you have to put up with it until you can replace it.. To me it seems the attack surface isn't really the OS anymore -- it's the app. More likely whoever is hitting you is going to rip your DB off via some dodgy post sanitation bug than remotely pwning your internal boxes with kernel exploits -- so this does little to protect anyone..

fragmede 10y ago

Sun tried to do it for Solaris in the 80's, Microsoft tried to do it for Vista, and they both failed, so it might be a little more complicated than "modify whatever the kernel equiv of the GOT/PLT is" (hint: if you don't even know the names of things, would it be possible for the problem to be a bit more involved than you realize?) Remember when Heartbleed came out, and the ensuing panic if you hadn't gotten advanced disclosure? Instead of that, how about if all your Linux servers weren't vulnerable but in exchange, you give money to Oracle. Ksplice now patches userspace as well.

cyberpunk 10y ago
Sure the technical details are more involved, I wasn't trying to downplay the cleverness of the tech; but the point stands -- the logic is pretty simple and AFAIK that is how kSplice works it's not black magic or anything.. Some function has an issue, a module is loaded with a replacement, and instead of calling the orig function anymore, all calls go to the one we just loaded -- via some brain surgery... Heartbleed and userspace issues such as that are a much wider topic, but ok.. sure.. I can pay someone for easy access to updates that I can apply without downtime.. I think we're missing the point though.. Tools such as ksplice and livepatch seem to pander to environments which are unable to handle the sort of patch maintenence we've been doing as long as we've had patches available.. I've seen many of these in the wild, and what we needed to do with them wasn't patch them online but sort the infrastructure out to the point where we could handle regular maintenance.. Maybe someone really needs this sort of functionality; I struggle to think of a reason besides poor architecture though.

AdamGibbins 10y ago

> Oracle absorbed the kSplice thing though, and I assume the IP around it -- so unless this does something different to that general procedure, then is there a possibility of oracle doing their usual crap here? The live patching stuff is part of kernel core https://lkml.org/lkml/2014/11/6/387 https://lkml.org/lkml/2014/11/6/387 The Ubuntu proprietary stuff is creating the patches and distributing them.