6 ms·

Kernel.org has been hacked (see site news)

167 points by KonradKlause 15y ago

rhygar 15y ago
Thank god for Git.
- mkr-hn 15y ago
  I don't do any development and I've still heard about git's reputation. edit: Why is this being downvoted?
  - aaronblohowiak 15y ago
    because it doesn't make me smarter to have read it.
    - mkr-hn 15y ago
      I pointed out that git's good reputation is known even outside developer circles. It might not be the greatest contribution ever, but is it really worth -x?
- patrickaljord 15y ago
  Thank Linus for Git.
  - dredmorbius 15y ago
    But you repeat rhygar. "My name is Linus Torvalds and I am your god." - Linus Torvalds http://en.wikiquote.org/wiki/Linus_Torvalds http://en.wikiquote.org/wiki/Linus_Torvalds
iBercovich 15y ago
This is a valid argument against open source operative systems running top clearance environments such as the military / police / government agencies. If this modifications had gone undetected for a few months, it's possible that the compromised code could have made it into a lot of critical systems.I am a Linux user, but I remember this being a Microsoft argument in the past for promoting their OS to run in government agencies.
- ars 15y ago
  Did you actually read the page? Because it says exactly the opposite. It says that it would be impossible to modify the source without hundreds of people noticing immediately.
- mkr-hn 15y ago
  Proprietary codebases aren't fortresses. edit: And what ars said. edit: And the chorus.
- lgeek 15y ago
  > we currently believe that the source code repositories were unaffected So for now it looks like no compromised code has been distributed.
  - recoiledsnake 15y ago
    kernel.org hosts more than the GIT repo. http://www.google.com/search?client=opera&rls=en&q=download+linux+kernel&sourceid=opera&ie=utf-8&oe=utf-8 http://www.google.com/search?client=opera&rls=en&q=d...
- mindstab 15y ago
  no modifications to the code appear to have been made. Like they said, that would be hard/impossible because it's all signed off by Linus in git, so ever if they compromised the server it gets them nothing. They'd then have to compromise some accounts and submit patches and still get them approved. This argument is completely bogus. I could just as easily have happened to any one else including Microsoft, and in those cases we might not even have heard about it. It already has happened repeatedly to some hardware vendors where an actual payload was injected into their drivers, and they weren't open source. Between open source and git it's dramatically more likely an injected payload would be detected long before dissemination could take place.
  - kevindication 15y ago
    Go with "impossible." Gov't agencies don't just upgrade every time a new kernel comes out.
zobzu 15y ago
id like to point out that while you can't modify tagged+gpg signed tag commits, you can modify non signed commits (aka any commit that isnt a tag) and it'll get signed when it gets tagged. that's also assuming that tags are gpg signed, which they often are not,even on kernel.org non signed can be actually tampered with and they do need to check the code thoroughly with 3rd party older archives which is a PITA. Code signing > x, including per commit signing mr Linus T and GIT maintainers. Note that they were against per commit signing because "when you sign the tag, you sign everything so its ok". Except you wont read all the patches you sign when you sign the tag, and if any has been modified, as explained, you don't know. Again, per commit signing solves it. Food for though I guess.
- ajross 15y ago
  It's not a question about whether you read them; code review isn't the mechanism. The modified commits will have different hashes and thus cause merge failures for anyone that already has them in a local repo. At the very least, that's going to include Linus and everyone downstream as far as the original author. None of these people need to do anything to notice the tampering beyond trying to merge from mainline. That may not be perfect, but it sounds pretty robust to me.
  - zobzu 15y ago
    it does not matter, you can add a new commit (or modify an old one anyway since it'll work fine - but lets say you notice it, so we add a commit) you make the commit look legit and all, its going to be pretty difficult to say where its coming from and which has been tampered with, since, of course, linus himself will have it when he pulls it. and that's not just linux.git, but also the ton of other .git's in there, which linus will pull from and will not read every line of. The caveat here is as real as it can get. usually when it looks "slightly complex" people overlook the security aspect "no one is going to do such an attack" then the attack happens, often unnoticed, because the attackers aren't lazy if there's a hole they use it.
    - ajross 15y ago
cooldeal 15y ago
>However, it's also useful to note that the potential damage of cracking kernel.org is far less than typical software repositories What about code that's hosted on kernel.org itself? Isn't kernel.org a source for the public to get the kernel and not git? http://www.kernel.org/pub/ http://www.kernel.org/pub/ ftp://ftp.kernel.org/pub/ rsync://rsync.kernel.org/pub/ It would be easy for the exploiter to insert trojaned/rootkitted kernels into those places.
- KonradKlause 15y ago
  Currently all files get verified. The kernel.org admins have digital signatures of all files...
  - Torn 15y ago
    Hopefully those signatures aren't on the compromised machines?
    - KonradKlause 15y ago
      Of course not. As one of the users@kernel.org members I can tell you that the kernel.org admins are very competent.
plainOldText 15y ago
If the security at kernel.org has been breached, then I don't know what to think about me, the little guy - who runs a small VPS - or about the small start-ups out there. I guess having secure systems is not that easy after all, and people overlook this or fail to allocate enough resources to this process.
- comex 15y ago
  The bigger you are, the more servers you have to manage, the more services might provide a way in, the more users have access and might have their credentials stolen... it's actually much easier to be secure when you're small.
  - waitwhat 15y ago
    The real money-quote from the original article was: We are currently working with the 448 users of kernel.org to change their credentials and change their SSH keys. With 448 users, something like this was inevitable...
    - Dylan16807 15y ago
      I wouldn't call a root exploit inevitable just from user count.
- sushantsharma 15y ago
  FTA: "We believe they may have gained this access via a compromised user credential". I hope this is true, and the access was not gained via some unknown vulnerability.
  - potatolicious 15y ago
    It sounds like they got in via a regular account and somehow escalated themselves to root: "how they managed to exploit that to root access is currently unknown and is being investigated." Simple social engineering of mundane user accounts is not worrisome. Escalating any old user into root is.
deleted 15y ago
[deleted]
bugsy 15y ago
I have a question. They note they have notified officials. I've tried notifying the FBI after our sites have been hacked, but they are not interested in it. Is there some other place to report? Does anyone really investigate this stuff?
- naner 15y ago
  Does anyone really investigate this stuff? Not unless there's a ton of money involved or some important asset is at risk. This type of thing happens much more frequently than they can manage so they prioritize by severity.
- xyzzyz 15y ago
  I don't know what it's like in the US, but in my home country you're supposed to report it to the police -- just like you report the burglary or mugging.
- Goladus 15y ago
  From a global security standpoint, kernel.org is an important site. Hacking the authoritative source for the linux kernel has serious implications for anyone who runs linux, which is lots of people probably including law enforcement. Note that, as mentioned on the site, actually doing this is much harder than simply rooting the servers.
- sp332 15y ago
  It's useful to make a report as soon as you notice something wrong, even if they don't do anything right away. If it turns out to be more serious than you (or they) first thought, it's nice to have a timestamp on a detailed report from when things first looked funny.
waffle_ss 15y ago
From a pure git perspective, the attacker(s) will have a very hard time rewriting any commit history, as changing a commit's SHA1 hash will trigger a cascading effect on the hashes of all child commits on down the DAG from the one that they change. Anyone with an existing clone of the repos would immediately know the repo has been corrupted when doing a pull (although fresh clones wouldn't be as lucky of course).
mtrnx 15y ago
if md5 of files exist then there seems no prob.
copper 15y ago
Anyone know how this affects the ArchLinux packages? kernel.org is still the fastest mirror around for them.
- naner 15y ago
  Probably not at all (the Arch userbase isn't much of a target). That being said, pacman doesn't implement package-signing last time I checked so it certainly would be easier to pass off bogus packages than with other distros.
  - crocowhile 15y ago
    Pacman now supports package signature verification (it'svery recent news.)
dnagir 15y ago
I wonder what was the reason behind this?

167 points by KonradKlause 15y ago

rhygar 15y ago

Thank god for Git.

mkr-hn 15y ago
I don't do any development and I've still heard about git's reputation. edit: Why is this being downvoted?
- aaronblohowiak 15y ago
  because it doesn't make me smarter to have read it.
  - mkr-hn 15y ago
    I pointed out that git's good reputation is known even outside developer circles. It might not be the greatest contribution ever, but is it really worth -x?
patrickaljord 15y ago
Thank Linus for Git.
- dredmorbius 15y ago
  But you repeat rhygar. "My name is Linus Torvalds and I am your god." - Linus Torvalds http://en.wikiquote.org/wiki/Linus_Torvalds http://en.wikiquote.org/wiki/Linus_Torvalds

iBercovich 15y ago

This is a valid argument against open source operative systems running top clearance environments such as the military / police / government agencies. If this modifications had gone undetected for a few months, it's possible that the compromised code could have made it into a lot of critical systems.I am a Linux user, but I remember this being a Microsoft argument in the past for promoting their OS to run in government agencies.

ars 15y ago
Did you actually read the page? Because it says exactly the opposite. It says that it would be impossible to modify the source without hundreds of people noticing immediately.
mkr-hn 15y ago
Proprietary codebases aren't fortresses. edit: And what ars said. edit: And the chorus.
lgeek 15y ago
> we currently believe that the source code repositories were unaffected So for now it looks like no compromised code has been distributed.
- recoiledsnake 15y ago
  kernel.org hosts more than the GIT repo. http://www.google.com/search?client=opera&rls=en&q=download+linux+kernel&sourceid=opera&ie=utf-8&oe=utf-8 http://www.google.com/search?client=opera&rls=en&q=d...
mindstab 15y ago
no modifications to the code appear to have been made. Like they said, that would be hard/impossible because it's all signed off by Linus in git, so ever if they compromised the server it gets them nothing. They'd then have to compromise some accounts and submit patches and still get them approved. This argument is completely bogus. I could just as easily have happened to any one else including Microsoft, and in those cases we might not even have heard about it. It already has happened repeatedly to some hardware vendors where an actual payload was injected into their drivers, and they weren't open source. Between open source and git it's dramatically more likely an injected payload would be detected long before dissemination could take place.
- kevindication 15y ago
  Go with "impossible." Gov't agencies don't just upgrade every time a new kernel comes out.

zobzu 15y ago

id like to point out that while you can't modify tagged+gpg signed tag commits, you can modify non signed commits (aka any commit that isnt a tag) and it'll get signed when it gets tagged. that's also assuming that tags are gpg signed, which they often are not,even on kernel.org non signed can be actually tampered with and they do need to check the code thoroughly with 3rd party older archives which is a PITA. Code signing > x, including per commit signing mr Linus T and GIT maintainers. Note that they were against per commit signing because "when you sign the tag, you sign everything so its ok". Except you wont read all the patches you sign when you sign the tag, and if any has been modified, as explained, you don't know. Again, per commit signing solves it. Food for though I guess.

ajross 15y ago
It's not a question about whether you read them; code review isn't the mechanism. The modified commits will have different hashes and thus cause merge failures for anyone that already has them in a local repo. At the very least, that's going to include Linus and everyone downstream as far as the original author. None of these people need to do anything to notice the tampering beyond trying to merge from mainline. That may not be perfect, but it sounds pretty robust to me.
- zobzu 15y ago
  it does not matter, you can add a new commit (or modify an old one anyway since it'll work fine - but lets say you notice it, so we add a commit) you make the commit look legit and all, its going to be pretty difficult to say where its coming from and which has been tampered with, since, of course, linus himself will have it when he pulls it. and that's not just linux.git, but also the ton of other .git's in there, which linus will pull from and will not read every line of. The caveat here is as real as it can get. usually when it looks "slightly complex" people overlook the security aspect "no one is going to do such an attack" then the attack happens, often unnoticed, because the attackers aren't lazy if there's a hole they use it.
  - ajross 15y ago

cooldeal 15y ago

>However, it's also useful to note that the potential damage of cracking kernel.org is far less than typical software repositories What about code that's hosted on kernel.org itself? Isn't kernel.org a source for the public to get the kernel and not git? http://www.kernel.org/pub/ http://www.kernel.org/pub/ ftp://ftp.kernel.org/pub/ rsync://rsync.kernel.org/pub/ It would be easy for the exploiter to insert trojaned/rootkitted kernels into those places.

KonradKlause 15y ago
Currently all files get verified. The kernel.org admins have digital signatures of all files...
- Torn 15y ago
  Hopefully those signatures aren't on the compromised machines?
  - KonradKlause 15y ago
    Of course not. As one of the users@kernel.org members I can tell you that the kernel.org admins are very competent.

plainOldText 15y ago

If the security at kernel.org has been breached, then I don't know what to think about me, the little guy - who runs a small VPS - or about the small start-ups out there. I guess having secure systems is not that easy after all, and people overlook this or fail to allocate enough resources to this process.

comex 15y ago
The bigger you are, the more servers you have to manage, the more services might provide a way in, the more users have access and might have their credentials stolen... it's actually much easier to be secure when you're small.
- waitwhat 15y ago
  The real money-quote from the original article was: We are currently working with the 448 users of kernel.org to change their credentials and change their SSH keys. With 448 users, something like this was inevitable...
  - Dylan16807 15y ago
    I wouldn't call a root exploit inevitable just from user count.
sushantsharma 15y ago
FTA: "We believe they may have gained this access via a compromised user credential". I hope this is true, and the access was not gained via some unknown vulnerability.
- potatolicious 15y ago
  It sounds like they got in via a regular account and somehow escalated themselves to root: "how they managed to exploit that to root access is currently unknown and is being investigated." Simple social engineering of mundane user accounts is not worrisome. Escalating any old user into root is.

deleted 15y ago

[deleted]

bugsy 15y ago

I have a question. They note they have notified officials. I've tried notifying the FBI after our sites have been hacked, but they are not interested in it. Is there some other place to report? Does anyone really investigate this stuff?

naner 15y ago
Does anyone really investigate this stuff? Not unless there's a ton of money involved or some important asset is at risk. This type of thing happens much more frequently than they can manage so they prioritize by severity.
xyzzyz 15y ago
I don't know what it's like in the US, but in my home country you're supposed to report it to the police -- just like you report the burglary or mugging.
Goladus 15y ago
From a global security standpoint, kernel.org is an important site. Hacking the authoritative source for the linux kernel has serious implications for anyone who runs linux, which is lots of people probably including law enforcement. Note that, as mentioned on the site, actually doing this is much harder than simply rooting the servers.
sp332 15y ago
It's useful to make a report as soon as you notice something wrong, even if they don't do anything right away. If it turns out to be more serious than you (or they) first thought, it's nice to have a timestamp on a detailed report from when things first looked funny.

waffle_ss 15y ago

From a pure git perspective, the attacker(s) will have a very hard time rewriting any commit history, as changing a commit's SHA1 hash will trigger a cascading effect on the hashes of all child commits on down the DAG from the one that they change. Anyone with an existing clone of the repos would immediately know the repo has been corrupted when doing a pull (although fresh clones wouldn't be as lucky of course).

mtrnx 15y ago

if md5 of files exist then there seems no prob.

copper 15y ago

Anyone know how this affects the ArchLinux packages? kernel.org is still the fastest mirror around for them.

naner 15y ago
Probably not at all (the Arch userbase isn't much of a target). That being said, pacman doesn't implement package-signing last time I checked so it certainly would be easier to pass off bogus packages than with other distros.
- crocowhile 15y ago
  Pacman now supports package signature verification (it'svery recent news.)

dnagir 15y ago

I wonder what was the reason behind this?