5 ms·

This is such a poorly written article: * no detailed analysis of how the attack was undertaken. Its not even clear how the attacker managed to get in (was it a

by pm90 8y ago

This is such a poorly written article: * no detailed analysis of how the attack was undertaken. Its not even clear how the attacker managed to get in (was it a publicly exposed Jenkins? vulnerable bastion? what?) * no analysis of what the existing matrix.org security perimeter looked like or how it could be made better. * repetition of security tropes. Use VPN. Use Github Enterprise (wait wtf? Why not private repos in Github?). Don't use Ansible, use salt. Ridiculous. I was looking forward to a nice long read about how this breach was undertaken. Hugely disappointed.

bifrost 8y ago
If you click through to the GH Issues I linked to there are some pretty good data points as to what happened. I didn't feel the need to copypasta. But yes, publicly exposed jenkins and repos lead to the compromise, not an uncommon story unfortunately. Perimeter - I didn't see much evidence of one existing and I didn't go probing their networks to find out. Security tropes are real for a reason, you don't have to believe me though. Private repos in GitHub are still publicly hosted and are orders of magnitude easier to get into than having an in perimeter repo. They've leaked before and they'll keep on leaking. GitHub even made it harder for people to fork private repos to their own public accounts but it still happens.
- pm90 8y ago
  > They've leaked before and they'll keep on leaking. GitHub even made it harder for people to fork private repos to their own public accounts but it still happens Can you provide some actual instances of this happening? Genuinely curious, as my org is currently migrating from enterprise to cloud.
  - bifrost 8y ago
    I've mostly seen this reported in forums and during discussion, if you Google around you'll find some pretty useful hits. Here's a good one from reddit: https://www.reddit.com/r/github/comments/9odnvw/someone_forked_a_private_repo_and_made_it_public/ https://www.reddit.com/r/github/comments/9odnvw/someone_fork... Its also discussed reasonably well in the infosec community. Basically GitHub is a great place to find other people's passwords and API keys.
    - baroffoos 8y ago
      Thats unrelated to github though. It sounds like the person did a git clone and then created a new repo and pushed it. You could do that with a self hosted git repo as well. To stop that you would have to have your git server block logins from non company machines and have some serious logging on all company machines to stop anyone moving it off via usb
    - bifrost 8y ago
      baroffoos: Thats pretty close to what I'm suggesting, no public repo access. It works.
- bobwaycott 8y ago
  > But yes, publicly exposed jenkins and repos lead to the compromise … You mean the past-tense verb led, not its metallic homonym lead. :)
netsectoday 8y ago
* this idiot claimed "Ansible was used to keep the attacker in the system" which in all reality Ansible did what it was supposed to by altering the correct authorized_keys file and the attacker leveraged an old default in the sshd config. This is a sshd config issue, not Ansible. The sales-pitch for Salt (against Ansible) is ridiculous and misguided. I just checked out the Salt SSH module and even if they used salt they would still have this issue. Then answer here is to not use the default /etc/ssh/sshd_config value of #AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2. Uncomment and remove authorized_keys2.
- viralpoetry 8y ago
  Why are you downvoted? Your Ansible explanation is indeed correct. It's SSH default problem not deployment toolkit one.
nobatron 8y ago
There's a lot wrong with this article. Firstly having a private network for your infrastructure isn't a one stop solution for keeping attackers out. Secondly using Github Enterprise or self hosted GitLab doesn't make up for storing secrets in Git. Looking forwards to the proper write up.
- bifrost 8y ago
  I've never claimed it was a "one stop", but it certainly keeps the random internet users to a minimum. And yes, using GHE or self hosted GitLab doesn't make up for storing secrets, but it at least keeps them out of the public eye so the effects are less brutal. Its still bad to store secrets in a code repository. My whole point is that you can reduce risks easily, yet some people don't for some reason.

by pm90 8y ago

bifrost 8y ago

If you click through to the GH Issues I linked to there are some pretty good data points as to what happened. I didn't feel the need to copypasta. But yes, publicly exposed jenkins and repos lead to the compromise, not an uncommon story unfortunately. Perimeter - I didn't see much evidence of one existing and I didn't go probing their networks to find out. Security tropes are real for a reason, you don't have to believe me though. Private repos in GitHub are still publicly hosted and are orders of magnitude easier to get into than having an in perimeter repo. They've leaked before and they'll keep on leaking. GitHub even made it harder for people to fork private repos to their own public accounts but it still happens.

pm90 8y ago
> They've leaked before and they'll keep on leaking. GitHub even made it harder for people to fork private repos to their own public accounts but it still happens Can you provide some actual instances of this happening? Genuinely curious, as my org is currently migrating from enterprise to cloud.
- bifrost 8y ago
  I've mostly seen this reported in forums and during discussion, if you Google around you'll find some pretty useful hits. Here's a good one from reddit: https://www.reddit.com/r/github/comments/9odnvw/someone_forked_a_private_repo_and_made_it_public/ https://www.reddit.com/r/github/comments/9odnvw/someone_fork... Its also discussed reasonably well in the infosec community. Basically GitHub is a great place to find other people's passwords and API keys.
  - baroffoos 8y ago
    Thats unrelated to github though. It sounds like the person did a git clone and then created a new repo and pushed it. You could do that with a self hosted git repo as well. To stop that you would have to have your git server block logins from non company machines and have some serious logging on all company machines to stop anyone moving it off via usb
  - bifrost 8y ago
    baroffoos: Thats pretty close to what I'm suggesting, no public repo access. It works.
bobwaycott 8y ago
> But yes, publicly exposed jenkins and repos lead to the compromise … You mean the past-tense verb led, not its metallic homonym lead. :)

netsectoday 8y ago

* this idiot claimed "Ansible was used to keep the attacker in the system" which in all reality Ansible did what it was supposed to by altering the correct authorized_keys file and the attacker leveraged an old default in the sshd config. This is a sshd config issue, not Ansible. The sales-pitch for Salt (against Ansible) is ridiculous and misguided. I just checked out the Salt SSH module and even if they used salt they would still have this issue. Then answer here is to not use the default /etc/ssh/sshd_config value of #AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2. Uncomment and remove authorized_keys2.

viralpoetry 8y ago
Why are you downvoted? Your Ansible explanation is indeed correct. It's SSH default problem not deployment toolkit one.

nobatron 8y ago

There's a lot wrong with this article. Firstly having a private network for your infrastructure isn't a one stop solution for keeping attackers out. Secondly using Github Enterprise or self hosted GitLab doesn't make up for storing secrets in Git. Looking forwards to the proper write up.

bifrost 8y ago
I've never claimed it was a "one stop", but it certainly keeps the random internet users to a minimum. And yes, using GHE or self hosted GitLab doesn't make up for storing secrets, but it at least keeps them out of the public eye so the effects are less brutal. Its still bad to store secrets in a code repository. My whole point is that you can reduce risks easily, yet some people don't for some reason.