5 ms·

I’m cackling at how great this is. This is what happens when you trust the internet forever and just scarf down any old thing at build time. Of course it’ll ge

I’m cackling at how great this is. This is what happens when you trust the internet forever and just scarf down any old thing at build time. Of course it’ll get exploited! That’s what evil people do.

greggman3 6y ago
There's more coming.... tons of github integrations ask for blanket access to your account vs Oauth, (https://github.com/marketplace https://github.com/marketplace). Tons of github users give that access, the access_tokens are only a password type breach away. If you have these access_tokens you can edit the repos they are for all you want.
- PartiallyTyped 6y ago
  Different access tokens have different permissions, you can't just do whatever you want.
  - tomashubelbauer 6y ago
    Yeah, but at least with PATs (not sure about other token types), you can't scope them to a particular repo, so whenever you need to allow something to even see a private repo or write to a public repo, the token you supply to allow that can do that for all repos and that alone is potentially really destructive. I am not sure if there is a good reason for why PATs can't be scoped to a repository, because if they were allowed to be, it would do a lot for security I think.
    - RyJones 6y ago
      Worse: you can’t even scope it to an org, and some integration points are only available to org owners! GitHub is terrible.
      - seg_lol 6y ago
        That explains why at a place I worked we had multiple github orgs for different subteams, and/or they were really cheap ... yeah confused now.
        RyJones 6y ago
        We also have multiple orgs, but we hit the requirement that you only have one bot account. It would be super nice if GitHub allowed much tighter scoping for PATs. I don’t know how many GitHub orgs the Linux Foundation has, but... hundreds? Having one bit account with wide permissions is a non starter
  - greggman3 6y ago
    Most integrations just ask for blanket all permissions. They do this because it means they can give you a list of repos and let you choose which ones to integrate their service into with no work on your part except "click yes to give us permission to do everything for you and ... we'll do everything for you"
- leetrout 6y ago
  Apparently the Oauth scopes are much worse than GitHubs apps. Only GitHub apps allow read only access to the “metadata” by default whereas Oauth apps get access to the code, deploy keys, etc with no way to limit that access per repo. https://docs.github.com/en/developers/apps/scopes-for-oauth-apps https://docs.github.com/en/developers/apps/scopes-for-oauth-... https://docs.github.com/en/rest/reference/permissions-required-for-github-apps https://docs.github.com/en/rest/reference/permissions-requir...
- gardaani 6y ago
  At the moment, there's a story about github1s.com on the front page of HN and people are asking how to give it access to their company private repos [1][2]. Scary. [1] https://news.ycombinator.com/item?id=26087017 https://news.ycombinator.com/item?id=26087017 [2] https://news.ycombinator.com/item?id=26086789 https://news.ycombinator.com/item?id=26086789
- prepend 6y ago
  I wish GitHub would create a proper auth design. I won’t grant blanket permissions to tokens because there’s too much risk of something going wrong. It seems dumb that they don’t have per repo tokens. I think the issue is with their licensing as if they made proper tokens users could abuse it by giving tokens to their friends. But this should be detectable in a friendly (please don’t do that) way. I want to be able to give read-only access to private repos. I want to be able to give fine grained function level and repo level access. If I’m an admin on multiple repos, I want to be able to issue a token for just a single repo so I can give that to a CI job without worrying if every single repo I admin is at risk. They allow ssh keys with some similar functionality, but ssh keys can’t be used as much as tokens. I’ve been waiting for a story about how some third party app granted access to my whole org gets taken over and wreaks havoc. Eventually this will probably be the attack that alters real packages instead of these name overloading packages.
  - 0xbadcafebee 6y ago
    > It seems dumb that they don’t have per repo tokens. Technically you can create one new GitHub account per repo and generate a token for that... But that is highly annoying :) They need to support IAM / RBAC style policies and tie every authn+z method to those policies, but my guess is they have different auth methods strung all throughout their codebase so implementing it will take a few years. Then of course they have to make it "user friendly" as we all know how painful IAM can be...
    - prepend 6y ago
      Comically, that’s why my GitHub recommended. Of course that’s a nightmare for a user to manage, violates our sso requirement, and GitHub charges per user.
      - leetrout 6y ago
        We do this at work. And you pay per user per team so we’re paying double or triple in some cases by managing access via teams.
geek_at 6y ago
I'm more amazed by the fact that they got bounties because the attack wouldn't be (easily) possible without insider knowledge on which dependencies their internal build system used
- knowhy 6y ago
  > To test this hypothesis, Birsan began hunting for names of private internal packages that he could find in manifest files on GitHub repositories or in CDNs of prominent companies but did not exist in a public open-source repository. If I'm not mistaken insider knowledge wasn't necessary.
  - geek_at 6y ago
    my mistake
peteretep 6y ago
You say this, but I feel like in the 20-odd years I’ve been using package managers I’ve seen very very few real world exploits?
- hanselot 6y ago
  Why would the ones getting away with it bother publishing articles about it? Rather, 20 years is a plenty long enough time to cover your tracks.
- Cthulhu_ 6y ago
  Consider yourself lucky I guess? I mean in the 20-odd years I've been driving I've never had an accident either.
- dspillett 6y ago
  While the risk is low, the potential impact of a successful exploit is massive so the matter should be taken seriously. Remember how much was temporarily broken in the leftPad event? Imagine if all that had been silently back-doored instead?
CaptArmchair 6y ago
There are a lot of expensive things you can outsource. Responsibility isn't among those. Free software / open source propels engineering as you can share and leverage the results of collective efforts. However, at no point did the concept come with inherent guarantees about concerns such as security. esr defined 19 points for "good" open source software development in his seminal essay "The Cathedral and the Bazaar". I feel some of those are sometimes easily thrown out of the window for the sake of "efficiency" or "cost-effectiveness". This issue resonates with bullet point 17 in particular: > A security system is only as secure as its secret. Beware of pseudo-secrets. I think this issue has less to do with package managers, and a lot with companies rushing into the convenience of public code platforms such as Github without properly vetting whether or not they might be inadvertently leaking internal information through packaging manifests. https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar
- iib 6y ago
  Offtopic, but I found nowhere to actually ask this question. Does anybody know if ESR is still alive? His blog [1] has not been updated in months--and looking at his post dates, this seems really out of character--, he hasn't posted anything on twitter, or his usual channels. [1] http://esr.ibiblio.org/ http://esr.ibiblio.org/
  - knowhy 6y ago
    This [0] is from Jan 2021. [0]: https://lists.nongnu.org/archive/html/gpsd-dev/2021-01/msg00059.html https://lists.nongnu.org/archive/html/gpsd-dev/2021-01/msg00...
  - tyingq 6y ago
    https://github.com/eric-s-raymond https://github.com/eric-s-raymond Scroll down to contribution activity. Forked a repo, created a pull request, etc, on Feb 1st 2021.
- syshum 6y ago
  >>There are a lot of expensive things you can outsource. Responsibility isn't among those. That is not true at all, the industry both Development and even more so in Operations has been outsourcing responsibility for a long time, they is why we have support contracts, SLA's and other very expensive services we pay many many times more than the cost of hardware for... To outsource responsibility... Network down -- Call Cisco... Storage Down Call EMC or Nimble... etc
walrus01 6y ago
Hey let's just sudo curl | bash what could possibly go wrong?
- thombles 6y ago
  Not sure if serious, but I will point out this is significantly different. If I'm installing an application like homebrew or the Rust toolchain then I am explicitly giving them the right to code execution. It doesn't much matter whether they get it through the script on their website or the binaries downloaded from that website. Random libraries, possibly pulled in by a dependency of a dependency of a dependency... not so much.
  - hda2 6y ago
    I know this is besides the point, but rust can be installed - without piping the output of a public-facing server into a privileged shell - by using their offline installer: https://forge.rust-lang.org/infra/other-installation-methods.html#standalone https://forge.rust-lang.org/infra/other-installation-methods... They offer PGP signatures too!
- da_big_ghey 6y ago
  That's curl | sudo bash smh what an amateur
- beermonster 6y ago
  You even see Microsoft offering those types of install one-liners. e.g. curl -sSL https://dot.net/v1/dotnet-install.sh https://dot.net/v1/dotnet-install.sh | bash /dev/stdin <additional install-script args> See https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-in... There are other examples I've seen from time to time.
beermonster 6y ago
And wonder how long it will take for similar repeats on other repos such as dockerhub, apt, nuget, homebrew etc etc.
christophilus 6y ago
Honestly, it’s one of the things that makes me nervous about running Linux on all of my computers. At least with Windows (and probably OSX), my updates come from a single vendor who has strict internal code audits and security requirements. With Linux (I’m using Pop), my updates come from a package manager with a crapload of packages, each maintained by a different team / group with no central policy. There’s no way the small team at Pop can review and audit all of the things in the apt package system, and there have to be plenty of maintainers of popular packages who get sweet offers to sell out. Anyway. I’m sticking with Pop / Linux. But it does make me nervous!
- jonny_eh 6y ago
  For Linux, it's about choosing a distro you can trust. Most manage their own repos, some more carefully than others.
- danhor 6y ago
  At least with windows, the drivers aren't checked that much and accordingly, have had some serious issues. I'd guess distros are generally better off in that respect, but kernel space & user space aren't that different nowadays, when caring about your own security
tyingq 6y ago
Also happy. I'm very happy to finally have a real world example to motivate all the folks that eye-rolled me every time I've raised it in the past. It just resonates better, especially with less technical leadership folks.

greggman3 6y ago

There's more coming.... tons of github integrations ask for blanket access to your account vs Oauth, (https://github.com/marketplace https://github.com/marketplace). Tons of github users give that access, the access_tokens are only a password type breach away. If you have these access_tokens you can edit the repos they are for all you want.

PartiallyTyped 6y ago
Different access tokens have different permissions, you can't just do whatever you want.
- tomashubelbauer 6y ago
  Yeah, but at least with PATs (not sure about other token types), you can't scope them to a particular repo, so whenever you need to allow something to even see a private repo or write to a public repo, the token you supply to allow that can do that for all repos and that alone is potentially really destructive. I am not sure if there is a good reason for why PATs can't be scoped to a repository, because if they were allowed to be, it would do a lot for security I think.
  - RyJones 6y ago
    Worse: you can’t even scope it to an org, and some integration points are only available to org owners! GitHub is terrible.
    - seg_lol 6y ago
      That explains why at a place I worked we had multiple github orgs for different subteams, and/or they were really cheap ... yeah confused now.
      - RyJones 6y ago
        We also have multiple orgs, but we hit the requirement that you only have one bot account. It would be super nice if GitHub allowed much tighter scoping for PATs. I don’t know how many GitHub orgs the Linux Foundation has, but... hundreds? Having one bit account with wide permissions is a non starter
- greggman3 6y ago
  Most integrations just ask for blanket all permissions. They do this because it means they can give you a list of repos and let you choose which ones to integrate their service into with no work on your part except "click yes to give us permission to do everything for you and ... we'll do everything for you"
leetrout 6y ago
Apparently the Oauth scopes are much worse than GitHubs apps. Only GitHub apps allow read only access to the “metadata” by default whereas Oauth apps get access to the code, deploy keys, etc with no way to limit that access per repo. https://docs.github.com/en/developers/apps/scopes-for-oauth-apps https://docs.github.com/en/developers/apps/scopes-for-oauth-... https://docs.github.com/en/rest/reference/permissions-required-for-github-apps https://docs.github.com/en/rest/reference/permissions-requir...
gardaani 6y ago
At the moment, there's a story about github1s.com on the front page of HN and people are asking how to give it access to their company private repos [1][2]. Scary. [1] https://news.ycombinator.com/item?id=26087017 https://news.ycombinator.com/item?id=26087017 [2] https://news.ycombinator.com/item?id=26086789 https://news.ycombinator.com/item?id=26086789
prepend 6y ago
I wish GitHub would create a proper auth design. I won’t grant blanket permissions to tokens because there’s too much risk of something going wrong. It seems dumb that they don’t have per repo tokens. I think the issue is with their licensing as if they made proper tokens users could abuse it by giving tokens to their friends. But this should be detectable in a friendly (please don’t do that) way. I want to be able to give read-only access to private repos. I want to be able to give fine grained function level and repo level access. If I’m an admin on multiple repos, I want to be able to issue a token for just a single repo so I can give that to a CI job without worrying if every single repo I admin is at risk. They allow ssh keys with some similar functionality, but ssh keys can’t be used as much as tokens. I’ve been waiting for a story about how some third party app granted access to my whole org gets taken over and wreaks havoc. Eventually this will probably be the attack that alters real packages instead of these name overloading packages.
- 0xbadcafebee 6y ago
  > It seems dumb that they don’t have per repo tokens. Technically you can create one new GitHub account per repo and generate a token for that... But that is highly annoying :) They need to support IAM / RBAC style policies and tie every authn+z method to those policies, but my guess is they have different auth methods strung all throughout their codebase so implementing it will take a few years. Then of course they have to make it "user friendly" as we all know how painful IAM can be...
  - prepend 6y ago
    Comically, that’s why my GitHub recommended. Of course that’s a nightmare for a user to manage, violates our sso requirement, and GitHub charges per user.
    - leetrout 6y ago
      We do this at work. And you pay per user per team so we’re paying double or triple in some cases by managing access via teams.

geek_at 6y ago

I'm more amazed by the fact that they got bounties because the attack wouldn't be (easily) possible without insider knowledge on which dependencies their internal build system used

knowhy 6y ago
> To test this hypothesis, Birsan began hunting for names of private internal packages that he could find in manifest files on GitHub repositories or in CDNs of prominent companies but did not exist in a public open-source repository. If I'm not mistaken insider knowledge wasn't necessary.
- geek_at 6y ago
  my mistake

peteretep 6y ago

You say this, but I feel like in the 20-odd years I’ve been using package managers I’ve seen very very few real world exploits?

hanselot 6y ago
Why would the ones getting away with it bother publishing articles about it? Rather, 20 years is a plenty long enough time to cover your tracks.
Cthulhu_ 6y ago
Consider yourself lucky I guess? I mean in the 20-odd years I've been driving I've never had an accident either.
dspillett 6y ago
While the risk is low, the potential impact of a successful exploit is massive so the matter should be taken seriously. Remember how much was temporarily broken in the leftPad event? Imagine if all that had been silently back-doored instead?

CaptArmchair 6y ago

There are a lot of expensive things you can outsource. Responsibility isn't among those. Free software / open source propels engineering as you can share and leverage the results of collective efforts. However, at no point did the concept come with inherent guarantees about concerns such as security. esr defined 19 points for "good" open source software development in his seminal essay "The Cathedral and the Bazaar". I feel some of those are sometimes easily thrown out of the window for the sake of "efficiency" or "cost-effectiveness". This issue resonates with bullet point 17 in particular: > A security system is only as secure as its secret. Beware of pseudo-secrets. I think this issue has less to do with package managers, and a lot with companies rushing into the convenience of public code platforms such as Github without properly vetting whether or not they might be inadvertently leaking internal information through packaging manifests. https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar

iib 6y ago
Offtopic, but I found nowhere to actually ask this question. Does anybody know if ESR is still alive? His blog [1] has not been updated in months--and looking at his post dates, this seems really out of character--, he hasn't posted anything on twitter, or his usual channels. [1] http://esr.ibiblio.org/ http://esr.ibiblio.org/
- knowhy 6y ago
  This [0] is from Jan 2021. [0]: https://lists.nongnu.org/archive/html/gpsd-dev/2021-01/msg00059.html https://lists.nongnu.org/archive/html/gpsd-dev/2021-01/msg00...
- tyingq 6y ago
  https://github.com/eric-s-raymond https://github.com/eric-s-raymond Scroll down to contribution activity. Forked a repo, created a pull request, etc, on Feb 1st 2021.
syshum 6y ago
>>There are a lot of expensive things you can outsource. Responsibility isn't among those. That is not true at all, the industry both Development and even more so in Operations has been outsourcing responsibility for a long time, they is why we have support contracts, SLA's and other very expensive services we pay many many times more than the cost of hardware for... To outsource responsibility... Network down -- Call Cisco... Storage Down Call EMC or Nimble... etc

walrus01 6y ago

Hey let's just sudo curl | bash what could possibly go wrong?

thombles 6y ago
Not sure if serious, but I will point out this is significantly different. If I'm installing an application like homebrew or the Rust toolchain then I am explicitly giving them the right to code execution. It doesn't much matter whether they get it through the script on their website or the binaries downloaded from that website. Random libraries, possibly pulled in by a dependency of a dependency of a dependency... not so much.
- hda2 6y ago
  I know this is besides the point, but rust can be installed - without piping the output of a public-facing server into a privileged shell - by using their offline installer: https://forge.rust-lang.org/infra/other-installation-methods.html#standalone https://forge.rust-lang.org/infra/other-installation-methods... They offer PGP signatures too!
da_big_ghey 6y ago
That's curl | sudo bash smh what an amateur
beermonster 6y ago
You even see Microsoft offering those types of install one-liners. e.g. curl -sSL https://dot.net/v1/dotnet-install.sh https://dot.net/v1/dotnet-install.sh | bash /dev/stdin <additional install-script args> See https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-in... There are other examples I've seen from time to time.

beermonster 6y ago

And wonder how long it will take for similar repeats on other repos such as dockerhub, apt, nuget, homebrew etc etc.

christophilus 6y ago

Honestly, it’s one of the things that makes me nervous about running Linux on all of my computers. At least with Windows (and probably OSX), my updates come from a single vendor who has strict internal code audits and security requirements. With Linux (I’m using Pop), my updates come from a package manager with a crapload of packages, each maintained by a different team / group with no central policy. There’s no way the small team at Pop can review and audit all of the things in the apt package system, and there have to be plenty of maintainers of popular packages who get sweet offers to sell out. Anyway. I’m sticking with Pop / Linux. But it does make me nervous!

jonny_eh 6y ago
For Linux, it's about choosing a distro you can trust. Most manage their own repos, some more carefully than others.
danhor 6y ago
At least with windows, the drivers aren't checked that much and accordingly, have had some serious issues. I'd guess distros are generally better off in that respect, but kernel space & user space aren't that different nowadays, when caring about your own security

tyingq 6y ago

Also happy. I'm very happy to finally have a real world example to motivate all the folks that eye-rolled me every time I've raised it in the past. It just resonates better, especially with less technical leadership folks.