8 ms·

What happens when packages go bad?

74 points by jaffathecake 8y ago

hartator 8y ago
No real solution in there. Running a hacked package time to time is just rthe cost of doing business.
canadev 8y ago
> It feels like it has a small attack surface, yet we use over 1700 packages. We only have 49 direct dependencies, so the vast majority of these are dependencies of dependencies of dependencies etc etc. The web app itself doesn't use a lot of third party code, so most of these packages are part of the build system. Wow. Small attack surface with 1700 dependencies. What a hard concept to swallow.
- blattimwind 8y ago
  "We only have 49 direct dependencies"
  - randallsquared 8y ago
    Yeah, I work in a bunch of node.js codebases, and 49 deps seems pretty high, unless they're including development environment deps.
    - untog 8y ago
      Presumably they are, since he talks about the possibility of developer machines being attacked.
      - randallsquared 8y ago
        Well, regular dependencies get installed on developer machines as well, so I'm not sure that follows.
        untog 8y ago
        Well why wouldn't he be including them in a conversation that includes developer machines?
    - mcguire 8y ago
      Did you look at the application? Image manipulation on the client.
    - jaffathecake 8y ago
      Almost exclusively build stuff https://github.com/GoogleChromeLabs/squoosh/blob/master/package.json#L17 https://github.com/GoogleChromeLabs/squoosh/blob/master/pack...
slededit 8y ago
As a C++ developer I can’t imagine 1700 dependencies in a relatively new code base. What are all those libraries doing?
- canadev 8y ago
  I think the answer is everything in your standard library plus some of the stuff you would use 3rd party dependencies for in your own project. Maybe standard libraries need to get bigger for JS.
  - slededit 8y ago
    Even in C with its tiny standard library you don’t see this kind of explosion. There has to be more to it than that.
    - bfred_it 8y ago
      npm handles sub-dependencies with ease so people aren't afraid to pull in smaller dependencies. Also since the code has to be downloaded (or uploaded to small lambdas, for example), JS developers are generally weary of depending on large libraries.
      - krapp 8y ago
        >Also since the code has to be downloaded (or uploaded to small lambdas, for example), JS developers are generally weary of depending on large libraries. Isn't a dependency tree of thousands of small, interdependent libraries essentially just a large, distributed library? Is there really a benefit to that versus compiling those dependencies to a single file, especially given how many NPM packages are just a single small function? The end result in many cases would probably be smaller than a lot of old JQuery plugins.
        eeZah7Ux 8y ago
        > Isn't a dependency tree of thousands of small, interdependent libraries essentially just a large, distributed library? Yes and it's a thousand times worse than a single, large library.
lozenge 8y ago
Commit your dependencies and don't update them. The chances of your site actually exposing a security issue or bug that comes from one of your dependencies is near zero, unless it's an actual security-relevant dependency (sanitize-html, etc).
- bfred_it 8y ago
  That's one way to bloat your repo. There's no reason to commit your dependencies if your intent is security. Just use the lock file. The packages won't change on npm and you'll always get the same version for all sub-dependencies.
  - izacus 8y ago
    Doesn't JavaScript world know anything like Java Maven proxy servers? Which retrieve artifacts on first lookup and cache them forever?
    - canadev 8y ago
      Another solution is to check them in as git Large File Storage files. https://git-lfs.github.com/ https://git-lfs.github.com/
    - vvillena 8y ago
      That's what I was thinking about. The Java ecosystem solved this problem 20 years ago!
      - Silhouette 8y ago
        It probably is fair to say that the state-of-the-art tools for JS development today are at least 1-2 decades behind the state-of-the-art for programming more generally. Trying to build larger applications in JS is a relatively new thing, and a community full of people who are enthusiastic and keen to do new things but often lacking in experience with larger and more long-lived software systems is getting a crash course in what does and doesn't work at that scale. Give it another 5-10 years and/or the more dramatic replacement of JS with better languages if new possibilities like WebAssembly change the landscape, and hopefully more of the experience will transfer over and the tools will catch up.
perpetualcrayon 8y ago
I say force the new maintainer to fork and then leave a note in your README that the currently maintained version is (here). This way anyone who decides to trust the new maintainer will be able to act as a "canary in the coal mine", notifying others if they run into issues. This also delays the gratification to the new maintainer. If they're truly malicious they'll need to spend maybe months / years maintaining the code / fixing bugs until they'll be able to hit pay dirt. I think most malicious devs will not want to pay this price. EDIT: This would also act as a window for (a) folks to find other alternatives for their projects, and (b) inspire folks to build alternative options. I think the willingness to fork over access to widely used packages isn't just a reflection on your desire to move on from the project, but it also reflects your blatant disregard of the thousands, maybe millions, of people who depend on what you've built.
- ATsch 8y ago
  The problem here is that often, there's not just the github repo, but also the non-namespaced language package manager, e.g. npm, pypi. You'd have to transfer that at some point. I'm also unsure how you can notify your users... I have not looked at the readme of some dependencies for years.
  - perpetualcrayon 8y ago
    I'd say force a fork there too. It's not difficult to switch to a new package once you find it and know the URL. I say force each user (whether it's end user or library maintainer) to switch manually to the new maintainer's fork of your project. "I have not looked at the readme...". Maybe this will create a market for a new type of project. The one that lets folks know the status of the packages they use in their project.
    - mcguire 8y ago
      jquery, jquery-forked, new-jquery, forked-jquery, my-jquery, this-is-the-real-jquery, potato-headed-rat-bastard-jquery, ...
menzoic 8y ago
Mirror dependencies and lock them down...
dblotsky 8y ago
Lock the dependencies: checksums, lock file, “=“ versions, whatever. Even better, store them yourself: cache, mirror, zips in the repo, whatever. This won’t protect you from an initial hack, but it means that if the dependency was good when you got it, it will stay good.
- eeZah7Ux 8y ago
  No. People deploying code must be able to manage vulnerabilities in dependencies by updating them.
  - danShumway 8y ago
    Are you sure you're thinking about this from a Javascript perspective? Vendoring dependencies on a Node project doesn't make it easier or harder for me to update those dependencies later. I run the same commands from the command line either way. It also doesn't make it harder for me to update dependencies of dependencies either -- quite the opposite; I wouldn't advise people to do this, but in an emergency if my dependencies are being tracked via source control rather than fresh-downloaded on every install, I can make security tweaks or change their dependencies, and those changes will actually stick between installs. Even on a website, vendoring my dependencies doesn't mean my dependencies need to be a single blob -- I can still have a dependency deployed as a separately updatable file. Is there a specific scenario you're thinking of where vendoring a JS dependency would prevent someone from updating it?
  - dblotsky 8y ago
    In the scenario I proposed, nothing stops you from updating your dependencies. It just doesn't happen without your knowledge and consent (unlike the default NPM settings).

74 points by jaffathecake 8y ago

No real solution in there. Running a hacked package time to time is just rthe cost of doing business.

> It feels like it has a small attack surface, yet we use over 1700 packages. We only have 49 direct dependencies, so the vast majority of these are dependencies of dependencies of dependencies etc etc. The web app itself doesn't use a lot of third party code, so most of these packages are part of the build system. Wow. Small attack surface with 1700 dependencies. What a hard concept to swallow.

blattimwind 8y ago
"We only have 49 direct dependencies"
- randallsquared 8y ago
  Yeah, I work in a bunch of node.js codebases, and 49 deps seems pretty high, unless they're including development environment deps.
  - untog 8y ago
    Presumably they are, since he talks about the possibility of developer machines being attacked.
    - randallsquared 8y ago
      Well, regular dependencies get installed on developer machines as well, so I'm not sure that follows.
      - untog 8y ago
        Well why wouldn't he be including them in a conversation that includes developer machines?
  - mcguire 8y ago
    Did you look at the application? Image manipulation on the client.
  - jaffathecake 8y ago
    Almost exclusively build stuff https://github.com/GoogleChromeLabs/squoosh/blob/master/package.json#L17 https://github.com/GoogleChromeLabs/squoosh/blob/master/pack...

slededit 8y ago

As a C++ developer I can’t imagine 1700 dependencies in a relatively new code base. What are all those libraries doing?

canadev 8y ago
I think the answer is everything in your standard library plus some of the stuff you would use 3rd party dependencies for in your own project. Maybe standard libraries need to get bigger for JS.
- slededit 8y ago
  Even in C with its tiny standard library you don’t see this kind of explosion. There has to be more to it than that.
  - bfred_it 8y ago
    npm handles sub-dependencies with ease so people aren't afraid to pull in smaller dependencies. Also since the code has to be downloaded (or uploaded to small lambdas, for example), JS developers are generally weary of depending on large libraries.
    - krapp 8y ago
      >Also since the code has to be downloaded (or uploaded to small lambdas, for example), JS developers are generally weary of depending on large libraries. Isn't a dependency tree of thousands of small, interdependent libraries essentially just a large, distributed library? Is there really a benefit to that versus compiling those dependencies to a single file, especially given how many NPM packages are just a single small function? The end result in many cases would probably be smaller than a lot of old JQuery plugins.
      - eeZah7Ux 8y ago
        > Isn't a dependency tree of thousands of small, interdependent libraries essentially just a large, distributed library? Yes and it's a thousand times worse than a single, large library.

lozenge 8y ago

Commit your dependencies and don't update them. The chances of your site actually exposing a security issue or bug that comes from one of your dependencies is near zero, unless it's an actual security-relevant dependency (sanitize-html, etc).

bfred_it 8y ago
That's one way to bloat your repo. There's no reason to commit your dependencies if your intent is security. Just use the lock file. The packages won't change on npm and you'll always get the same version for all sub-dependencies.
- izacus 8y ago
  Doesn't JavaScript world know anything like Java Maven proxy servers? Which retrieve artifacts on first lookup and cache them forever?
  - canadev 8y ago
    Another solution is to check them in as git Large File Storage files. https://git-lfs.github.com/ https://git-lfs.github.com/
  - vvillena 8y ago
    That's what I was thinking about. The Java ecosystem solved this problem 20 years ago!
    - Silhouette 8y ago
      It probably is fair to say that the state-of-the-art tools for JS development today are at least 1-2 decades behind the state-of-the-art for programming more generally. Trying to build larger applications in JS is a relatively new thing, and a community full of people who are enthusiastic and keen to do new things but often lacking in experience with larger and more long-lived software systems is getting a crash course in what does and doesn't work at that scale. Give it another 5-10 years and/or the more dramatic replacement of JS with better languages if new possibilities like WebAssembly change the landscape, and hopefully more of the experience will transfer over and the tools will catch up.

perpetualcrayon 8y ago

I say force the new maintainer to fork and then leave a note in your README that the currently maintained version is (here). This way anyone who decides to trust the new maintainer will be able to act as a "canary in the coal mine", notifying others if they run into issues. This also delays the gratification to the new maintainer. If they're truly malicious they'll need to spend maybe months / years maintaining the code / fixing bugs until they'll be able to hit pay dirt. I think most malicious devs will not want to pay this price. EDIT: This would also act as a window for (a) folks to find other alternatives for their projects, and (b) inspire folks to build alternative options. I think the willingness to fork over access to widely used packages isn't just a reflection on your desire to move on from the project, but it also reflects your blatant disregard of the thousands, maybe millions, of people who depend on what you've built.

ATsch 8y ago
The problem here is that often, there's not just the github repo, but also the non-namespaced language package manager, e.g. npm, pypi. You'd have to transfer that at some point. I'm also unsure how you can notify your users... I have not looked at the readme of some dependencies for years.
- perpetualcrayon 8y ago
  I'd say force a fork there too. It's not difficult to switch to a new package once you find it and know the URL. I say force each user (whether it's end user or library maintainer) to switch manually to the new maintainer's fork of your project. "I have not looked at the readme...". Maybe this will create a market for a new type of project. The one that lets folks know the status of the packages they use in their project.
  - mcguire 8y ago
    jquery, jquery-forked, new-jquery, forked-jquery, my-jquery, this-is-the-real-jquery, potato-headed-rat-bastard-jquery, ...

menzoic 8y ago

Mirror dependencies and lock them down...

dblotsky 8y ago

Lock the dependencies: checksums, lock file, “=“ versions, whatever. Even better, store them yourself: cache, mirror, zips in the repo, whatever. This won’t protect you from an initial hack, but it means that if the dependency was good when you got it, it will stay good.

eeZah7Ux 8y ago
No. People deploying code must be able to manage vulnerabilities in dependencies by updating them.
- danShumway 8y ago
  Are you sure you're thinking about this from a Javascript perspective? Vendoring dependencies on a Node project doesn't make it easier or harder for me to update those dependencies later. I run the same commands from the command line either way. It also doesn't make it harder for me to update dependencies of dependencies either -- quite the opposite; I wouldn't advise people to do this, but in an emergency if my dependencies are being tracked via source control rather than fresh-downloaded on every install, I can make security tweaks or change their dependencies, and those changes will actually stick between installs. Even on a website, vendoring my dependencies doesn't mean my dependencies need to be a single blob -- I can still have a dependency deployed as a separately updatable file. Is there a specific scenario you're thinking of where vendoring a JS dependency would prevent someone from updating it?
- dblotsky 8y ago
  In the scenario I proposed, nothing stops you from updating your dependencies. It just doesn't happen without your knowledge and consent (unlike the default NPM settings).