9 ms·

The casual culture of pulling in hundred of dependencies, and mushrooming language specific package managers is ridiculously insecure and has to go. There is n

• reificator 9y ago
  NPM is my primary reason for not using NodeJS. I installed a specific package in an empty project and got over 690 dependencies. For running `npm install {package-name}`. What's worse is that I skimmed the tree to check for anything particularly heinous, but there was nothing that stood out as unneeded. With such a tiny stdlib, especially out of the browser environment, there's not really a better alternative than to make it easy to include dependencies for your dependencies. Without NPM, the Node community would be tiny if not already dead. I don't have a better idea either. For my purposes, it just means I use a different language, but that's not really a solution.
  • zizek23 9y ago
    A package manager must at a minimum address code security, dependency hell and deployment. A json parser that wgets urls into a folder is not a package manager. These things must be thought of upfront and cannot be left vague and open to adhoc practices that leak to end users and deployment creating insecurity. For instance a package should be a package, it should not be a simple class or function that in turn pulls in a hundred dependencies. 10 packages like this can pull in 600 packages all with conflicting versions. This is exactly dependency hell that creates insecure practice as no one can verify the chain and sheer number of packages. Language developers cannot just sit back and let this happen. Eventually you will pay the price for this kind of shoddy engineering. This should be the minimum required from any responsible language. And best then if you don't use the system package manager, do not mix up both to create a frankenstein language that multiplies complexity for everyone. The biggest thing is stop being faddish. Recognize upfront that there are always developers and groups jockeying for influence and creating dependencies on their apps and platforms. Discourage them from hoisting their self serving bad ideas on the ecosystem and multiplying complexity for everyone. Ultimately the language and ecosystem have to develop consensus on robust engineering practices or a wildland which will eventually lose users.
    • namelost 9y ago
      Who says? npm is proof that package managers need to do none of these things to be successful, and indeed the traditional design of a package manager as a constraint solver is self-defeating, because it puts a perverse pressure on libraries to have as few dependencies as possible and to avoid using the package manager to resolve their dependencies. The reason npm has teething problems is because it's one of the first package managers to actually reliably address the problem of libraries depending on other libraries. Almost incredibly, nobody bothered to solve this problem before npm. i.e. npm addresses the case of library X depending on Z v2.0 and library Y depending on Z v3.0 without falling flat on its face.
      • chii 9y ago
        How does npm solve the transitive dependencies problem?
        • Ajedi32 9y ago
          If by "transitive dependencies problem" you mean a case where package A depends on package B which depends on package C and you want to use A in your project, then it solves that by simply pulling in all three of those packages. Or did you mean the case where you want package D as well, and it depends on a conflicting version of package C? In that case, it solves the problem by pulling in both versions of package C and running both side by side.
          • chii 9y ago
            I do mean the latter - A and D both using different versions of C. Running both C side-by-side cannot work if C is incompatible (that is, even if both versions are api compatible, but each assumes it's the sole C being loaded - and therefore, do some static/singleton crap that might get clobbered when loaded again).
            • Ajedi32 9y ago
              > Running both C side-by-side cannot work if C is incompatible I've honestly never encountered an NPM package which couldn't be run side-by-side with another version of itself. This is due to the nature of how Node's module system (CommonJS) works; packages are isolated from each other and only share resources with each other via explicit exports and imports. I suppose a conflict might be possible if the package was using native extensions or connecting to some external service or something, but for the most part NPM's module system makes conflicts very unlikely.
  • grrowl 9y ago
    You specifically chose a large dependency with many sub-dependencies, so yes, that will happen. There's also the risk of installing an outdated, unmaintained dependency. Do a little research — check the package's npm page, assess whether it's too light or too heavy for your use-case. Check its github page to assess whether it's currently maintained (and how important that is for your use-case). If you're unsure, look at similar packages and/or peruse the source code. It only takes a few minutes and you'll have much greater confidence because you know you picked the correct multi-byte-string-length-calculating dependency for your use case, not the naive implementation which is 100x slower (for example).
    • omginternets 9y ago
      >You specifically chose a large dependency with many sub-dependencies, so yes, that will happen. You're too kind. JavaScript is outright unusable without pulling in hundreds of dependencies. NPM's ecosystem is 80% band-aids over terrible language design, which in turn leads to things like this: https://github.com/stevemao/left-pad/issues/4 https://github.com/stevemao/left-pad/issues/4 NPM is the symptom. JS is the problem.
    • deleted 9y ago
      [deleted]
    • reificator 9y ago
      > You specifically chose a large dependency with many sub-dependencies, so yes, that will happen. To clarify, I did NOT choose that package. Because it brought in 690 dependencies... Javascript didn't even have a pad left in the stdlib until the kik fiasco. Pulling in dependencies isn't really optional unless you want to start from first principles. Am I saying padding a string is difficult? No. But I am saying it's an incredibly common operation as evidenced by how much broke when it was pulled from NPM. As as this talk from 2016 shows, the versions that used to be available on NPM don't even pass a reasonable set of tests for a left pad: https://youtu.be/FyCYva9DhsI?t=605 https://youtu.be/FyCYva9DhsI?t=605 Not even being in the spotlight was enough to catch the bugs there, if you reimplement the world from scratch you're bound to make some errors yourself.
  • ceejayoz 9y ago
    My favorite NPM moment was the time I realized a project's dependencies included all three of https://www.npmjs.com/package/array-unique https://www.npmjs.com/package/array-unique, https://www.npmjs.com/package/array-uniq https://www.npmjs.com/package/array-uniq, and https://www.npmjs.com/package/uniq https://www.npmjs.com/package/uniq.
• flukus 9y ago
  On top of this, updating dependencies is becoming just as important as patching the OS itself, except without the culture of providing non-breaking updates. Often you're left with the choice of having to either upgrade to the latest version and deal with breaking changes or leave the insecure versions in place. This compounds the dislike companies already have for updating software, updating packages isn't something that's ever planned and budgeted for. A standard enterprise app will be using tonnes of outdated and potentially insecure packages, I've come across some that are a decade out of date with no pain-free update path. And now it seems that even "systems languages" are heading down this path.
• viraptor 9y ago
  You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? Packages taken over include misspelling of urllib. What exactly do you propose as an alternative here? I'm all for limiting bloat, but your rant here seems completely inappropriate for the issue bring described.
  • flukus 9y ago
    > You mean that every time you write a new project you write your own request parser, your own server, your own web framework, your own database access libraries, etc. from scratch? They made it pretty clear that they get them from a distro repository. Most are much better about managing breaking changes and offering years of security patches (at least to core packages). > Packages taken over include misspelling of urllib. What exactly do you propose as an alternative here? Stricter guidelines and auditing should have caught this.
  • zizek23 9y ago
    You are fighting a strawman. Either use the system package manager which has already thought about and solved these issues or if you must use a 'home brew' package manager, design a secure one. And if you can't do that limit the circus to developer machines.
    • viraptor 9y ago
      I've not seen a non-trivial project which doesn't require at least one package outside of the repo, or an updated version. And then you either have to run your own repo and packaging process. (Been there, done that, not fun) Or mix sources. (noooooooope) Or just switch to pypi completely. Even if you can work with that, you need to sort out differences between your destination system, test system, and all of your developers machines (who most likely run Mac, not your server Linux flavour). Sure, there's still docker, vagrant, etc. But this means slapping more and more layers just to make the same packages from the same source available. And unless you want a "works on my machine" environment, you can't just leave developers to do whatever and use a different system in production. A home brew, limited system will just result in shadow it, which will use pypi unless your solution is much better and easier to use (and allows fresh versions to be imported) I've been doing this stuff for years in big projects. It's not easy.
• yeukhon 9y ago
  Everything we do in life relies on a trust system more or less. Reproducible build only guarantees integrity, not the trustworthiness of the code. > There is no way for anyone to know what all this code is doing, there is little way to verify updates and its simply untenable. Because few people have time to read source code. Take Django as an example. Big community, lots of contributors. Can we say we should trust the code because the number of eyes? Probably, but not always. Why? We can overlook and pretend the code is legitmate. Take Linux kernel or some of the crypto projects out there. A lot of unreadable old tricks made backdoor really easy. [1] is interesting because someone apparently made an authorized changeset to BitKeeper. Are you going to read JRE code to make sure no backdoor? Nah. Is there a solution? Nope and will never have a solution. The best one can do is using a DVCS to prevent unathorized changeset (provided every contributor does a sanity check for pulling in changeset) and trust the community. No machine can distinugish a good code from a bad code. Humans can't even tell unless raising a red flag. Sometimes change of project ownership can also introduce some uncertainty, but that is a rare case for major projects. I am pretty confident there are code in the Linux kernel no one has ever touched or ever read for years. [1]: https://freedom-to-tinker.com/2013/10/09/the-linux-backdoor-attempt-of-2003/ https://freedom-to-tinker.com/2013/10/09/the-linux-backdoor-...
  • rlpb 9y ago
    Distributions - let's take Debian as a concrete example - provide an audit trail to individual identified developers as a mitigation for users relying on trust. Just because we must necessarily rely on some level of trust does not mean that we must blindly trust, which is what happens when anyone can upload to a repository such as PyPI.
    • deleted 9y ago
      [deleted]
    • yeukhon 9y ago
      As far as I know, once the project name is taken, that ownership belongs to the account first created this package in PyPI.
  • 9y ago
• sametmax 9y ago
  Sooo, you read the OpenSSL source code and avoided heart bleed ?
• jacalata 9y ago
  > Users should simply boycott such user hostile developers and languages I await with interest your newsletter to "the internet" on how to boycott JavaScript.
  • zizek23 9y ago
    There are a lot of constructive ways to begin to solve this but I guess you prefer the camel in sand approach. Every single Ruby post has commentators complaining about dependency hell and steering clear of Ruby apps. This was not the case even a couple of years ago. This is effectively a user boycott which Ruby may not deserve but has brought on itself by letting the 'break everything crowd' run amok. They have moved on to Node and will move again to the next big thing but it's Ruby left dealing with the fallout.
    • fish_fan 9y ago
      > Every single Ruby post has commentators complaining about dependency hell and steering clear of Ruby apps. This was not the case even a couple of years ago. Strangely the problems with rubygems and related package managers were apparent five years ago (at my first contact with earnest gem development). It seems the major change is major gems get abandoned—take for example bcrypt. I wonder if the rise in distrust and the rise of abandonded but critical gems are related.
    • seanp2k2 9y ago
      What has changed in the past few years in the Ruby world that has changed dep mgmt? Bundler has been the thing to use for quite a while now, and it used a version lock for all dependencies. It took Node a little while to figure that one out, and shrinkwrap seemingly led to yarn then back into NPM improvements. Ruby is comparatively not that bad, excepting libs with e.g. C dependencies which Node has as well.
    • jacalata 9y ago
      That's developers boycotting the language, not users boycotting developers of the language.
  • 9y ago
• ageofwant 9y ago
  Bulshit. The benefits of this "casual culture of pulling in hundred of dependencies" vastly outweigh the harm. I'll be assembling another $6000 job that will take me maybe 12 hours to complete while you write your compiler, from scratch, in your own assembler, made for your own cpu, that you youre gonna cook up from a bucket of sand you collected yourself, from a sandpit you trust.
  • tway6667 9y ago
    Damn straight. While you're assembling that $6k job, I'm selling your client's data from out from under them. Everyone gets paid.
  • flachsechs 9y ago
    i agree that the dependencies are probably unavoidable and probably a net good but... well... just don't forget the system and network security best practices while you quickly put together that project, especially if it's in a public cloud. btw you should be making $6000 + $1000 or so a month for at least a couple of years doing that, not $6000 a single time. the total lifetime value of setting up an application should be above $30k, hopefully way above, but at the entry level where you are that's a nice healthy number to shoot for. we bill our total customer base over $6000 a day for this kind of work and it all starts with 'assembling a job' but certainly doesn't end there, or ever (hopefully). good luck buddy!
  • Stranger43 9y ago
    the problem is that your $6000,- one off will cause your client to face either completely trivial to exploit production systems or exponentially growing operating costs, because lets face it you are going to provide zero hours of post deployment support for that task.
    • ageofwant 9y ago
      Of course, support is $300/h, what am I, a chump?
• gjtorikian 9y ago
  > Many have suffered and do not even bother with Ruby apps anymore. These sort of grandiose statements have of course existed on the Internet for decades, but with the rise of demagogues like Trump, we see that such statements can be readily believed en masse without a second thought. Could you provide data or numbers explaining that Ruby's package management system is a factor in new apps not being built in that language anymore? I don't mean to necessarily add politics in here, but hastily throwing out intimidating messages--"Ruby's package manager is broken and now no one is using it; Node and others will of course head down the same path"--helps no one. First, it harms the morale of new programmers learning these languages who may be lead to believe they are wasting their time. Second, it frankly is rude to the groups of individuals who do work on the package managers themselves; if you have a better idea, build or sponser one. The burden of proof is on you to provide these statistics, so until then, there's also the third result: it makes you look like a crank, not a professional who has considered the pros and cons of different package management systems.
  • jpttsn 9y ago
    Don't you think bringing Trump into this is an example of the same things you admonish GP for?
    • gjtorikian 9y ago
      How so? My point is that stating something as a fact without proof can have serious consequences.
  • thehardsphere 9y ago
    I wasn't aware that anybody on Hacker News had an implicit moral obligation to "not harm the morale of new programmers", especially in cases where it might be a open question if said programmers are "wasting their time." Also, there's a third choice if you have a better idea about how package managers could work: you can use an existing alternative that works better. If said existing alternative happens to be in some other language, and you don't have a strong reason to stay on your current language, then switching languages is perfectly sensible.
• 7jewndjwq 9y ago
  > Users should simply boycott such user hostile developers and languages that encourage this kind of insecurity. We are talking about something that started as a catalog: package index. You are an entitled loser who is too dumb or lazy to maintain a list of checksums. Please don't use Python.
• throwme211345 9y ago
  I agree. This type of casual 'language manager install X', or even worse; use this container or vagrant image as your base, is the root of all evil. It is the political core of the devops revolution which allows developers to pollute operations with shortcuts and balloon code to avoid the hard work of maintaining/documenting a system and performing due diligence.
• devdas 9y ago
  https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...