8 ms·

> A little copying is better than a little dependency. I actually disagree with this. Given that we're already working with projects that can have several hund

> A little copying is better than a little dependency. I actually disagree with this. Given that we're already working with projects that can have several hundred dependencies - I think the goal ought to be in removing the weight of including the dependency. sindresorhus says it better than me though https://github.com/sindresorhus/ama/issues/10#issuecomment-117766328 https://github.com/sindresorhus/ama/issues/10#issuecomment-1... I imagine it also depends on whether your place of employment is writing one giant single-repository codebase vs. several modular applications, as the latter clearly affords a package manager while the former eschews the complexity of composing systems.

thecopy 11y ago
But we have to draw the line somewhere, i.e. where do we break DRY? If the if-statement for negative-zero ought be in it's own package, how smaller granularity can we go before we should start copying the code? What about checking if two numbers are equal? To play on your link to github where he talks about is-negative-zero: For example. I have this module is-equal. Its job is to tell me if two numbers are equal. Normally you wouldn't have to care about this, but it could happen. How do you figure out if two numbers are equal? Well easy x == y. Or is it? Do you really want to have to know this might be wrong? I would rather require is-equal and be productive on other things.
- pluma 11y ago
  Did you read Sindre's post? You're still arguing about numbers of characters and lines of code. Determining whether a value is negative zero in JS is non-trivial and easy to get wrong. JS tries its best to shield you from the existence of negative zero but it still exists and if you do have to test for it, your intuitive solution (x === -0, String(x).charAt(0) === '-', Math.sign(x) === -1, etc) will likely be wrong. It's not equivalent to "comparing numbers for equality". It's a specific problem with a specific but obscure solution. If you had to work this solution out yourself (if, indeed, as you're more likely to just get it wrong or arrive at something more complex or less efficient) you would put it in a function and you'd have to write a couple of tests for that function to make sure it actually works and doesn't produce false positives. The same goes for figuring out the user's home directory: most Node developers use a flavour of *nix (usually OSX or Linux) so they may know `process.env.HOME` but what about Windows users? Most likely they will just forego Windows or assume it's also called `HOME` on Windows (because on many developer Windows machines it might be for compatibility). Instead, don't guess and just add a small module that does that one thing and can be extended should this change in the future (e.g. if OS11 comes along and starts calling it `APPLE_HOME` or if Win11 goes full POSIX and also calls it `HOME` but still passes the platform check for Windows). Your strawman of number equality OTOH is not about a weird edge case or a cross-platform concern but merely about knowing the language. Of course it'd be ridiculous to add a function to compare two numbers for equality in an if statement because the language already provides an operator for that. The only excuse I can see for that would be if you take functional programming to its logical conclusion and want every operator to be a function your religious views prohibit using modules that provide entire collections of these functions so you need to add every operator as its own dependency. Then again, I have published a module in the past that today can literally be replaced with four characters of code in ES2015 (plus whitespace) -- and the API documentation is even wrong on what it does (but at least it has 100% test coverage). Everything can be taken to its absurd extreme, what point you should bail at depends on the situation.
  - esailija 11y ago
    It is extremely trivial, just not intuitive. It's much harder to actually come up with a case where you know you even need it.
raverbashing 11y ago
I disagree "Copying" might just be: check your needed dependencies into your source code tree. Managing complexity is important, depending on several hundred things (especially if you don't have control over them) is not managing it. Depending on simple things like "left pad" is just shooting yourself in the foot in a future time, as this has shown In today's world it's easy to forget the million ways in which a castle of cards might fail.
- pluma 11y ago
  > Depending on simple things like "left pad" is just shooting yourself in the foot in a future time, as this has shown But it hasn't. Left-pad was republished by a third party shortly after it was unpublished by its original author. It was open source and so it was trivial to fork and republish. Personally my only direct dependency on azer's modules was a module for shuffling an array in place. A quick look in the registry provided me with a substitute that had the same API and used the same algorithm and had the same test coverage, by another reputable author. Now imagine instead of dozens of small tiny modules it had been jQuery, or React, or Babel. Or anything else that survives the "is it too large to copy-paste into your codebase" test the article puts forth. Good luck replacing that kind of dependency in less than an hour (if your argument is "well, I can just download a copy elsewhere and paste it: what if it was unpublished because of an undisclosed vulnerability with no patch available and you need to replace it). Facebook and others "copy" by checking in dependencies into version control. They will never be affected by a decision to unpublish something from the registry. But even so unpublishing is a red herring: it's a quirk of npm Inc's registry. The real takeaway from #unpublishgate is: don't depend on external sources for continuous deployment. If you need a registry, proxy it (hint: there are free alternatives to npm Inc's on premise offering). If your deploys only work when npm (or GitHub) is online, your deployment process is already broken.
  - danieldk 11y ago
    Facebook and others "copy" by checking in dependencies into version control. This is the standard method for 'copying' in Go as well. It's called 'vendoring' and is often applied to Go programs (as opposed to Go libraries). You basically put a checkout of the dependency in the vendor/ folder and the go tool will prefer it over fetching the dependency and/or grabbing it from GOPATH (since Go 1.6).
blub 11y ago
It's the third time I see that link already, he's wrong, so please stop posting it :) The way to reuse N lines of code is to refactor them in a function. The way to reuse functions is to pack them into libraries. This is something that probably everyone agrees with. Many have issues with the idea that one should create libraries containing one function only! You've touched yourself upon the why: having hundreds of dependencies adds both technical and cognitive overhead. It's very hard to design things to be extended, so in practice one ends up with a fragile module, not one that can be improved by improving a dependency (I wonder how a function that checks if a number is negative can be improved...). As far as I know there is no other toolchain on this planet where it's done this way, which together with the clarification above should be telling you and sindresorhus something!
- danieldk 11y ago
  Many have issues with the idea that one should create libraries containing one function only! You've touched yourself upon the why: having hundreds of dependencies adds both technical and cognitive overhead. Indeed. Moreover, it's a sign of a weak or incomplete standard library. Early Java had this problem as well. However, since distribution consisted of passing around JAR files, most utility code was bundled in larger libraries (e.g., see commons-.*).
- dozzie 11y ago
  > As far as I know there is no other toolchain on this planet where it's done this way [...] If by "this way" you mean microdependencies in JavaScript, you should also look at Ruby, which goes in a similar direction. And at Python, which apparently tries to follow the lead.
  - pjmlp 11y ago
    Somehow I see a common theme there...
    - workusername 11y ago
      Yes, let's dig into it.
dozzie 11y ago
> Given that we're already working with projects that can have several hundred dependencies - I think the goal ought to be in removing the weight of including the dependency. You can't remove the intrinsic weight of incorporating foreign code to your codebase, which includes giving control over bugs to a third party, need for keeping another thing up-to-date or else..., and risk of being dead in the water if the maintainer does anything weird to the library (like removing the library from repository, what happened to left-pad, but also suddenly changing interface and/or call semantics, bumping only patchlevel part of version number). And happy debugging when the library changes in a very subtle way, so your code doesn't break and blow up, instead silently starts dropping some requests. You increase risk of happening anything from the list above when you include a dependency, and this risk cannot be lessened by tooling (at least in most languages, which have type systems less powerful than Haskell). So when you have several hundred dependencies you don't need to remove weight of including a dependency, you need to actually make the dependency list shorter.
- spion 11y ago
  > like removing the library from repository, what happened to left-pad This was entirely npm's fault for allowing it. They recognized that and fixed it. Nothing to do with tiny modules. Moving on. > but also suddenly changing interface and/or call semantics, bumping only patchlevel part of version number This is actually less likely with tiny modules. If it were a large module like underscore, and its just changing this one tiny function a little bit, its okay if they bump the minor verision number, right? Well, it broke your program because out of those 100 functions you were using exactly that one.
  - dozzie 11y ago
    >> like removing the library from repository, what happened to left-pad > This was entirely npm's fault for allowing it. Not entirely. npm's fault is more along the lines of fsckup, and that happens from time to time. It's everybody else's fault for including such a trivial module to their code (I mean, those who included it directly), what leads to much bigger exposure to such things (because there's much more modules to depend on). >> but also suddenly changing interface and/or call semantics, bumping only patchlevel part of version number > This is actually less likely with tiny modules. Times the number of the modules. Or rather, (1.0 - likeness) ^ count(modules). You end up with much more fragile application. And even when we put that aside, you still have large surface for all those unlikely events when somebody makes a mistake, but it's not from the publishing and versioning ground. > If it were a large module like underscore, and its just changing this one tiny function a little bit, its okay if they bump the minor verision number, right? Well, it broke your program because out of those 100 functions you were using exactly that one. There are two issues here. First, the new release was probably totally unnecessary. Why not bundle several such changes together? And semantic versioning scheme calls for increase of major number whenever the change is backwards-incompatible. Too bad that the change was tiny. So no, it's not "okay if they bump the minor version number". Second, if you were only using one function for those 100 functions, you screwed up. You probably didn't needed that one tiny function that much, so you added an unnecessary dependency. It's the same mentality of micromodules that leads to a dependency fractal, which in turn leads to a fragile codebase. Adding a dependency has its cost, but it costs in the (somewhat) distant future. So you're taking some debt now, to repay later at much higher price, except probably you don't see the price at the decision moment and you don't realize it's an instalment when you need to repay part of it.
mofle 11y ago
I'm sindresorhus. My thoughts on the left-pad situation: https://github.com/sindresorhus/ama/issues/367 https://github.com/sindresorhus/ama/issues/367
zaphar 11y ago
So this is a risk assessment problem. If you simple command line app depends on more than a hundred different packages then you have to balance the risk of a larger more fragmented "attack" surface in your dependencies against the cost of maintaining more of your own code. No one is going to draw their line in the sand in the same place. It's affected by the languages culture, the programmers experiences, the business needs for speed of development. It will change over time as the developer has good or bad experiences and it will change as the business needs change. I don't disagree or agree with the statement because I don't live in an ideal world where I get infinite time for dev and qa.

thecopy 11y ago

But we have to draw the line somewhere, i.e. where do we break DRY? If the if-statement for negative-zero ought be in it's own package, how smaller granularity can we go before we should start copying the code? What about checking if two numbers are equal? To play on your link to github where he talks about is-negative-zero: For example. I have this module is-equal. Its job is to tell me if two numbers are equal. Normally you wouldn't have to care about this, but it could happen. How do you figure out if two numbers are equal? Well easy x == y. Or is it? Do you really want to have to know this might be wrong? I would rather require is-equal and be productive on other things.

pluma 11y ago
Did you read Sindre's post? You're still arguing about numbers of characters and lines of code. Determining whether a value is negative zero in JS is non-trivial and easy to get wrong. JS tries its best to shield you from the existence of negative zero but it still exists and if you do have to test for it, your intuitive solution (x === -0, String(x).charAt(0) === '-', Math.sign(x) === -1, etc) will likely be wrong. It's not equivalent to "comparing numbers for equality". It's a specific problem with a specific but obscure solution. If you had to work this solution out yourself (if, indeed, as you're more likely to just get it wrong or arrive at something more complex or less efficient) you would put it in a function and you'd have to write a couple of tests for that function to make sure it actually works and doesn't produce false positives. The same goes for figuring out the user's home directory: most Node developers use a flavour of *nix (usually OSX or Linux) so they may know `process.env.HOME` but what about Windows users? Most likely they will just forego Windows or assume it's also called `HOME` on Windows (because on many developer Windows machines it might be for compatibility). Instead, don't guess and just add a small module that does that one thing and can be extended should this change in the future (e.g. if OS11 comes along and starts calling it `APPLE_HOME` or if Win11 goes full POSIX and also calls it `HOME` but still passes the platform check for Windows). Your strawman of number equality OTOH is not about a weird edge case or a cross-platform concern but merely about knowing the language. Of course it'd be ridiculous to add a function to compare two numbers for equality in an if statement because the language already provides an operator for that. The only excuse I can see for that would be if you take functional programming to its logical conclusion and want every operator to be a function your religious views prohibit using modules that provide entire collections of these functions so you need to add every operator as its own dependency. Then again, I have published a module in the past that today can literally be replaced with four characters of code in ES2015 (plus whitespace) -- and the API documentation is even wrong on what it does (but at least it has 100% test coverage). Everything can be taken to its absurd extreme, what point you should bail at depends on the situation.
- esailija 11y ago
  It is extremely trivial, just not intuitive. It's much harder to actually come up with a case where you know you even need it.

raverbashing 11y ago

I disagree "Copying" might just be: check your needed dependencies into your source code tree. Managing complexity is important, depending on several hundred things (especially if you don't have control over them) is not managing it. Depending on simple things like "left pad" is just shooting yourself in the foot in a future time, as this has shown In today's world it's easy to forget the million ways in which a castle of cards might fail.

pluma 11y ago
> Depending on simple things like "left pad" is just shooting yourself in the foot in a future time, as this has shown But it hasn't. Left-pad was republished by a third party shortly after it was unpublished by its original author. It was open source and so it was trivial to fork and republish. Personally my only direct dependency on azer's modules was a module for shuffling an array in place. A quick look in the registry provided me with a substitute that had the same API and used the same algorithm and had the same test coverage, by another reputable author. Now imagine instead of dozens of small tiny modules it had been jQuery, or React, or Babel. Or anything else that survives the "is it too large to copy-paste into your codebase" test the article puts forth. Good luck replacing that kind of dependency in less than an hour (if your argument is "well, I can just download a copy elsewhere and paste it: what if it was unpublished because of an undisclosed vulnerability with no patch available and you need to replace it). Facebook and others "copy" by checking in dependencies into version control. They will never be affected by a decision to unpublish something from the registry. But even so unpublishing is a red herring: it's a quirk of npm Inc's registry. The real takeaway from #unpublishgate is: don't depend on external sources for continuous deployment. If you need a registry, proxy it (hint: there are free alternatives to npm Inc's on premise offering). If your deploys only work when npm (or GitHub) is online, your deployment process is already broken.
- danieldk 11y ago
  Facebook and others "copy" by checking in dependencies into version control. This is the standard method for 'copying' in Go as well. It's called 'vendoring' and is often applied to Go programs (as opposed to Go libraries). You basically put a checkout of the dependency in the vendor/ folder and the go tool will prefer it over fetching the dependency and/or grabbing it from GOPATH (since Go 1.6).

blub 11y ago

It's the third time I see that link already, he's wrong, so please stop posting it :) The way to reuse N lines of code is to refactor them in a function. The way to reuse functions is to pack them into libraries. This is something that probably everyone agrees with. Many have issues with the idea that one should create libraries containing one function only! You've touched yourself upon the why: having hundreds of dependencies adds both technical and cognitive overhead. It's very hard to design things to be extended, so in practice one ends up with a fragile module, not one that can be improved by improving a dependency (I wonder how a function that checks if a number is negative can be improved...). As far as I know there is no other toolchain on this planet where it's done this way, which together with the clarification above should be telling you and sindresorhus something!

danieldk 11y ago
Many have issues with the idea that one should create libraries containing one function only! You've touched yourself upon the why: having hundreds of dependencies adds both technical and cognitive overhead. Indeed. Moreover, it's a sign of a weak or incomplete standard library. Early Java had this problem as well. However, since distribution consisted of passing around JAR files, most utility code was bundled in larger libraries (e.g., see commons-.*).
dozzie 11y ago
> As far as I know there is no other toolchain on this planet where it's done this way [...] If by "this way" you mean microdependencies in JavaScript, you should also look at Ruby, which goes in a similar direction. And at Python, which apparently tries to follow the lead.
- pjmlp 11y ago
  Somehow I see a common theme there...
  - workusername 11y ago
    Yes, let's dig into it.

dozzie 11y ago

> Given that we're already working with projects that can have several hundred dependencies - I think the goal ought to be in removing the weight of including the dependency. You can't remove the intrinsic weight of incorporating foreign code to your codebase, which includes giving control over bugs to a third party, need for keeping another thing up-to-date or else..., and risk of being dead in the water if the maintainer does anything weird to the library (like removing the library from repository, what happened to left-pad, but also suddenly changing interface and/or call semantics, bumping only patchlevel part of version number). And happy debugging when the library changes in a very subtle way, so your code doesn't break and blow up, instead silently starts dropping some requests. You increase risk of happening anything from the list above when you include a dependency, and this risk cannot be lessened by tooling (at least in most languages, which have type systems less powerful than Haskell). So when you have several hundred dependencies you don't need to remove weight of including a dependency, you need to actually make the dependency list shorter.

spion 11y ago
> like removing the library from repository, what happened to left-pad This was entirely npm's fault for allowing it. They recognized that and fixed it. Nothing to do with tiny modules. Moving on. > but also suddenly changing interface and/or call semantics, bumping only patchlevel part of version number This is actually less likely with tiny modules. If it were a large module like underscore, and its just changing this one tiny function a little bit, its okay if they bump the minor verision number, right? Well, it broke your program because out of those 100 functions you were using exactly that one.
- dozzie 11y ago
  >> like removing the library from repository, what happened to left-pad > This was entirely npm's fault for allowing it. Not entirely. npm's fault is more along the lines of fsckup, and that happens from time to time. It's everybody else's fault for including such a trivial module to their code (I mean, those who included it directly), what leads to much bigger exposure to such things (because there's much more modules to depend on). >> but also suddenly changing interface and/or call semantics, bumping only patchlevel part of version number > This is actually less likely with tiny modules. Times the number of the modules. Or rather, (1.0 - likeness) ^ count(modules). You end up with much more fragile application. And even when we put that aside, you still have large surface for all those unlikely events when somebody makes a mistake, but it's not from the publishing and versioning ground. > If it were a large module like underscore, and its just changing this one tiny function a little bit, its okay if they bump the minor verision number, right? Well, it broke your program because out of those 100 functions you were using exactly that one. There are two issues here. First, the new release was probably totally unnecessary. Why not bundle several such changes together? And semantic versioning scheme calls for increase of major number whenever the change is backwards-incompatible. Too bad that the change was tiny. So no, it's not "okay if they bump the minor version number". Second, if you were only using one function for those 100 functions, you screwed up. You probably didn't needed that one tiny function that much, so you added an unnecessary dependency. It's the same mentality of micromodules that leads to a dependency fractal, which in turn leads to a fragile codebase. Adding a dependency has its cost, but it costs in the (somewhat) distant future. So you're taking some debt now, to repay later at much higher price, except probably you don't see the price at the decision moment and you don't realize it's an instalment when you need to repay part of it.

mofle 11y ago

I'm sindresorhus. My thoughts on the left-pad situation: https://github.com/sindresorhus/ama/issues/367 https://github.com/sindresorhus/ama/issues/367

zaphar 11y ago

So this is a risk assessment problem. If you simple command line app depends on more than a hundred different packages then you have to balance the risk of a larger more fragmented "attack" surface in your dependencies against the cost of maintaining more of your own code. No one is going to draw their line in the sand in the same place. It's affected by the languages culture, the programmers experiences, the business needs for speed of development. It will change over time as the developer has good or bad experiences and it will change as the business needs change. I don't disagree or agree with the statement because I don't live in an ideal world where I get infinite time for dev and qa.