9 ms·

LeftPad and Go: can tooling help?

119 points by divan 11y ago

thegeekpirate 11y ago
@divan, any reason why you haven't submitted this for inclusion into gometalinter yet? I'm sure Alec would be more than happy to include it =)
- divan 11y ago
  After some polishing and testing on large projects, it will probably will be submitted.
iofj 11y ago
In case anyone wonders : yes. Left-pad left-pads a string. That's all it does. Not to worry though : it's now going to be implemented as a linux kernel syscall: https://lkml.org/lkml/2016/3/31/1108 https://lkml.org/lkml/2016/3/31/1108
catnaroek 11y ago
The author seems to make the assumption that the problem with left-pad is how small the library is. But that's just the symptom. The real problem with left-pad is how little functionality is actually provides. A 200-LOC C library that only left-pads strings should be subject to just as much scorn as the original 11-LOC JavaScript one. Unfortunately, it's difficult to assess how much functionality a library provides just from its LOC count.
- TickleSteve 11y ago
  Maybe a complexity metric would be a better measure?
  - catnaroek 11y ago
    I don't think any algorithmic analysis of the code can determine how much value it provides to the user. Some things simply require human judgment.
    - TickleSteve 11y ago
      but we're trying to infer that things dont have value to the user based on complexity.... the lower threshold is an easier target.
- mhd 11y ago
  API surface is often misleading, too. A JSON library that basically just provides a "json_encode" and a "json_decode" could be perfectly cromulent, whereas "trimlib", providing "right_trim" and "left_trim"...
gravity13 11y ago
> A little copying is better than a little dependency. I actually disagree with this. Given that we're already working with projects that can have several hundred dependencies - I think the goal ought to be in removing the weight of including the dependency. sindresorhus says it better than me though https://github.com/sindresorhus/ama/issues/10#issuecomment-117766328 https://github.com/sindresorhus/ama/issues/10#issuecomment-1... I imagine it also depends on whether your place of employment is writing one giant single-repository codebase vs. several modular applications, as the latter clearly affords a package manager while the former eschews the complexity of composing systems.
- thecopy 11y ago
  But we have to draw the line somewhere, i.e. where do we break DRY? If the if-statement for negative-zero ought be in it's own package, how smaller granularity can we go before we should start copying the code? What about checking if two numbers are equal? To play on your link to github where he talks about is-negative-zero: For example. I have this module is-equal. Its job is to tell me if two numbers are equal. Normally you wouldn't have to care about this, but it could happen. How do you figure out if two numbers are equal? Well easy x == y. Or is it? Do you really want to have to know this might be wrong? I would rather require is-equal and be productive on other things.
  - pluma 11y ago
    Did you read Sindre's post? You're still arguing about numbers of characters and lines of code. Determining whether a value is negative zero in JS is non-trivial and easy to get wrong. JS tries its best to shield you from the existence of negative zero but it still exists and if you do have to test for it, your intuitive solution (x === -0, String(x).charAt(0) === '-', Math.sign(x) === -1, etc) will likely be wrong. It's not equivalent to "comparing numbers for equality". It's a specific problem with a specific but obscure solution. If you had to work this solution out yourself (if, indeed, as you're more likely to just get it wrong or arrive at something more complex or less efficient) you would put it in a function and you'd have to write a couple of tests for that function to make sure it actually works and doesn't produce false positives. The same goes for figuring out the user's home directory: most Node developers use a flavour of *nix (usually OSX or Linux) so they may know `process.env.HOME` but what about Windows users? Most likely they will just forego Windows or assume it's also called `HOME` on Windows (because on many developer Windows machines it might be for compatibility). Instead, don't guess and just add a small module that does that one thing and can be extended should this change in the future (e.g. if OS11 comes along and starts calling it `APPLE_HOME` or if Win11 goes full POSIX and also calls it `HOME` but still passes the platform check for Windows). Your strawman of number equality OTOH is not about a weird edge case or a cross-platform concern but merely about knowing the language. Of course it'd be ridiculous to add a function to compare two numbers for equality in an if statement because the language already provides an operator for that. The only excuse I can see for that would be if you take functional programming to its logical conclusion and want every operator to be a function your religious views prohibit using modules that provide entire collections of these functions so you need to add every operator as its own dependency. Then again, I have published a module in the past that today can literally be replaced with four characters of code in ES2015 (plus whitespace) -- and the API documentation is even wrong on what it does (but at least it has 100% test coverage). Everything can be taken to its absurd extreme, what point you should bail at depends on the situation.
brad0 11y ago
IMO the best way to move to the next generation of dependencies is through tooling. Instead of ignoring node_modules in source control, why don't we commit it and allow the tooling to handle upgrades? Disclaimer: I haven't thought about the implications of committing external packages to your own source control at all. I'm guessing I'm missing something.
- petetnt 11y ago
  > I haven't thought about the implications of committing external packages to your own source control at all. I'm guessing I'm missing something. Committing external dependencies to source control means tons of bloat to the repos themselves. Want to clone huge_framework just to change a tiny thing? Well too bad, you'll need to clone the hundred megabytes of external dependencies too. Multiple libraries/frameworks/modules containing same versions of same modules? Too bad, everyone needs to commit their dependencies. Frameworks shouldn't commit their dependencies then? Or just libraries? Or just modules? Where is the line and what are the rules? Why not just handle dependencies manually? Because that's a pain in the ass and exact reason why package managers exist in first place.
  - scrame 11y ago
    >Committing external dependencies to source control means tons of bloat to the repos themselves. Want to clone huge_framework just to change a tiny thing? Oh, geez, the first thing that makes code _legacy_ code is people checking in "slightly modified" library/framework code. Not to mention different lib versions. Things like java's Spring libs where it seems like the jar layout, names and hosting changes every couple minor versions. Different dependencies that need different versions of things. Different apps in the same ecosystem that rely on newer versions than the legacy in-house libs they are compatible with. I've put up with a _lot_ of dependency nonsense on many different stacks over the years, and am trying to adapt to devs bringing in NPM modules for dev use, but for sure I've learned that letting devs check in random libs into source control just because its convenient ends up with a lot of problems. That includes java, php, ruby, perl, and python. That its yet another generation of landgrabbing devs re-inventing a broken wheel, or people just permanently checking in libs for convenience seems like a devils choice. ...until you see the dev that tries to write it all from scratch themselves.
XorNot 11y ago
In Go tooling does help. Vendor everything, commit it to source control, filter vendor/ when looking at diffs.
gtirloni 11y ago
I don't know if the Node.js project has the resources for this, but if they did, it would be much better if all these 10-line dependencies everybody loves to use could be included in some sort of standard library. That way they could all benefit from centralized services (e.g. security vulnerability handling process) and offer a smoother developer experience.
pfooti 11y ago
One of the weird problems here is performance, though. A naive approach to left-pad, might involve (new Array(n+1)).join(padChar), for example, since javascript doesn't actually have a consistent language-native way to create an arbitrary-length string. That technique ends up, in some javascript runtimes, taking orders of magnitude longer to run compared to a for loop with string concatenation. The array.join method is one line though. So, imagine the scenario where you've got a commonly-used function that gets vendored and from there copypasta'd into a lib/common or some other location in your project. You've basically just cut yourself off from any updates that the module author might make when new javascript optimization hacks come around (or obscure corner-case bugs need protecting against). It's a small problem, but it is at least worth considering. There's also this whole thing with licenses - there's a fundamental difference between using a library and copying its source into your build tree. The MIT license doesn't care, but the LGPL certainly does. I do like the idea in general. You should extend it by doing tree-shaking or dead-code analysis. "This library is big, but you're just using one function call, let's just copypasta that one in here".
- esailija 11y ago
  The code needs to pass threshold of originality to be copyrightable, otherwise you don't need to care about the license. For example `(new Array(n+1)).join(padChar)` is definitely not copyrightable.

119 points by divan 11y ago

@divan, any reason why you haven't submitted this for inclusion into gometalinter yet? I'm sure Alec would be more than happy to include it =)

divan 11y ago
After some polishing and testing on large projects, it will probably will be submitted.

iofj 11y ago

In case anyone wonders : yes. Left-pad left-pads a string. That's all it does. Not to worry though : it's now going to be implemented as a linux kernel syscall: https://lkml.org/lkml/2016/3/31/1108 https://lkml.org/lkml/2016/3/31/1108

catnaroek 11y ago

The author seems to make the assumption that the problem with left-pad is how small the library is. But that's just the symptom. The real problem with left-pad is how little functionality is actually provides. A 200-LOC C library that only left-pads strings should be subject to just as much scorn as the original 11-LOC JavaScript one. Unfortunately, it's difficult to assess how much functionality a library provides just from its LOC count.

TickleSteve 11y ago
Maybe a complexity metric would be a better measure?
- catnaroek 11y ago
  I don't think any algorithmic analysis of the code can determine how much value it provides to the user. Some things simply require human judgment.
  - TickleSteve 11y ago
    but we're trying to infer that things dont have value to the user based on complexity.... the lower threshold is an easier target.
mhd 11y ago
API surface is often misleading, too. A JSON library that basically just provides a "json_encode" and a "json_decode" could be perfectly cromulent, whereas "trimlib", providing "right_trim" and "left_trim"...

gravity13 11y ago

> A little copying is better than a little dependency. I actually disagree with this. Given that we're already working with projects that can have several hundred dependencies - I think the goal ought to be in removing the weight of including the dependency. sindresorhus says it better than me though https://github.com/sindresorhus/ama/issues/10#issuecomment-117766328 https://github.com/sindresorhus/ama/issues/10#issuecomment-1... I imagine it also depends on whether your place of employment is writing one giant single-repository codebase vs. several modular applications, as the latter clearly affords a package manager while the former eschews the complexity of composing systems.

thecopy 11y ago
But we have to draw the line somewhere, i.e. where do we break DRY? If the if-statement for negative-zero ought be in it's own package, how smaller granularity can we go before we should start copying the code? What about checking if two numbers are equal? To play on your link to github where he talks about is-negative-zero: For example. I have this module is-equal. Its job is to tell me if two numbers are equal. Normally you wouldn't have to care about this, but it could happen. How do you figure out if two numbers are equal? Well easy x == y. Or is it? Do you really want to have to know this might be wrong? I would rather require is-equal and be productive on other things.
- pluma 11y ago
  Did you read Sindre's post? You're still arguing about numbers of characters and lines of code. Determining whether a value is negative zero in JS is non-trivial and easy to get wrong. JS tries its best to shield you from the existence of negative zero but it still exists and if you do have to test for it, your intuitive solution (x === -0, String(x).charAt(0) === '-', Math.sign(x) === -1, etc) will likely be wrong. It's not equivalent to "comparing numbers for equality". It's a specific problem with a specific but obscure solution. If you had to work this solution out yourself (if, indeed, as you're more likely to just get it wrong or arrive at something more complex or less efficient) you would put it in a function and you'd have to write a couple of tests for that function to make sure it actually works and doesn't produce false positives. The same goes for figuring out the user's home directory: most Node developers use a flavour of *nix (usually OSX or Linux) so they may know `process.env.HOME` but what about Windows users? Most likely they will just forego Windows or assume it's also called `HOME` on Windows (because on many developer Windows machines it might be for compatibility). Instead, don't guess and just add a small module that does that one thing and can be extended should this change in the future (e.g. if OS11 comes along and starts calling it `APPLE_HOME` or if Win11 goes full POSIX and also calls it `HOME` but still passes the platform check for Windows). Your strawman of number equality OTOH is not about a weird edge case or a cross-platform concern but merely about knowing the language. Of course it'd be ridiculous to add a function to compare two numbers for equality in an if statement because the language already provides an operator for that. The only excuse I can see for that would be if you take functional programming to its logical conclusion and want every operator to be a function your religious views prohibit using modules that provide entire collections of these functions so you need to add every operator as its own dependency. Then again, I have published a module in the past that today can literally be replaced with four characters of code in ES2015 (plus whitespace) -- and the API documentation is even wrong on what it does (but at least it has 100% test coverage). Everything can be taken to its absurd extreme, what point you should bail at depends on the situation.

brad0 11y ago

IMO the best way to move to the next generation of dependencies is through tooling. Instead of ignoring node_modules in source control, why don't we commit it and allow the tooling to handle upgrades? Disclaimer: I haven't thought about the implications of committing external packages to your own source control at all. I'm guessing I'm missing something.

petetnt 11y ago
> I haven't thought about the implications of committing external packages to your own source control at all. I'm guessing I'm missing something. Committing external dependencies to source control means tons of bloat to the repos themselves. Want to clone huge_framework just to change a tiny thing? Well too bad, you'll need to clone the hundred megabytes of external dependencies too. Multiple libraries/frameworks/modules containing same versions of same modules? Too bad, everyone needs to commit their dependencies. Frameworks shouldn't commit their dependencies then? Or just libraries? Or just modules? Where is the line and what are the rules? Why not just handle dependencies manually? Because that's a pain in the ass and exact reason why package managers exist in first place.
- scrame 11y ago
  >Committing external dependencies to source control means tons of bloat to the repos themselves. Want to clone huge_framework just to change a tiny thing? Oh, geez, the first thing that makes code _legacy_ code is people checking in "slightly modified" library/framework code. Not to mention different lib versions. Things like java's Spring libs where it seems like the jar layout, names and hosting changes every couple minor versions. Different dependencies that need different versions of things. Different apps in the same ecosystem that rely on newer versions than the legacy in-house libs they are compatible with. I've put up with a _lot_ of dependency nonsense on many different stacks over the years, and am trying to adapt to devs bringing in NPM modules for dev use, but for sure I've learned that letting devs check in random libs into source control just because its convenient ends up with a lot of problems. That includes java, php, ruby, perl, and python. That its yet another generation of landgrabbing devs re-inventing a broken wheel, or people just permanently checking in libs for convenience seems like a devils choice. ...until you see the dev that tries to write it all from scratch themselves.

XorNot 11y ago

In Go tooling does help. Vendor everything, commit it to source control, filter vendor/ when looking at diffs.

gtirloni 11y ago

I don't know if the Node.js project has the resources for this, but if they did, it would be much better if all these 10-line dependencies everybody loves to use could be included in some sort of standard library. That way they could all benefit from centralized services (e.g. security vulnerability handling process) and offer a smoother developer experience.

pfooti 11y ago

One of the weird problems here is performance, though. A naive approach to left-pad, might involve (new Array(n+1)).join(padChar), for example, since javascript doesn't actually have a consistent language-native way to create an arbitrary-length string. That technique ends up, in some javascript runtimes, taking orders of magnitude longer to run compared to a for loop with string concatenation. The array.join method is one line though. So, imagine the scenario where you've got a commonly-used function that gets vendored and from there copypasta'd into a lib/common or some other location in your project. You've basically just cut yourself off from any updates that the module author might make when new javascript optimization hacks come around (or obscure corner-case bugs need protecting against). It's a small problem, but it is at least worth considering. There's also this whole thing with licenses - there's a fundamental difference between using a library and copying its source into your build tree. The MIT license doesn't care, but the LGPL certainly does. I do like the idea in general. You should extend it by doing tree-shaking or dead-code analysis. "This library is big, but you're just using one function call, let's just copypasta that one in here".

esailija 11y ago
The code needs to pass threshold of originality to be copyrightable, otherwise you don't need to care about the license. For example `(new Array(n+1)).join(padChar)` is definitely not copyrightable.