3 ms·

How is it not an intrinsic trait? Is "bloated", "huge", "slow" an intrinsic trait of a software? If so then the opposite should be true as well. The claims are

How is it not an intrinsic trait? Is "bloated", "huge", "slow" an intrinsic trait of a software? If so then the opposite should be true as well. The claims are true intrinsic or not and they are also claims other software developers classify different libraries with.

29athrowaway 7y ago
You define it/describe it as being "the fastest". This claim translates into: for the universe of libraries that do this same thing, this has the best performance for each one of the features this library provides. That is claim that is not very hard to disprove. You only need 1 library that does better in one specific way and the claim is invalidated. It is a very broad claim. Then, performance in node is very subjective. The same code can perform better or worse depending on node/v8 versions. Can perform better or worse depending on which parameters you used, or how many times a function was invoked. You say "1m files in < 1s" but your benchmark has N=7386. Then: what filesystem was used? was it encrypted? was it in a RAID array? There is some missing context here as well.
- thecodrr 7y ago
  You seem to have read that statement wrong. I never said the fastest directory crawler on Planet earth. I said "the fastest directory crawler for NodeJS" restricting that claim because I am aware of the implications and how far-fetched it would sound. As for the "1m files in < 1s", yes I benchmarked that privately. That is not a false statement and to back that up, I will be uploading a gif showing the time (once the cli for fdir is done). > The same code can perform better or worse depending on node/v8 versions. I provided benchmarks for 2 node versions. The benchmark was fair for all libraries. It was run in the same way on the same machine for the same number of times on the same directory. So relatively the results should be fair. They were all also used barebones without parameters. Of course each library has its own approach. I cannot be held accountable for that. > what filesystem was used? was it encrypted? was it in a RAID array? There is some missing context here as well. You seem to have missed the whole point. You seem knowledgeable enough so let me say it in simple words. All node libraries that crawl directories for files use the `fs` module internally. Now, fs uses LibUV underneath. That is true for all libraries. There's no exception (to my knowledge). Now since the underlying i/o and the v8 engine are exactly the same, the only thing that makes the difference (and very slight at that) is the logic. Fdir wins only against other node libraries through that logic not because of some new i/o approach. The figure 1m files in < 1s is just to give an idea of speed not to show that you'll be getting the same exact number. I am aware of how fragile this claim is but until there comes a faster libraries I have the right to hold it. :)
  - 29athrowaway 7y ago
    - There are millions of modules on npm and you are claiming this is the fastest of them all? I doubt it, and not only I doubt it: I find it misleading since it's hard to verify and reflects a lack of humility. To say "Our goal is to have the fastest node module" or "A module with performance in mind" is VERY different to "The fastest module". - If I decide to use your code for something serious I cannot take your numbers for granted. The benchmark methodology does matter. You need show some signs of rigor so other people can assess how credible your claims are. - Did you run this benchmark in a controlled environment without other processes running? Did you disable frequency scaling before running your benchmark? If you ran it in a desktop computer with a bunch of other stuff running and frequency scaling enabled then your processor speed was constantly oscillating while running your benchmark and the numbers are completely random. - Did you know that v8 is full of optimizations that are conditional to how your code is written? Did you know that even a comment can dramatically affect how your code is treated by v8? In some cases V8 just gives up trying to optimize your code. Did you profile your code? did you trace v8 deoptimizations? Even if you did, you should not say this: Why are all the other libraries so slow? Because they did not spend enough time optimizing it. Most developers give readability and cool code more importance than actual performance and usability. It's really the self-promoting tone and unbelievable claims that make the README file so irritating.

29athrowaway 7y ago

You define it/describe it as being "the fastest". This claim translates into: for the universe of libraries that do this same thing, this has the best performance for each one of the features this library provides. That is claim that is not very hard to disprove. You only need 1 library that does better in one specific way and the claim is invalidated. It is a very broad claim. Then, performance in node is very subjective. The same code can perform better or worse depending on node/v8 versions. Can perform better or worse depending on which parameters you used, or how many times a function was invoked. You say "1m files in < 1s" but your benchmark has N=7386. Then: what filesystem was used? was it encrypted? was it in a RAID array? There is some missing context here as well.

thecodrr 7y ago
You seem to have read that statement wrong. I never said the fastest directory crawler on Planet earth. I said "the fastest directory crawler for NodeJS" restricting that claim because I am aware of the implications and how far-fetched it would sound. As for the "1m files in < 1s", yes I benchmarked that privately. That is not a false statement and to back that up, I will be uploading a gif showing the time (once the cli for fdir is done). > The same code can perform better or worse depending on node/v8 versions. I provided benchmarks for 2 node versions. The benchmark was fair for all libraries. It was run in the same way on the same machine for the same number of times on the same directory. So relatively the results should be fair. They were all also used barebones without parameters. Of course each library has its own approach. I cannot be held accountable for that. > what filesystem was used? was it encrypted? was it in a RAID array? There is some missing context here as well. You seem to have missed the whole point. You seem knowledgeable enough so let me say it in simple words. All node libraries that crawl directories for files use the `fs` module internally. Now, fs uses LibUV underneath. That is true for all libraries. There's no exception (to my knowledge). Now since the underlying i/o and the v8 engine are exactly the same, the only thing that makes the difference (and very slight at that) is the logic. Fdir wins only against other node libraries through that logic not because of some new i/o approach. The figure 1m files in < 1s is just to give an idea of speed not to show that you'll be getting the same exact number. I am aware of how fragile this claim is but until there comes a faster libraries I have the right to hold it. :)
- 29athrowaway 7y ago
  - There are millions of modules on npm and you are claiming this is the fastest of them all? I doubt it, and not only I doubt it: I find it misleading since it's hard to verify and reflects a lack of humility. To say "Our goal is to have the fastest node module" or "A module with performance in mind" is VERY different to "The fastest module". - If I decide to use your code for something serious I cannot take your numbers for granted. The benchmark methodology does matter. You need show some signs of rigor so other people can assess how credible your claims are. - Did you run this benchmark in a controlled environment without other processes running? Did you disable frequency scaling before running your benchmark? If you ran it in a desktop computer with a bunch of other stuff running and frequency scaling enabled then your processor speed was constantly oscillating while running your benchmark and the numbers are completely random. - Did you know that v8 is full of optimizations that are conditional to how your code is written? Did you know that even a comment can dramatically affect how your code is treated by v8? In some cases V8 just gives up trying to optimize your code. Did you profile your code? did you trace v8 deoptimizations? Even if you did, you should not say this: Why are all the other libraries so slow? Because they did not spend enough time optimizing it. Most developers give readability and cool code more importance than actual performance and usability. It's really the self-promoting tone and unbelievable claims that make the README file so irritating.