3 ms·

You don’t see the irony in that statement? You’re one person saying simdjson is faster, and you are also lacking a reproducible test. That doesn’t make your sid

You don’t see the irony in that statement? You’re one person saying simdjson is faster, and you are also lacking a reproducible test. That doesn’t make your side of the argument particularly compelling either. I’ve never used either library, so I don’t care which one is faster, but I would generally trust the author of the library to have tested the performance at some point more than I would trust someone who isn’t the author to comment on the performance of code they’ve never tested. The author of that library isn’t present to make their case, so you’re the only one of the two who could provide evidence right now. EDIT: the author is present now: https://news.ycombinator.com/item?id=23738512 https://news.ycombinator.com/item?id=23738512

the-dude 6y ago
Your comment would improve without the last sentence.
- coder543 6y ago
  I was iterating on the comment rapidly after posting to try to make my point in the most neutral way possible, so I’m not sure which “last sentence” you saw, but hopefully it is better now.
  - the-dude 6y ago
    This is much better, no need for ad hominems. But please keep the editorializing in check too, it is etiquette to mention edits, and continually editting breaks continuity in the discussion. As you pointed out yourself.
nemothekid 6y ago
>You don’t see the irony in that statement? You’re one person saying simdjson is faster, and you are also lacking a reproducible test. That doesn’t make your side of the argument particularly compelling either. simdjson is a massively popular library (~9.9k stars, the third most popular json library on GitHub), that is well known to beat the pants off of every other json library. Likewise the simdjson library has reproducible benchmarks in its own repo with charts and the like. So yes, I would consider that statement pretty much "worthless" given the popularity of simdjson.
glangdale 6y ago
Given how things turned out, this might be a good lesson as to whether "I measured my own library informally on some benchmarks that I couldn't be bothered to describe to you and it WINS!" is trustworthy in future. When we introduced simdjson, we went out of our way to explain (a) why we thought it might be faster than other libraries and (b) how people could replicate our measurements or find fault in our methodology. There were zero performance claims, public or private about the performance of simdjson before this material was available. In my long experience, undocumented and non-reproducible performance claims suffer from the fact that people generally stop benchmarking when they get the result that they wanted to hear, even if the absolute numbers involved are unreasonable. It's very easy to make mistakes in performance measurement (compiling simdjson with anything short of -O2 or -O3, doing repeated runs of a tiny input and thus getting perfect branch prediction behavior), but people generally fix these mistakes only on the side that makes their stuff look good. Back when I worked on the Hyperscan project ( https://github.com/intel/hyperscan https://github.com/intel/hyperscan ), we had an API call that allocated scratch memory (necessary due to Hyperscan's rather constrained approach to memory usage). I lost count of the number of times we had users run this allocation (which only ever needed to be done once per thread, effectively) inside their measurement loop. And whenever we were competing against an incumbent system and the people doing the measurement were the authors of that incumbent system, it was amazing how quickly they'd accept the "truth" of their benchmark run.
- coder543 6y ago
  I think it’s worth clarifying that I said I would “generally trust {author} more than {random internet person who hasn’t even tested author’s library}”. Are you saying that you trust random internet person more by default? Trusting one more than the other is not the same as blindly trusting one and never running my own tests. The author who put in the effort to develop and benchmark their library deserves more benefit of the doubt than some drive-by commenter, in my opinion. If I needed a high performance JSON parsing library, I would definitely set up a benchmark representative of my use case, and test a few different libraries. JSON libraries are usually pretty ergonomic and easy to swap between, so it shouldn’t be an arduous task. Your very first two comments in this thread were far more insightful than {random internet person} just saying over and over that simdjson can’t possibly be slower, and those comments of yours on their own merit would have removed the need for me to leave any comments whatsoever to defend the original HN post, even without needing to observe that you’re an expert in this field, which just lends further credibility to your comments. I think the rest of the discussion that occurred was really interesting to watch!

the-dude 6y ago

Your comment would improve without the last sentence.

coder543 6y ago
I was iterating on the comment rapidly after posting to try to make my point in the most neutral way possible, so I’m not sure which “last sentence” you saw, but hopefully it is better now.
- the-dude 6y ago
  This is much better, no need for ad hominems. But please keep the editorializing in check too, it is etiquette to mention edits, and continually editting breaks continuity in the discussion. As you pointed out yourself.

nemothekid 6y ago

>You don’t see the irony in that statement? You’re one person saying simdjson is faster, and you are also lacking a reproducible test. That doesn’t make your side of the argument particularly compelling either. simdjson is a massively popular library (~9.9k stars, the third most popular json library on GitHub), that is well known to beat the pants off of every other json library. Likewise the simdjson library has reproducible benchmarks in its own repo with charts and the like. So yes, I would consider that statement pretty much "worthless" given the popularity of simdjson.

glangdale 6y ago

Given how things turned out, this might be a good lesson as to whether "I measured my own library informally on some benchmarks that I couldn't be bothered to describe to you and it WINS!" is trustworthy in future. When we introduced simdjson, we went out of our way to explain (a) why we thought it might be faster than other libraries and (b) how people could replicate our measurements or find fault in our methodology. There were zero performance claims, public or private about the performance of simdjson before this material was available. In my long experience, undocumented and non-reproducible performance claims suffer from the fact that people generally stop benchmarking when they get the result that they wanted to hear, even if the absolute numbers involved are unreasonable. It's very easy to make mistakes in performance measurement (compiling simdjson with anything short of -O2 or -O3, doing repeated runs of a tiny input and thus getting perfect branch prediction behavior), but people generally fix these mistakes only on the side that makes their stuff look good. Back when I worked on the Hyperscan project ( https://github.com/intel/hyperscan https://github.com/intel/hyperscan ), we had an API call that allocated scratch memory (necessary due to Hyperscan's rather constrained approach to memory usage). I lost count of the number of times we had users run this allocation (which only ever needed to be done once per thread, effectively) inside their measurement loop. And whenever we were competing against an incumbent system and the people doing the measurement were the authors of that incumbent system, it was amazing how quickly they'd accept the "truth" of their benchmark run.

coder543 6y ago
I think it’s worth clarifying that I said I would “generally trust {author} more than {random internet person who hasn’t even tested author’s library}”. Are you saying that you trust random internet person more by default? Trusting one more than the other is not the same as blindly trusting one and never running my own tests. The author who put in the effort to develop and benchmark their library deserves more benefit of the doubt than some drive-by commenter, in my opinion. If I needed a high performance JSON parsing library, I would definitely set up a benchmark representative of my use case, and test a few different libraries. JSON libraries are usually pretty ergonomic and easy to swap between, so it shouldn’t be an arduous task. Your very first two comments in this thread were far more insightful than {random internet person} just saying over and over that simdjson can’t possibly be slower, and those comments of yours on their own merit would have removed the need for me to leave any comments whatsoever to defend the original HN post, even without needing to observe that you’re an expert in this field, which just lends further credibility to your comments. I think the rest of the discussion that occurred was really interesting to watch!