3 ms·

> Migrating a Python script to C++ could well be worthwhile in some instances. However, it’s not all sunshine and rainbows with C++. Diving into C++ requires mo

> Migrating a Python script to C++ could well be worthwhile in some instances. However, it’s not all sunshine and rainbows with C++. Diving into C++ requires more mental effort. That's a non-trivial statement! A statement which makes this article seem like it was written 10 years ago. Did anyone believe the C++ cited is all the C++ required to make this work? Go see the C++ as tested: https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/blob/master/2024/09/27/main.cpp https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/... By this I mean -- if Python isn't fast enough -- isn't that why we have Rust? Compare to my quick Rust: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=4b5b1632c74b7e8391a866d135311bf4 https://play.rust-lang.org/?version=stable&mode=debug&editio... It kinda crushes the CLI tool use case. Especially w/r/t JSON.

blub 2y ago
Your rust code is 32 lines vs 16 and it probably downloads dozens of micro-dependencies. Unfortunately, the C++ also uses CMake fetch to get its dependencies. I guess this is fine for a small demo, but it would be better to inline them in the repo. At least that’s a realistic option for C++ and thise deps are easier to validate anyway. As far as readability goes, I’m surprised that the C++ is consistently better than Rust in this regard. Most of those extra lines which double the size are for ceremony which nobody cares about when trying to understand the algorithm. As an example, we have a Rust tool at work which is quite nice (and memory safe :D), but because it has a fat match in one file, most of the code is actually in the middle of the screen. I also spent half an hour looking at the 150 micro-deps it was pulling in and then another half hour verifying if I unknowingly pulled the serde binary blob on my machine (I didn’t). Validating Rust deps is a major PITA both for regular users and distros. That’s why I write my command tools either in Golang (preferred), Python (if no unusual external deps needed) or C++ if I need speed. Most of the time I am fine with Golang and Python.
- mustache_kimono 2y ago
  > Your rust code is 32 lines vs 16 and it probably downloads dozens of micro-dependencies. You counted my comments and whitespace as LOC? > As far as readability goes, I’m surprised that the C++ is consistently better than Rust in this regard. Most of those extra lines which double the size are for ceremony which nobody cares about when trying to understand the algorithm. Wait, seriously? I mean -- I'm kinda shocked anyone feels this way, but different strokes for different folks. For instance, I've never understood why something like this is normal for every C++ code example: simdjson::padded_string json = simdjson::padded_string::load(argv[1]); You can raw dog imports in Rust code too, I suppose, but why not spend another line of code and add a `using` at the top? Notice, also, one way the C++ version saved LOC is the C++ library takes a string path, opens the file path, and read its contents all at once, which is like half my program? I'd also suggest that, for many, the LOC has very little to do with how understandable the program is. > Validating Rust deps is a major PITA both for regular users and distros. First, doesn't the C++ version also draw in its own dependency? AFAIK serde is pretty lightweight in terms of deps. Second, is jerking off CMake and/or writing your own JSON parser better? I'm starting to understand -- C++ is this strange land where mind numbing pain is actually better. Up is down. Down is up! The segfaults are character building! This reminds me of one of the more amusing conversations I've had re: C++. Someone was writing JIT compiler for their language and had chosen to implement in Rust. A person rather pointedly asked: Safety isn't a concern here, so why didn't you use C++? This is amusing to me only because C++ is much more difficult for me than Rust. It's often inscrutably complex. So my response would have been -- Let me get this straight -- someone isn't making me? I have a choice, and you want me to choose C++? Why should I be miserable forever?
pkulak 2y ago
Are all those lambdas to walk the tree down to the field you want so that it doesn’t have to parse all the bytes you don’t care about? Is it the efficiency of SAX without being horrible to read and write?
- mustache_kimono 2y ago
  > Are all those lambdas to walk the tree down to the field you want so that it doesn’t have to parse all the bytes you don’t care about? Is it the efficiency of SAX without being horrible to read and write? Just the ugly/simple-minded way I wrote it. You can index into JSON with serde too, like so: let wealth_str: &str = data["user"]["wealth"].as_str().expect("Could not parse JSON value."); See: https://docs.rs/serde_json/latest/serde_json/value/trait.Index.html https://docs.rs/serde_json/latest/serde_json/value/trait.Ind...
  - pkulak 2y ago
    Ah, gotcha, thanks.
anonymoushn 2y ago
Can you post benchmarks? serde_json isn't known for its throughput.
- mustache_kimono 2y ago
  I think you may have missed my point? Which was -- I think the Rust is more understandable, is easier to write, and is easier to get correct, more quickly. My version also isn't 1:1. I only wrote an approximation of what the author noted was the most relevant portion in the blog post. If you'd like to benchmark my version you're certainly welcome to, once you've cleaned it up a bunch. The C++ version seems to be SIMD accelerated though? If you want a more relevant performance comparison, perhaps you should find SIMD accelerated Rust JSON library.
  - dxuh 2y ago
    Frankly the C++ version is lacking whitespace and is shorter than it needs to be and as someone who uses C++ daily and has used simdjson before it is still much easier to read than the Rust version. Have you considered that this is somewhat subjective?
    - tialaramex 2y ago
      They seem very similar to me except that 1) I can see the Rust has error handling and I can't tell whether the C++ just doesn't bother or whether there is hidden control flow, since both these very different outcomes are usual in C++. My impression of simdjson is that it just doesn't care - handling errors wouldn't be "ridiculously fast". 2) "Filenames are text" nonsense in the C++. In a sense this is related to error handling, but it's not an error on the popular systems for your files to have names which aren't text, the OS is too lazy to require that. Mainly, when I look at the C++ I see bug opportunities, some of which existed in the Python and some did not, and I imagine the expenditure of time to hunt down those bugs and fix them (or work around them) compared to the tiny saving of using this code.
  - tourmalinetaco 2y ago
    Both C++ and Rust are equally unintuitive to understand, especially compared to Python which is effectively pseudo-code already. Whichever is more “understandable” is left solely up to the familiarity of the developer in question.
pjmlp 2y ago
Except nowadays vcpkg and conan exist, and I also don't have to wait for the world to be built for my tiny app, thanks to the ecosystem support for binary libraries.

blub 2y ago

Your rust code is 32 lines vs 16 and it probably downloads dozens of micro-dependencies. Unfortunately, the C++ also uses CMake fetch to get its dependencies. I guess this is fine for a small demo, but it would be better to inline them in the repo. At least that’s a realistic option for C++ and thise deps are easier to validate anyway. As far as readability goes, I’m surprised that the C++ is consistently better than Rust in this regard. Most of those extra lines which double the size are for ceremony which nobody cares about when trying to understand the algorithm. As an example, we have a Rust tool at work which is quite nice (and memory safe :D), but because it has a fat match in one file, most of the code is actually in the middle of the screen. I also spent half an hour looking at the 150 micro-deps it was pulling in and then another half hour verifying if I unknowingly pulled the serde binary blob on my machine (I didn’t). Validating Rust deps is a major PITA both for regular users and distros. That’s why I write my command tools either in Golang (preferred), Python (if no unusual external deps needed) or C++ if I need speed. Most of the time I am fine with Golang and Python.

mustache_kimono 2y ago
> Your rust code is 32 lines vs 16 and it probably downloads dozens of micro-dependencies. You counted my comments and whitespace as LOC? > As far as readability goes, I’m surprised that the C++ is consistently better than Rust in this regard. Most of those extra lines which double the size are for ceremony which nobody cares about when trying to understand the algorithm. Wait, seriously? I mean -- I'm kinda shocked anyone feels this way, but different strokes for different folks. For instance, I've never understood why something like this is normal for every C++ code example: simdjson::padded_string json = simdjson::padded_string::load(argv[1]); You can raw dog imports in Rust code too, I suppose, but why not spend another line of code and add a `using` at the top? Notice, also, one way the C++ version saved LOC is the C++ library takes a string path, opens the file path, and read its contents all at once, which is like half my program? I'd also suggest that, for many, the LOC has very little to do with how understandable the program is. > Validating Rust deps is a major PITA both for regular users and distros. First, doesn't the C++ version also draw in its own dependency? AFAIK serde is pretty lightweight in terms of deps. Second, is jerking off CMake and/or writing your own JSON parser better? I'm starting to understand -- C++ is this strange land where mind numbing pain is actually better. Up is down. Down is up! The segfaults are character building! This reminds me of one of the more amusing conversations I've had re: C++. Someone was writing JIT compiler for their language and had chosen to implement in Rust. A person rather pointedly asked: Safety isn't a concern here, so why didn't you use C++? This is amusing to me only because C++ is much more difficult for me than Rust. It's often inscrutably complex. So my response would have been -- Let me get this straight -- someone isn't making me? I have a choice, and you want me to choose C++? Why should I be miserable forever?

pkulak 2y ago

Are all those lambdas to walk the tree down to the field you want so that it doesn’t have to parse all the bytes you don’t care about? Is it the efficiency of SAX without being horrible to read and write?

mustache_kimono 2y ago
> Are all those lambdas to walk the tree down to the field you want so that it doesn’t have to parse all the bytes you don’t care about? Is it the efficiency of SAX without being horrible to read and write? Just the ugly/simple-minded way I wrote it. You can index into JSON with serde too, like so: let wealth_str: &str = data["user"]["wealth"].as_str().expect("Could not parse JSON value."); See: https://docs.rs/serde_json/latest/serde_json/value/trait.Index.html https://docs.rs/serde_json/latest/serde_json/value/trait.Ind...
- pkulak 2y ago
  Ah, gotcha, thanks.

anonymoushn 2y ago

Can you post benchmarks? serde_json isn't known for its throughput.

mustache_kimono 2y ago
I think you may have missed my point? Which was -- I think the Rust is more understandable, is easier to write, and is easier to get correct, more quickly. My version also isn't 1:1. I only wrote an approximation of what the author noted was the most relevant portion in the blog post. If you'd like to benchmark my version you're certainly welcome to, once you've cleaned it up a bunch. The C++ version seems to be SIMD accelerated though? If you want a more relevant performance comparison, perhaps you should find SIMD accelerated Rust JSON library.
- dxuh 2y ago
  Frankly the C++ version is lacking whitespace and is shorter than it needs to be and as someone who uses C++ daily and has used simdjson before it is still much easier to read than the Rust version. Have you considered that this is somewhat subjective?
  - tialaramex 2y ago
    They seem very similar to me except that 1) I can see the Rust has error handling and I can't tell whether the C++ just doesn't bother or whether there is hidden control flow, since both these very different outcomes are usual in C++. My impression of simdjson is that it just doesn't care - handling errors wouldn't be "ridiculously fast". 2) "Filenames are text" nonsense in the C++. In a sense this is related to error handling, but it's not an error on the popular systems for your files to have names which aren't text, the OS is too lazy to require that. Mainly, when I look at the C++ I see bug opportunities, some of which existed in the Python and some did not, and I imagine the expenditure of time to hunt down those bugs and fix them (or work around them) compared to the tiny saving of using this code.
- tourmalinetaco 2y ago
  Both C++ and Rust are equally unintuitive to understand, especially compared to Python which is effectively pseudo-code already. Whichever is more “understandable” is left solely up to the familiarity of the developer in question.

pjmlp 2y ago

Except nowadays vcpkg and conan exist, and I also don't have to wait for the world to be built for my tiny app, thanks to the ecosystem support for binary libraries.