4 ms·

Glanced through the article, and I see no comparisons on how performance of the DB is in Rust versus their current C++ implementation, no mention of if maintain

by 22SAS 4y ago

Glanced through the article, and I see no comparisons on how performance of the DB is in Rust versus their current C++ implementation, no mention of if maintaining the Rust code is easier than their C++ codebase, no stats on how devs are ramping up and how it's tackling their "hard to find a dev who knows both C++ and Python well" issue.

dxhdr 4y ago
Article also states that the switch from C++ to Rust improves "low level optimized instruction sets, memory layout, and running async tasks." The first two are also strengths of C++, and for the third the article says that "Rust is async, and Tokio is the one of the most popular async providers ... However, it’s not great for running CPU intensive workloads, like with Pinecone." Puzzling.
- rwaksmunski 4y ago
  I've had more luck with async-std over Tokio for more CPU intensive workloads. But then again, I ran it on a kqueue platform so my experience is probably not representative.
- pclmulqdq 4y ago
  My past experience with Rust async code is that both async-std and Tokio are fairly unimpressive on performance (as async code goes), particularly if you compare to ScyllaDB's runtime or other similar C++ async runtimes.
jeroenhd 4y ago
The next paragraph they state: We looked at and compared several languages - Go, Java, C++, and Rust. We knew that C++ was harder to scale and maintain high quality as you build a dev team; that Java doesn’t provide the flexibility and systems programming language we needed; and that Go is also a garbage collected language. This left us with Rust. With Rust, the pros around performance, memory management, and ease of use outweighed the cons of it not yet being a very established language. In other words, they wanted to unify the programming languages and evaluated several. Rust won out of those for performance reasons. The article is a short recap of a 40 minute video. The video has more context and explains the intentions much better than the web page. They show a graph of performance over time as the rewrite progressed. There were some small optimisations and problems, a few big regressions, and then a huge improvement that was maintained. Looks like the rewrite process made the database perform significantly better. There's nothing on how much this was caused by the language switch itself, but that's functionally impossible: nobody is rewriting their application twice to see what rewrite is better.
- lijogdfljk 4y ago
  > but that's functionally impossible: nobody is rewriting their application twice to see what rewrite is better. Agreed, and hypothetically the 2nd rewrite should still be better than the first. So the language would have to make it significantly worse to outweigh the yet again experience in improving things. To be clear though i'm not stating that every rewrite is assured to be better. However a carefully considered rewrite has a much easier time making decisions learned from any warts discovered in previous implementations. God knows there's always some warts. As a Rust fanatic, i wouldn't expect Rust itself to be due to the performance gains. It's not expected to be faster than C/C++ typically. Just comparable.

by 22SAS 4y ago

dxhdr 4y ago

Article also states that the switch from C++ to Rust improves "low level optimized instruction sets, memory layout, and running async tasks." The first two are also strengths of C++, and for the third the article says that "Rust is async, and Tokio is the one of the most popular async providers ... However, it’s not great for running CPU intensive workloads, like with Pinecone." Puzzling.

rwaksmunski 4y ago
I've had more luck with async-std over Tokio for more CPU intensive workloads. But then again, I ran it on a kqueue platform so my experience is probably not representative.
pclmulqdq 4y ago
My past experience with Rust async code is that both async-std and Tokio are fairly unimpressive on performance (as async code goes), particularly if you compare to ScyllaDB's runtime or other similar C++ async runtimes.

jeroenhd 4y ago

The next paragraph they state: We looked at and compared several languages - Go, Java, C++, and Rust. We knew that C++ was harder to scale and maintain high quality as you build a dev team; that Java doesn’t provide the flexibility and systems programming language we needed; and that Go is also a garbage collected language. This left us with Rust. With Rust, the pros around performance, memory management, and ease of use outweighed the cons of it not yet being a very established language. In other words, they wanted to unify the programming languages and evaluated several. Rust won out of those for performance reasons. The article is a short recap of a 40 minute video. The video has more context and explains the intentions much better than the web page. They show a graph of performance over time as the rewrite progressed. There were some small optimisations and problems, a few big regressions, and then a huge improvement that was maintained. Looks like the rewrite process made the database perform significantly better. There's nothing on how much this was caused by the language switch itself, but that's functionally impossible: nobody is rewriting their application twice to see what rewrite is better.

lijogdfljk 4y ago
> but that's functionally impossible: nobody is rewriting their application twice to see what rewrite is better. Agreed, and hypothetically the 2nd rewrite should still be better than the first. So the language would have to make it significantly worse to outweigh the yet again experience in improving things. To be clear though i'm not stating that every rewrite is assured to be better. However a carefully considered rewrite has a much easier time making decisions learned from any warts discovered in previous implementations. God knows there's always some warts. As a Rust fanatic, i wouldn't expect Rust itself to be due to the performance gains. It's not expected to be faster than C/C++ typically. Just comparable.