11 ms·
Rewriting a high performance vector database in Rust
- deleted 4y ago[deleted]
- fwip 4y ago> First of all, Python is a garbage collected language, which means it can be extremely slow for writing anything high performance at scale. I don't think garbage collection is in the top 3 causes of why Python is slow.
- mattnewton 4y agoThis might be a difference of semantics- there is a difference between garbage collection as a concept being slow and python's GIL approach. My understanding is that the GIL would almost always make the top 3 reason of why python is slow in practice - it works for a very specific single threaded execution model but can't really take advantage of modern processors.
- slt2021 4y agoI think GIL is not the reason for slowness, it just specifies single threaded interpreter execution model. You can always spin up more interpreters to take advantage of multiple cores. The reason for slowness - is the weak dynamic type system of Python. Every single instruction need to be type checked at runtime and thus making everything slow. Compare to C#/Java which have GC but both are amazingly fast, because these languages have stricter type system. If you add JIT on top (which can selectively replace MSIL/java opcodes with native machine instructions) and it makes perf on par with natively compiled languages like C++.
- kstrauser 4y agoI'd argue Python is strongly, dynamically typed, and that "weak" and "dynamic" are on different axes.
- deleted 4y ago[deleted]
- dymk 4y agoIt's always interesting how often people confuse the dichotomy of "strong/weak typing" with "static/dynamic typing". They are orthogonal to each other indeed - you can have languages that are in each of the four possible combinations.
- PartiallyTyped 4y ago> I think GIL is not the reason for slowness, it is the weak dynamic types of Python that make it slow. Python is structurally typed, that makes it dynamic, but it is not weak as there is no type coercion. > Every single instruction need to be type checked at runtime and thus making everything slow. This is also wrong, python does not type check anything, not in the "regular" manner of typechecking. It relies on structural typing, if it quacks like a duck, then it is treated like a duck. In fact, PyPy is an argument against your position as it still allows the same (more or less) behaviour that python has while operating a lot faster due to JIT. Python doesn't have the luxury of compiling that C# and Java, nor is the VM intended to be high performing.
- pjmlp 4y agoPyPy is still slower than the competition and largely ignored by the Python community.
- riku_iki 4y agoalso, is py still interpreted or jit-compiled?
- nicoburns 4y agoI believe the main reason Python is slow is because the cpython project has rejected performance enhancements in favour of keeping the code simple. JavaScript has similar semantics to Python, and its performance is not far behind Java/C# at all.
- orf 4y agoJS has vastly different semantics to Python, one that lends itself much more to the kind of JIT optimisations needed to make it more performant.
- fwip 4y agoWell, the author went on to rule out Java and Go because they garbage connected, so I don't think they meant the GIL.
- 0x457 4y ago> First of all, Python is a garbage, which means it can be extremely slow for writing anything high performance at scale. Fixed it.
- pjmlp 4y agoThankfully Fintech and military weapon control systems aren't high performance.
- jalino23 4y agorewriting everything in rust is not just a meme?
- likeabbas 4y agoMemes typically have some basis in reality. If your project reaches a point where it could benefit from fearless concurrency or better memory control, Rust is probably your best bet at the moment. I could see huge benefits from Kafka and Cassandra being re-written in Rust.
- dominotw 4y agokafka clone redpanda is written in rust ?
- tilt_error 4y agoNo. C++ and the C* (seastar) framework.
- eatonphil 4y agoHmm? I'm pretty sure it's written in C++. See also, their install dependencies script. https://github.com/redpanda-data/redpanda/blob/dev/install-dependencies.sh https://github.com/redpanda-data/redpanda/blob/dev/install-d...
- tilt_error 4y agoThey are rewritten in C++ already; Redpanda and ScyllaDB, respectively. Why waste the effort of rewriting it once again?
- arcticbull 4y agoC++ isn't better in the ways outlined. I'm not necessarily advocating it [1] but the parent's claim was that those programs could benefit from memory safety, thread safety and better concurrency and C++ does not deliver along that axis. [1] https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
- 22SAS 4y agoGlanced through the article, and I see no comparisons on how performance of the DB is in Rust versus their current C++ implementation, no mention of if maintaining the Rust code is easier than their C++ codebase, no stats on how devs are ramping up and how it's tackling their "hard to find a dev who knows both C++ and Python well" issue.
- dxhdr 4y agoArticle also states that the switch from C++ to Rust improves "low level optimized instruction sets, memory layout, and running async tasks." The first two are also strengths of C++, and for the third the article says that "Rust is async, and Tokio is the one of the most popular async providers ... However, it’s not great for running CPU intensive workloads, like with Pinecone." Puzzling.
- rwaksmunski 4y agoI've had more luck with async-std over Tokio for more CPU intensive workloads. But then again, I ran it on a kqueue platform so my experience is probably not representative.
- pclmulqdq 4y agoMy past experience with Rust async code is that both async-std and Tokio are fairly unimpressive on performance (as async code goes), particularly if you compare to ScyllaDB's runtime or other similar C++ async runtimes.
- jeroenhd 4y agoThe next paragraph they state: We looked at and compared several languages - Go, Java, C++, and Rust. We knew that C++ was harder to scale and maintain high quality as you build a dev team; that Java doesn’t provide the flexibility and systems programming language we needed; and that Go is also a garbage collected language. This left us with Rust. With Rust, the pros around performance, memory management, and ease of use outweighed the cons of it not yet being a very established language. In other words, they wanted to unify the programming languages and evaluated several. Rust won out of those for performance reasons. The article is a short recap of a 40 minute video. The video has more context and explains the intentions much better than the web page. They show a graph of performance over time as the rewrite progressed. There were some small optimisations and problems, a few big regressions, and then a huge improvement that was maintained. Looks like the rewrite process made the database perform significantly better. There's nothing on how much this was caused by the language switch itself, but that's functionally impossible: nobody is rewriting their application twice to see what rewrite is better.
- lesuorac 4y ago> As you can see in the above graph, a commit was merged that caused a huge spike. However, with Criterion, an open source benchmarking tool, we were easily able to identify it, mitigate it, and push a fix. Wonder what the commit was that caused a more than 2x regression and got a fix instead of an undo.
- rs_rs_rs_rs_rs 4y ago>In addition, it’s challenging to find developers with experience in both Python and C++ So you decided on a language that makes it even harder to find experienced developers?
- goodpoint 4y ago
- masklinn 4y agoAnecdotally, a lot of rust-curious people seem to know python. Projects like pyo3 help a lot as they make it much easier (= safe) to build native modules compared to C, let alone C++.
- pclmulqdq 4y agoRust is seen as more approachable by Javascript and Python devs, so they tend to learn it more often than C or C++. It is a lot more similar to JS than C++ is.
- 8jy89hui 4y ago> It is a lot more similar to JS than C++ is. That is strange as I have experienced the opposite. I've written all three languages and I've noticed that JS patterns don't translate well to Rust. Many C++ patterns translate well to Rust (albeit after a bit of borrow checker fighting). Thoughts?
- smilekzs 4y agoRust iterators gives you JS vibes, with gotchas mostly related to lambda captures lifetimes. Once you accept that sometimes a `collect` is the easiest way out, it feels okay at the end of the day.
- mistrial9 4y agoit is arguable that C++ in the modern days is no longer "one language" due to style, libraries, language features and code-base legacy; you have to find a coder that will fit your C++ world, not just C++
- Thaxll 4y ago
- ayewo 4y agoRewriting in Rust is not a meme, it's a cycle. Before Rust became viable, rewrites were done in Go. From the archives: - Rewriting a large production system in Go https://news.ycombinator.com/item?id=6234736 https://news.ycombinator.com/item?id=6234736 (2013) - How We Moved Our API From Ruby to Go https://news.ycombinator.com/item?id=9693743 https://news.ycombinator.com/item?id=9693743 (2015) - Matrix and Riot Confirmed as the Basis for France’s Secure Instant Messenger App https://news.ycombinator.com/item?id=16938545 https://news.ycombinator.com/item?id=16938545 (2018) - Toward Vagrant 3.0 https://news.ycombinator.com/item?id=27476676 https://news.ycombinator.com/item?id=27476676 (2021) - I’m porting the TypeScript type checker tsc to Go https://news.ycombinator.com/item?id=30074414 https://news.ycombinator.com/item?id=30074414 (2022)
- pjmlp 4y agoWhich is why old timers eventually learn to just deliver with boring technology.
- riskable 4y ago> Which is why old timers eventually learn to just deliver with ~~boring~~ buggy technology. There's a reason why folks take the time to rewrite things in Rust. No matter how good you are at C/C++ you will encounter bugs that you would not have if you had written it in Rust.
- pjmlp 4y agoAssuming there is even a Rust library replacement to start with. People keep forgetting C++ has 30 years of being deployed in production. Rust is 2022 is like using C++ in 1990's in terms of ecosystem.
- howinteresting 4y agoThis is just plain false. C++ in the 1990s had nothing like serde for example.
- ordiel 4y agoThis only proves some skilled people heve too much free time and no creativity...
- menaerus 4y ago
- deleted 4y ago[deleted]
- sbdivuvu 4y ago
- TehCorwiz 4y agoI care. Programming languages give us different frameworks and guardrails to express computational tasks similarly to how written and spoken languages give us a different set of concepts with which to express ideas. New languages mean a potentially different way of thinking about a problem. Some ideas which are difficult to express in one language are trivial in another. Discovering these differences is one of the joys of language learning. Language learning requires practice, and rewriting a known work (or translating it you might say) is a great way to deepen your understanding and test which ideas are easier or harder.
- menaerus 4y agoYou seem to come from a point as if I don't care about the intricate knowledge of programming languages or as if I don't care about learning how different languages manage to solve the same problem but from a different angle or with a different approach. I really do. What I don't see here in this article is none of that with some very loose arguments around picking Rust vs some other system programming language. Very typical of such type of articles regardless whether they're coming from Rust, C++, Go etc. "I rewrote XYZ in Rust" or "I rewrote XYZ in C++" or "I rewrote XYZ in Go" is basically like saying "I rewrote XYZ in a Turing-complete language". Go figure. How relevant that really is? Any problem can be solved in any programming language but the important distinction which is very often left unspoken and not demonstrated is what is it that you managed to improve? Is it that you managed to improve the code quality, shorten the development cycle, decreased the number of bugs or was it that you managed to improve the performance? In absence of ability to demonstrate the pros and cons of the approach one took, I will keep finding such articles biased with the lack of real content and therefore annoying. Some will call it programming language circle*erking.
- kaladin_1 4y agoI have no problem with people rewriting their projects in whatever language they see fit. What stood out for me in the article is him saying that it's difficult getting developers with experience in both Python and C++. So, I wonder, if his in-house devs could pick up Rust that they previously couldn't write, why does he think he can not hire a good programmer and charge him to learn the stack the company uses. Why must they employ someone that already writes Python or C++. Is Rust such a straight-forward language that people new to the language can write a very performant programme
- arriu 4y agoI agree, rust has a difficult learning curve. I’ve often heard at least a year is required to really feel confident.
- gotts 4y ago6 months is what I heard. I'm currently at ~2 and 6 sounds like a pretty good estimate.
- heavyset_go 4y agoIt takes a relatively short time to be proficient enough to make useful contributions, maybe half a year to a year to be confident. You can give an experienced developer the Rust book and have them contributing to a Rust codebase quickly.
- nicoburns 4y agoFor me it was ~2 weeks to get out of "learning mode" and into actually writing production code. 3 months to feel proficient. Maybe a year to feel that I had reached parity of fluency with languages I already knew.
- tialaramex 4y agoPeople really vary, it felt natural to me almost immediately but Rust's surveys indicate that some noticeable proportion of respondents report not being comfortable after more than a year. This is seen as an important thing to improve because of the slogan "A language empowering everyone to build reliable and efficient software". If Rust isn't empowering everyone because say, 10% of people who try to pick it up can't get anywhere that's not good enough.
- makmanalp 4y ago> If you’re using a higher level language, you’re not going to have access to how the memory is laid out. A simple change, like removing indirection in our list, was an order of magnitude improvement in our latencies since there’s memory prefetching in the compiler and the CPU can anticipate which vectors are going to be loaded next in order to improve the memory footprint. This is a common experience and I'm still surprised by the choice I constantly see to use a managed-memory languages to build a database - one of a very small set of special cases where having full control over the memory layout might just be a reasonable thing to want. In this universe (absent doing something completely absurd) it's not algorithmic complexity but managing data locality in the cache hierarchy (e.g. reading things from L3 vs main memory vs disk) that makes things orders of magnitude faster, especially if you're in the realm of doing things like SIMD operations to speed things up. Perhaps there's some level of suck we're willing to tolerate for all the other benefits you get, but I've been noticing a pattern of "align things just so at the higher level and hope they mostly turn out the way you want at the lower level" (e.g. also with the Apache java-y databases like hadoop / hbase / cassandra which I guess were mostly supposed to derive their total throughput from massive scale rather than per-node performance) which is a bit funny. But also it seems like part of Rust's promise was "low level but make it high level" which seems to be succeeding (zero-cost abstractions and whatnot), so I imagine this will get better over time - having not attempted a project like this myself, I'm not sure what the limitations you'd run up against are in terms of laying things out in memory in a favorable way - I imagine the kind of massive manually managed arena allocations and ad-hoc pointers going everywhere that one normally does doesn't really fly.
- einpoklum 4y agoThe authors of this post rewrote _their own_ DBMS in Rust. Which is perfectly ok, but I'm not sure I would trust them to decide that theirs is a "high-performance" DBMS. They don't have any benchmark results except images of their own internal performance measures; they don't offer any way of comparing their performance with other DBMSes (e.g. Vectorwise/Actian Vector, ClickHouse, DuckDB etc. - not to mention Oracle, MS or SAP offerings); and they only have marketing blurb about their numbers: "Up to 10x performance" (with no baseline of course). So, they took some DBMS (which is probably not so hot in terms of performance) and rewrote it in Rust. Surely possible, possibly useful, but not much to write home about if one is interested in DBMS performance.
- lupire 4y agoI liked the part where they said Python is too slow because it's garbage collected, and didn't show any metrics, and then built a new solution and Rust and didn't show metics to compare to the original system. Makes me think the eng lead just wanted to do Rust, and made up a rationalization.
- viig99 4y agoSame, "We knew that C++ was harder to scale and maintain high quality as you build a dev team" this just sounds arbitary and a weak excuse to use rust, C++-20 is as scalable as rust with a very rich ecosystem.
- throwaway2392 4y ago> ...with a very rich ecosystem. As someone who was heavy in C++14/17 it's much nicer in Go/Rust-land. As much as I love Bjarne, I'm not coming back.
- cercatrova 4y agoWell, we already know Python is inherently slower than Rust or any compiled language really, so does one really need metrics to know that the Rust implementation was faster?
- AtlasBarfed 4y agoShow me metrics where Python can compete with Java/Go, then we can bother with some discussion about Python vs C++/Rust. Actually, show me where Python is faster than Javascript. But yes, it is a given that C++/Rust is faster than Python unless there is some fundamental algorithmic foolishness done by the C++/Rust programmer. People with industry experience know why that is a given.
- smitty1e 4y agoWhat is a vector database? https://www.pinecone.io/learn/vector-database/ https://www.pinecone.io/learn/vector-database/ ...was less than informative.
- jiggawatts 4y agoStandard row-oriented databases store columns on disk like so: ABCABCABCABC Vector databases store them like this: AAAABBBBCCCC This allows faster queries if you just need one (or a few) columns, because unrelated columns don’t have to be processed at all. Caches are more efficient, vector CPU instructions can be used, etc… The downside is that random single row access is more expensive because a row has to be reassembled from many locations.
- code_biologist 4y agoYou're describing a column store there, not a vector store.
- smitty1e 4y agoI always thought a columnar store was a series of key:value dictionaries that could produce an RDBMS table result on demand.
- jiggawatts 4y agoThat’s a specific physical implantation. If using a key-value storage system then the keys are: Row store: “table id; row id; column id” Column store: “table id; column id; row id” The difference is the order used to sort the data in storage. In reality, most column store systems use a more complex partitioning scheme with row groups and the like…
- smitty1e 4y agoI mean, sure. But even DNS is a key:value store, or would you say that's over-simplifying?