43 ms·

Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

203 points by mountainview 4y ago

pharmakom 4y ago
Fascinating how defensive C++ lifers can be. Rust builds on the knowledge of decades of C++ programming. It’s basically a compiler enforced set of C++ best practices. It’s strange how hostile some in the C++ community are to Rust.
- spoiler 4y ago
  I could be wrong, but I think there's few big camps of people. The camp that thinks Rust is just unnecessary fuss because "you can do it all" in C++ already. They pride themselves with knowing C++ esoterica, and don't like that Rust lowers the entry barrier to writing similar software. They want an exclusive club. They also see ownership as a nuisance. They know you can be equally reckless in Rust too, but they don't like that Rust is so "in your face" about it, too. The camp that is afraid Rust will marginalise C++ and hence push them towards undesirable (even if high paying) jobs. There's the camp that's jaded by seeing too many fads come and go, and probably just burnt out at some point. So, they just want to work with what's there and be left alone. They don't want to learn something that they know (incorrectly or not isn't for me to judge here) will be dead in the water soon. And they'll have wasted their energy on it for naught. Or in short, they see it as an unwelcome distraction. This camp also includes "but it has no ISO specification for aviation/etc" too. I think all these are wrong. I don't think C++ will be marginalised, it just won't be the default choice for certain domains. I don't think the extra tooling is a nuisance, if anything in my experience it unburdens the mind from non-domain concerns. And I also don't think it's a fad, given how productive the language makes people feel, and the fact the Rust Foundation is very serious about the longevity of the project, and that it has many big-player backers. There's also obviously others too; people come in about 8 billion flavours.
  - dxuh 4y ago
    I don't fall into any of the camps you mentioned specifically. I have been using C++ as my main programming language for a good 15 years and am a big fan of modern C++. I tried Rust for a month (every day) and I just feel like it gets in my way too much and it's just not worth the extra friction. A friend of mine (using Python as a physicist) wanted to try system programming recently and I told them to just try Rust and not trouble themselves with C++. I think Rust is the easier language to learn and probably just as powerful, but C++ just gets in your way less. It's not as hard to use correctly as people pretend it is (though I admit that years of experience and studying are required) and if you can use it relatively well you can write safe code without much trouble and you can also have a very high (higher than Rust) level of control seamlessly at the same time. I don't know many people with similar preferences, but most of them do game development, if that helps to contextualize a bit. There safety is not as important and development speed is much more important than for other applications.
    - simiones 4y ago
      > It's not as hard to use correctly as people pretend it is (though I admit that years of experience and studying are required) The original statement and the one in parens are directly at odds with each other. If every C++ dev requires years of experience and studying to use correctly, it follows that they have left years worth of code that is not done correctly in their wake. Also, no other (non-esoteric) language requires this high a barrier to write correctly. > if you can use it relatively well you can write safe code without much trouble And yet there isn't a single large C++ project that doesn't suffer from violations of memory safety - the one class of bugs that we know how to eliminate completely, if using a proper runtime system.
      - menaerus 4y ago
        > violations of memory safety - the one class of bugs that we know how to eliminate completely, Do we? Why exactly do we need sanitizers in Rust then?
  - batty_alex 4y ago
- mjburgess 4y ago
  You have to remember that we die. So what you learn til 30, say, you capitalise on till 70, and then you retire & die. There isn't another live to live until 30 "again", and then be within the Rust culture. Rust has, by analogy, stapled it's "Ninety-five Theses to the door" of C++ and those C++atholics are afraid their way of life is ending before they do.
  - humanrebar 4y ago
    Taking that analogy further, Catholicism didn't die out. Protestantism definitely grew, then there were hostilities and outright fighting for hundreds of years in some cases.
  - FpUser 4y ago
    >"So what you learn til 30, say, you capitalise on till 70, and then you retire & die." False BS >"those C++atholics are afraid their way of life is ending before they do" We are into insults area now.
- margorczynski 4y ago
  Well it will suck out a lot of potential jobs from the C++ market, at least in the longer time horizon. And many people invested almost their whole lives into doing C++.
- lelanthran 4y ago
  > It’s basically a compiler enforced set of C++ best practices. It’s strange how hostile some in the C++ community are to Rust. Well, the syntax is alien and new, it doesn't do OO the way 9 out of 10 working developers expect it to, almost all C++ popular design patterns have to be rejected, many of the claims("fearless concurrency") are exaggerated and the Rust evangelists are really really toxic when referring to C++ or C, often simply claiming that fewer bugs is worth the tradeoff from switching to Rust. I literally saw a comment on HN this past week complaining that OSes are not switching to Rust, almost completely ignoring the fact that any rewrite of a 3 decades old codebase is going to introduce more bugs than it is going to prevent, even if you have perfect memory safety (which you won't, in an OS). In all honesty, Rust would have been adopted orders of magnitudes faster had Rust evangelists been welcoming, and not accused of being defensive or hostile. I mean, look through this very page full of comments, and be honest, how many of the comments talking about C++ developers are welcoming to C++ developers?
  - Taywee 4y ago
    I've seen an order of magnitude more people complaining about Rust evangelists as I've actually seen Rust evangelists, and they've been significantly more hostile and toxic. The majority of Rust programmers I know were C++ programmers in the past. Many, including myself, use both professionally.
    - humanrebar 4y ago
      The headlines these days are definitely hostile, though. C++ is unsafe, a legacy language, etc.
    - lelanthran 4y ago
      > I've seen an order of magnitude more people complaining about Rust evangelists as I've actually seen Rust evangelists, and they've been significantly more hostile and toxic. Does this, to you, look like a response that is welcoming to C++ devs? I gave careful and impassive reasons, and you strike back with literal ad hominem attacks. This sort of reply is what makes Rust, as a community, look shallow-minded and toxic.
- yagni_dev 4y ago
  the fact we might have got sick of your evangelism and lies hasn't crossed your mind I guess >It’s basically a compiler enforced set of C++ best practices Its not, its a whole different programming paradigm where the compiler dictates how you write your software. You dont let your creative facilities to drive you or whats best for your domain - you just obey the language rules...
ThePhysicist 4y ago
What's often neglected is the excellent tooling around Rust, that alone is reason enough to switch from C++, where you have a really fragmented tooling ecosystem and you're often required to deal with complex Makefiles or build config as well as download dependencies manually. Most of that just disappears with Rust, unless you're developing code that interfaces with existing C/C++ libraries.
- humanrebar 4y ago
  C++ could improve its tooling but systems programming is inherently a bit of a fragmented ecosystem. I'm not sure we can shoehorn cargo into all the situations where C, C++, and Fortran have adapted to already.
- qalmakka 4y ago
  This, also. C and C++'s build model is a train wreck, especially in C++ code gets rebuilt dozens of times due to headers being simply plopped into whatever translation unit you are building. Rust efficiently solves this by defining building abstractions in the language itself, instead of relying on hacks such as #include, PCH, ...
  - pjmlp 4y ago
    C++ modules....
    - runevault 4y ago
      Has support for these gotten any better? Last time I was reading about it compiler support was under baked at best. If they get it fully working and stable I agree modules should go a long way to getting rid of it (entirely if you are willing to wrap old header files in modules yourself).
      - selfmodruntime 4y ago
        No. GCC still produces weird output at times.
dizhn 4y ago
"So, using C++ was a no-brain decision." Oooff.
- eminence32 4y ago
  The expression "no brainer" means that some decision is so obvious as to not require any thought. I think this article is saying that using C++ with a team of seasoned C++ developers was such an obvious thing to do that they didn't give it much thought.
  - dizhn 4y ago
    It doesn't say no-brainer. I think it should have said "The decision was a no brainer".
heisenbit 4y ago
> We deleted 276,406 lines Seven months, 10+ team but let’s average that over time to 10. Let‘s take 20 days/month and we arrive at roughly 200LOC per day. C++, database server code. Interesting…
blinkingled 4y ago
> So, using C++ was a no-brain decision Yes
lr1970 4y ago
From the article: > But as more and more engineers joined us, some shortcomings of C++ came to bite us: unreadable coding style, memory leak, segmentation fault, and more. One advantage of Rust of C/C++ that is underreported is the that Rust makes it easier to operate a team with diverse levels of experience/expertise. One or two senior engineers set up the tone, design and coding standards. Junior folks try to follow but if they screw up Rust offers more checks and guard rails limiting the blast radius. Many bugs simply do not make past the borrow checker and type system. This also helps when churn and attrition happens in the team. I witnessed quite a few horror stories when aspiring but not-yet competent C++ coders wrecked havoc in C++ code-base when seniors were busy with something else.
StreamBright 4y ago
As a hobby software engineer who mostly writes ETL jobs in Python the biggest selling point of Rust is Cargo. I usually use a lot of .clone() in my code and most of my fields are Strings which would make a seasoned Rust/C++ laugh at the code. However, the performance of novice Rust beats 10+ years of experience Python by a factor of 10 (favouring Rust). With Cargo I can build code that just runs. With Python it is always a gamble. Different arch? You need to install different packages to compile the C/C++/Fortran code if the library author did not care about WHL. Starting up an application always a gamble, do we have all the deps? Did we miss some? And so on. With Rust + Cargo I have confidence that the executable runs. Yes, the compilation is an extra step, but I would have that trade every single time for extra safety and reliability.
- ghostwriter 4y ago
  > As a hobby software engineer who mostly writes ETL jobs in Python the biggest selling point of Rust is Cargo. I usually use a lot of .clone() in my code and most of my fields are Strings which would make a seasoned Rust/C++ laugh at the code. [...] Yes, the compilation is an extra step, but I would have that trade every single time for extra safety and reliability. It sounds like you'd be better off with higher-level Haskell for these tasks.
  - StreamBright 4y ago
    Absolutely not. The last time I have tried Haskell it was failing some pretty basic tasks. The JSON library required to be re-compiled for some reason and it used 20+G of RAM when the build crashed. The community is flat out hostile towards Mac users and a basic request was closed with a comment that Mac is a broken platform and it should not be used. I do not have time for these, Rust offers a much better experience and the community is very helpful even if you are asking silly questions.
    - ghostwriter 4y ago
      > The community is flat out hostile towards Mac users and a basic request was closed with a comment that Mac is a broken platform and it should not be used. Have you got a link to that request?
blub 4y ago
Please do yourself a favor and don’t waste 5m of your life reading this article like I did. It’s about a team of allegedly experienced C++ developers at a start-up that spent seven months building a project only to spend another two months rewriting it in Rust and then a few more months moving from async to Tokio. But those lost months have not been in vain, because they have 1600 stars on Github. They actually bragged about their Github stars. It seems the Rust euphoria has reached such levels that people have stopped thinking critically both about the inexplicably bad business decisions and the near-zero technical content described in the article.
- devinthenull 4y ago
  sounds like a nice lifestyle business to enjoy ones hobbies
tw1984 4y ago
You can't revert this trend - C++ is a legacy language being depreciated by more and more projects. C++ and Rust are simply not competing on the same level. You don't need a complicated "enterprise" building farm/cluster to build your project, dependencies are managed by the guys who also write your compiler, there is a growing community that actually listen to your feedback and you don't need to wait 40 years to get networking support into the standard library. What C++ can offer? Languages like Rust, Golang and Zig all want their users to focus on their applications, while C++ just wants you to focus on C++. OO was a cool concept back in the 90s, but that was 30 years ago when you have 8MBytes of RAM if you are lucky - who wants to model every program using OO in 2023? It has never been a better time to depreciate the C++ language and codebases written in C++. Museum is a much better place for C++.
- blub 4y ago
  I’ve noticed that the tone is becoming rougher with each new Rust story. I’d be curious to know what kind of professional would choose to express themselves in this way, it’s pretty embarrassing. First of all, that blog post - as many Rust-themed blogs from start-ups - is a submarine article attempting to increase the notoriety of their product. Not enough to disqualify it, but a hint that it won’t necessarily be technically solid. And indeed, on a technical level it’s superficial and doesn’t give any code examples to substantiate its claims. Code is critically important, because as another recent blog showed, some people don’t know why it’s a bad idea to pass non-zero-terminated char buffers to C string functions and then turn this into a big story about the dangers of C++. The story also has some red flags, such as not being able to enforce coding standards. Code review is a near-mandatory quality control and teaching method that easily handles this. Clang-format and clang-tidy take care of the low-hanging fruit. Having segfaults is a really bad sign. So is not mentioning the words address sanitizer, valgrind or similar. Having memory leaks in 2021 is incomprehensible. This is an amateur mistake of the worst kind in modern C++. It means they’re writing C. I don’t have time to dig more in depth, but there’s a disturbing lack of critical thinking and skepticism in these comments.
  - tw1984 4y ago
    > I’d be curious to know what kind of professional would choose to express themselves in this way, it’s pretty embarrassing. Embarrassing for pointing out that OO is dead and it is not acceptable to ask users to wait almost 40 years to get networking support into the standard library? No, C++ standard committee members should be embarrassed, not those disappointed users. > that blog post - as many Rust-themed blogs from start-ups - is a submarine article attempting to increase the notoriety of their product. As any article published by an commercial entity, it is always about increasing its economic benefits. Personally, I don't know anyone from the company but their blog is just one more success story of depreciating C++ and I applaud for that. > Having memory leaks in 2021 is incomprehensible. This is an amateur mistake of the worst kind in modern C++. Sorry but I stopped reading after this. I've been writing software for the past 20 years, worked for the biggest names in industrial, had my PhD in CS about 15 years ago, I write code on almost daily basis (Linux kernel modules, database kernel, infrastructure kind of code). I make the above mentioned amateur mistakes pretty often. Not saying I am highly skilled, just saying if I can make such mistakes after working 20 years in the field, it wouldn't be rare for other people especially those young developers. btw, I recently had a nasty memory leak bug in golang that took me a couple days to reproduce and debug, yes, in a GC based language.
Dowwie 4y ago
Rust asyncio is still in its minimum-viable-product stage. My impression is that it will no long be considered MVP when it has structured concurrency. It would be great if authors posted honest insights about the limitations and work-arounds, if such exist. I would like to know what the core authors of Materialize or InfluxDB-IOX have to say about using Rust for building a database. They've found the dead bodies, if there are any.
fasteo 4y ago
>>> As an early-stage database startup, we entirely deleted our C++ codebase after 7 months of development I would love to hear how you explained this to your investors.
TheDesolate0 4y ago
[dead]
sidlls 4y ago
The more I read the "cons" in this article's C++ section the more I scratch my head. Code style is something that should be uniform and enforced. It's bikeshedding, but unless you're using a language with opinionated formatting (e.g. Go), it's absolutely a must have, and should be one of the first things done even at a startup. This really looks like a case of not seasoned developers throwing crap at the wall and seeing what sticks. I doubt their Rust implementation will go any better.
- Shorel 4y ago
  I totally agree with your first paragraph. There should have been code reviews and new developers should have been coached. About Rust... I'm open-minded, let's see what happens.
bayesian_horse 4y ago
10 seasoned C++ devs could probably make a killing in the finance sector right now.
- dilippkumar 4y ago
  Can you elaborate?
  - ghostwriter 4y ago
    They (finances, hft, hedges) pay significantly more than FAANG, mostly with plain cash too.
  - bayesian_horse 4y ago
    The finance sector has lots of bespoke C++ code driving for example hedge funds, even real time trading and C++ coders are aging and retiring... They certainly pay better, upfront, than a startup does.
andrewstuart 4y ago
Building a database is brave or stupid or both. Over the last 15 or 20 years I’ve tried lots of databases of all flavors and every time come back to Postgres. What can’t it do. And like Linux it’s written in C. Not that means much. “Never bet against Postgres”.
- jen20 4y ago
  > What can’t it do. Fail over without downtime or operator intervention.
- ly3xqhl8g9 4y ago
  However, you didn't return to postgres:6.0 (first formal release in 1997 [1]), you returned to postgres:latest, which included the lessons learned from lots of other databases, thinking here of postgres:9.2 (in 2012) adding native JSON support making Postgres basically NoSQL with extra steps. Contrary to popular wit, reinventing the wheel is always beneficial, if it will not bring something new to the world, it will at least make you understand and appreciate wheels better. [1] https://en.wikipedia.org/wiki/PostgreSQL#Release_history https://en.wikipedia.org/wiki/PostgreSQL#Release_history
- nhourcard 4y ago
  It depends on your use case. For example, Postgres will hit limitations for large streams of time series data on the ingestion side, and the standard SQL language of Postgres may make it challenging to navigate a dataset by time: sampling the data, time intervals, time-series joins (ASOF join), and these kinds of things are not easy or possible to do on Postgres. Why not combine Postgres for OLTP workloads alongside another database for fast analytics/time series, optimised to deal with high throughput ingestion and low latency queries? NB: I am one of the co-founder of an open-source time series database (questdb)
  - sgt 4y ago
    TimescaleDB
  - jjgreen 4y ago
hdhrufjdi 4y ago
> Rust is easy to learn. For seasoned C++ programmers, Rust is easy to learn. When they first start out, Rust learners usually spend most of their time making sense of ownership and lifetime. Even if they don't explicitly express these concepts in code, experienced C++ engineers always keep these two concepts in mind when programming in C++. Finally somebody understands this.
- lubesGordi 4y ago
  I was happy to see this as well. As a 'seasoned' C++ dev myself, when I learned Rust, I was pleasantly surprised to see those implicit concepts made explicit in code.
  - monocasa 4y ago
    Same. I think that I didn't go through the Rc<RefCell> phase that so many new to Rust go through because of C++ teaching me the same lessons in a much more brutal way. By the time I got to Rust I had learned to structure my code to sort of use pointers/references as capabilities, such that simply having knowledge of the bit pattern of the pointer was implicit access through that pointer since as a codebase evolves someone will keep pointers around or not go through.whatever song and dance you need to do to keep accesses through that pointer valid. That thought process translates very well to Rust.
    - techdragon 4y ago
      What I love about Rust is what I love about Python… “explicit is better than implicit” When I open something up to read it, to try and understand it, to dig in and fix something broken… having everything be exposed in the explicitly written code… is vastly more useful to me than trying to mentally interpret a language as I read it and remember the rules, the meta-programming rules… and all the implicit stuff that can affect how the program will behave. Lot of programming languages aren’t good at this explicitness, and I’ll be honest, it’s a genuine struggle for me picking up the languages like this, only made worse when the documentation is very prose like and fails to convey how anything actually works, eschewing that in favour of, at length, demonstrating the feel of a programming language through countless demonstration example that lack mechanical “what is this doing under the hood” explanation.
foxbee 4y ago
"The STL library lacks support for some modern programming tools, for example, native co-routine support. As a result, developers must rely on many community projects, and most lack long-term support." I couldn't agree more with this comment. As great as the community is, there's a lot of projects that are just not maintained. I wish this was different.
- crabbone 4y ago
  Rust doesn't have support for native co-routines. You rely on community project for that (Tokyo). At first I even thought it was an argument against Rust, not in favor...
  - Tamschi 4y ago
    It's "Tokio". Coroutines in Rust are a native language feature and their API is part of `std`, but the executors indeed come from community libraries. Tokio is one option, but as long as you don't contextually fork (for parallelism) you can in theory run the same coroutines on any other executor, like the ones from async-std. Spawning a sibling task (as opposed to mixed child task polling, which can be macroed inline) unfortunately isn't available through the common API so far, which is where the bulk of the mentioned incompatibilities stems from.
    - RedPanda250 4y ago
      Sorry if I'm missing something important as I have experience with C++ but not rust, but is async/await the right abstraction in Rust vs. the underlying generators which I assume is lower level and closer to C++20 Co-routines in terms of performance ? [1] In other words, it seems Rust co-routines at the level of Tokio are not as performant as C++ native Co-routines [2] and further down the thread, it's mentioned that once generators are stabilized, it might help build lower cost and performant stackless coroutines in Rust.[3] [1] https://github.com/rust-lang/rust/issues/43122#issuecomment-716871690 https://github.com/rust-lang/rust/issues/43122#issuecomment-... [2] https://github.com/rust-lang/rust/issues/43122#issuecomment-716256938 https://github.com/rust-lang/rust/issues/43122#issuecomment-... [3] https://github.com/rust-lang/rust/issues/43122#issuecomment-716263636 https://github.com/rust-lang/rust/issues/43122#issuecomment-...
ReflectedImage 4y ago
After 7 months of writing a database, you probably learned what your early bad decisions were, so in the rewrite you got to fix them.
mzimbres 4y ago
> But as more and more engineers joined us, some shortcomings > of C++ came to bite us: unreadable coding style, memory leak, > segmentation fault, and more. * unreadable coding style: This is not a C++ problem. * memory leak: Memory leak is an old C++ problem, since C++11 there is no reason for not using smart pointers. The only point that could be attributed to C++ is the segfaults perhaps, due to its lack of safety. But your other two points made me skeptical about Rust bringing you much benefit. I am specially concerned about throwing away months worth of code. Rewriting everything from scratch is what newbies love to do and will most likely get you in the same mess you were in. Try to learn with the error of others [1]. And btw, the problem is usually who writes the code and not which language is used. [1] https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
- flohofwoe 4y ago
  > since C++11 there is no reason for not using smart pointers. Smart pointers lure you into a dark alley where each tiny object lives in its own tiny heap allocation, and before you know it you have so much memory management sprinkled decentralized all over your code base that the memory management overhead becomes a performance problem, but at the same time it's too late to do anything about it because it would mean rewriting everything (this is then usually when the desperate search for a silver bullet like a "high performance memory allocator" starts). (not that Rust is necessarily better in that regard when people start to work around borrow checker restrictions with Box and Rc...)
  - gpderetta 4y ago
    I don't understand. Smart pointers replace dumb pointers not normal stack/inline object placement.
    - flohofwoe 4y ago
      Yes, but people need to be aware that "auto obj = make_unique..." or "auto obj = make_shared..." should be a very rare thing and not the norm (and I've seen plenty code bases like that). There needs to be a proper memory management strategy with the goal of minimizing heap allocation in random places in the code. E.g. automatic memory management doesn't resolve you from thinking just as much about memory management than doing it manually in the first place.
ahachete 4y ago
Interesting journey. > C/C++ is undoubtedly one of the most popular programming languages for building database systems. Most well-known database systems, including MySQL, PostgreSQL, Oracle, and IBM Db2, are created in C/C++. Considering C and C++ to be the same language is quite a stretch. They are fundamentally different.
- dzogchen 4y ago
  It is not a stretch it is just plain wrong. C++ has good C interoperability (most C is even valid C++) and most compilers that support compiling one also support the other, but I can't take seriously anyone that refers to "C/C++" as one language.
  - ahachete 4y ago
    Relationship is at best one-way only. C++ can mix and use C code. A C++ compiler will do fine with C. A C++ programmer will do more or less reasonable C code. C code will not accept C++. A C compiler will not support C++ (unless it's explicitly designed as a C++ compiler too). A C-only programmer will be as lost on a C++ codebase as on a Rust, Go or Java codebase.
    - l33t233372 4y ago
      This just makes sense since C++ is supposed to be an extension of C.
smallstepforman 4y ago
Author quoted C/C++, I immediately knew what conclusions I’d find. If on the other hand they had seasoned C++ devs (instead of C/C++), they might have had a different outcome.
- epage 4y ago
  What is with the modern C++ community treating "C/C++" as a shibboleth to gatekeep over? I'm a seasoned C++ dev (using it before C++98 was available, kept up with trends) and I've never seen this before until the last year or so. I've never had a problem with it as I see my C++ skills being tranferrable to C.
- pjmlp 4y ago
  That is the current issue with most C++ codebases nowadays, I only see modern C++ on conference slides, when I look into codebases even from ISO C++ members, it is always C++ full of C idioms no matter what. I bet most candidates to C++ job offers end up discovering the hardly reality of existing code.
  - bobajeff 4y ago
    That's a fundamental C++ problem. That and all the other idioms it supports. C++ is really like 10 languages and you always have find a mentor to show you the specific one you're working with in a codebase.
    - pjmlp 4y ago
      While I agree, note that it isn't exclusive of C++, most languages of similar age present similar issues. How many styles do we now have on C#11, Java 20, Python 3.11, PHP 8.2,....? This is why static analysis and linters are so relevant for keeping teams into a specific way of doing things, it is not like we can always justify start from scratch into simpler languages.
  - karterk 4y ago
    Can you give some examples of problematic C idioms in C++?
gizmo 4y ago
The popular databases we use today such as Postgres and Mysql are pretty terrible. Backups are extremely complicated to set up, and restoring from backups is hard. Multimaster setups almost work, but not completely. You want a geo-distributed high availability database? Forget about it. The popular databases today just don't do the things every web startup wants from a database. So I totally get why people want to make better database software, but at the same time, I don't understand how startups think the "move fast break things" approach will work here. The value of a buggy, incomplete, poorly documented database is 0, and the value of a database system that actually works is infinite. It's totally discrete. How this is not going to end up like every other database startup? Funding gets cut eventually because they can't demonstrate traction, but they can't demonstrate traction until the database product is actually way way better than the old dogs. Which will take 10 years if not 20. MongoDB is the only new database that managed to break through, and it was a single-threaded document store without proper ACID properties. For a database, it was simple, but it would still regularly eat your data. Today, something like MongoDB would never get traction. I wish these rising wave guys luck, but I don't see any indication they understand the obstacles they're facing.
- threeseed 4y ago
  > MongoDB is the only new database that managed to break through Snowflake, Cassandra, DuckDB, Clickhouse, Redis, InfluxDB, DynamoDB, BigQuery etc. There are many databases that have broken through they just are far more niche focused. > but it would still regularly eat your data No it didn't which is why it became so popular. The whole fsync saga was always overblown because (a) every client that shipped set it a safe default anyway and (b) it was fixed almost immediately.
  - crabbone 4y ago
    I would love to hear more about "fsync saga" if you have any references. I know that PostgreSQL had "problems" with fsync(), but they never ended. They just accepted defeat, but so would everyone else who relies on filesystems to store their data. It's even more ingrained than that. The way hardware works, and, especially, the communication protocols around it (eg. SCSI or NVMe) are structured, fsync() is always going to be a problem.
andreistan26 4y ago
To be honest golang is amazing for dbs.
habibur 4y ago
Every time I read of memory leaks in C++ codebase, I get no idea why RAII didn't work in those cases. Even if the codebase was huge and complex. Circular references? Some special cases why one can't use RAII? Forget unique pointer or smart pointers. People had been coding in C++ far before that without fearing memory leaks by following RAII principals.
- sidlls 4y ago
  RAII isn't a perfect safeguard against memory leaks. Rust doesn't provide 100% protection against them, either, especially when `unsafe` gets involved. It's better than C++, though, in this regard. There is no fool-proof way to avoid them, other than by never dynamically allocating memory.
  - Shorel 4y ago
    If everything is stack allocated, then the never dynamically allocating memory part will be true.
- crabbone 4y ago
  Unless you are very deep in system / embedded, most of the code that constitutes your program is not your code -- it's various libraries, frameworks, code generated by utilities etc. Often times this is the source of both unsafe interfaces and misunderstanding of the functionality on the part of the developer writing the code. Second to it is, probably, concurrency. RAII isn't going to save you here either. Then there's also input processing where you may run into a situation that causes resources not being de-allocated because you never anticipated a particular shape of the input. RAII works for very local and very well-defined things, it's not a good answer to dynamic situations or resources with complicated ownership.
yagni_dev 4y ago
>But as more and more engineers joined us, some shortcomings of C++ came to bite us: unreadable coding style, memory leak, segmentation fault, and more. - Seasoned devs but they never heard of clang tidy/format - Memory leaks on a new db codebase where each component should have clear ownership of data or pass to the next pipeline/stage - you dont even need smart poirters here, just a half decent design. - I am not even going to mention sanitizers, probably rocket science for them
- jasonhansel 4y ago
  Ownership is great! Which is why you should use a language that actually supports it, instead of one where it is just a semi-enforced convention.
- Shorel 4y ago
  From what I understood, clangd was not enough to make new developers write good enough code. The problem was not the seasoned core of developers, but the fresh hires. Now, this would be an interesting problem. Just as Python has PEP8 and some other programmable coding guidelines, we should have different sets of modern programmable C++ coding guidelines, which can be checked by clangd. And I mean modern, logical, Rust challenging, C++20 coding guidelines. Something this particular team would follow. Nonsense like Orthodox C++ doesn't cut it.
- crabbone 4y ago
  In my experience, segfaults are often a result of using C libraries in C++. In the project I work on, OpenSSL integration supplies a constant trickle of segfaults (it's not my general area, I'm just witnessing those and report to whoever works on it). This is especially true if you are trying to use those libraries in multi-threaded environment.
  - yagni_dev 4y ago
    Segfaults happen, you run valgrind/asan/tsan etc. As long as the codebase is not horrendous its easy to fix. Third party packages complicate things, have to choose/use wisely/appropriately
flumpcakes 4y ago
I really like Rust. I like C, and C++ too. I find a lot of C++ and C detractors are nit-picking and over-promising the-new-shiny-thing, or are probably not great engineers to begin with and are blaming the tool. In this instance, I'm leaning towards the latter. Although Rust is an improvement in many ways, I actually think it is becoming more C++ like with every release. I think it has a good chance of over-taking C++'s complexity, even if it has a much better user story around build & package/module/library management.
- pdimitar 4y ago
  As a guy who is writing Rust and found it a true value-add in his career, I agree with this and it's my #1 worry. Rust can already be very arcane to decipher sometimes, especially when you start getting errors from async libraries. My entirely egotistical opinion is that the maintainers of the language have to take a very good and critical look of the current status quo and start systematically killing off any complexity they can. Personally I don't want C++ 2.0. Tire people enough and they'll just move to another language again. I know I have, several times in my career, and will do so again if I have to.
  - spoiler 4y ago
    > Rust can already be very arcane to decipher sometimes, especially when you start getting errors from async libraries. For what it's worth, the community, library authors, and most importantly the language design teams are all very aware of this. And there's efforts being made to make all these pain points things more ergonomics and approachable. I think it just a slower process than we'd like. I think in part it's so slow might be due to how the decision making and work is structured; it's very open, but also slow. Stuff like GATs took ages to design and implement, and yeet and parts of async are similar. These are done very carefully and they also let decisions marinate for a while before further action. It's not a critique, since I think it's good and wouldn't know a recommendation to speed it up. So, it's getting better, but slowly
    - flumpcakes 4y ago
jayp1418 4y ago
I wonder why people don't use Ada Programming Language also ? It has Spark subset for formal verification also
- peoplefromibiza 4y ago
  [flagged]
- deleted 4y ago
  [deleted]
- dist1ll 4y ago
  Does Ada have strong concurrency support and compile-time data race freedom?
  - ghostwriter 4y ago
    Haskell does have both
    - dist1ll 4y ago
      Haskell is not a high-performance systems programming language, not sure how it's relevant in a discussion about Rust, C++ and Ada
      - ghostwriter 4y ago
        It's relevant because many people using Rust are using it for building web backends where high-performance systems language is not needed, yet the safe concurrency is required.

203 points by mountainview 4y ago

Fascinating how defensive C++ lifers can be. Rust builds on the knowledge of decades of C++ programming. It’s basically a compiler enforced set of C++ best practices. It’s strange how hostile some in the C++ community are to Rust.

spoiler 4y ago
I could be wrong, but I think there's few big camps of people. The camp that thinks Rust is just unnecessary fuss because "you can do it all" in C++ already. They pride themselves with knowing C++ esoterica, and don't like that Rust lowers the entry barrier to writing similar software. They want an exclusive club. They also see ownership as a nuisance. They know you can be equally reckless in Rust too, but they don't like that Rust is so "in your face" about it, too. The camp that is afraid Rust will marginalise C++ and hence push them towards undesirable (even if high paying) jobs. There's the camp that's jaded by seeing too many fads come and go, and probably just burnt out at some point. So, they just want to work with what's there and be left alone. They don't want to learn something that they know (incorrectly or not isn't for me to judge here) will be dead in the water soon. And they'll have wasted their energy on it for naught. Or in short, they see it as an unwelcome distraction. This camp also includes "but it has no ISO specification for aviation/etc" too. I think all these are wrong. I don't think C++ will be marginalised, it just won't be the default choice for certain domains. I don't think the extra tooling is a nuisance, if anything in my experience it unburdens the mind from non-domain concerns. And I also don't think it's a fad, given how productive the language makes people feel, and the fact the Rust Foundation is very serious about the longevity of the project, and that it has many big-player backers. There's also obviously others too; people come in about 8 billion flavours.
- dxuh 4y ago
  I don't fall into any of the camps you mentioned specifically. I have been using C++ as my main programming language for a good 15 years and am a big fan of modern C++. I tried Rust for a month (every day) and I just feel like it gets in my way too much and it's just not worth the extra friction. A friend of mine (using Python as a physicist) wanted to try system programming recently and I told them to just try Rust and not trouble themselves with C++. I think Rust is the easier language to learn and probably just as powerful, but C++ just gets in your way less. It's not as hard to use correctly as people pretend it is (though I admit that years of experience and studying are required) and if you can use it relatively well you can write safe code without much trouble and you can also have a very high (higher than Rust) level of control seamlessly at the same time. I don't know many people with similar preferences, but most of them do game development, if that helps to contextualize a bit. There safety is not as important and development speed is much more important than for other applications.
  - simiones 4y ago
    > It's not as hard to use correctly as people pretend it is (though I admit that years of experience and studying are required) The original statement and the one in parens are directly at odds with each other. If every C++ dev requires years of experience and studying to use correctly, it follows that they have left years worth of code that is not done correctly in their wake. Also, no other (non-esoteric) language requires this high a barrier to write correctly. > if you can use it relatively well you can write safe code without much trouble And yet there isn't a single large C++ project that doesn't suffer from violations of memory safety - the one class of bugs that we know how to eliminate completely, if using a proper runtime system.
    - menaerus 4y ago
      > violations of memory safety - the one class of bugs that we know how to eliminate completely, Do we? Why exactly do we need sanitizers in Rust then?
- batty_alex 4y ago
mjburgess 4y ago
You have to remember that we die. So what you learn til 30, say, you capitalise on till 70, and then you retire & die. There isn't another live to live until 30 "again", and then be within the Rust culture. Rust has, by analogy, stapled it's "Ninety-five Theses to the door" of C++ and those C++atholics are afraid their way of life is ending before they do.
- humanrebar 4y ago
  Taking that analogy further, Catholicism didn't die out. Protestantism definitely grew, then there were hostilities and outright fighting for hundreds of years in some cases.
- FpUser 4y ago
  >"So what you learn til 30, say, you capitalise on till 70, and then you retire & die." False BS >"those C++atholics are afraid their way of life is ending before they do" We are into insults area now.
margorczynski 4y ago
Well it will suck out a lot of potential jobs from the C++ market, at least in the longer time horizon. And many people invested almost their whole lives into doing C++.
lelanthran 4y ago
> It’s basically a compiler enforced set of C++ best practices. It’s strange how hostile some in the C++ community are to Rust. Well, the syntax is alien and new, it doesn't do OO the way 9 out of 10 working developers expect it to, almost all C++ popular design patterns have to be rejected, many of the claims("fearless concurrency") are exaggerated and the Rust evangelists are really really toxic when referring to C++ or C, often simply claiming that fewer bugs is worth the tradeoff from switching to Rust. I literally saw a comment on HN this past week complaining that OSes are not switching to Rust, almost completely ignoring the fact that any rewrite of a 3 decades old codebase is going to introduce more bugs than it is going to prevent, even if you have perfect memory safety (which you won't, in an OS). In all honesty, Rust would have been adopted orders of magnitudes faster had Rust evangelists been welcoming, and not accused of being defensive or hostile. I mean, look through this very page full of comments, and be honest, how many of the comments talking about C++ developers are welcoming to C++ developers?
- Taywee 4y ago
  I've seen an order of magnitude more people complaining about Rust evangelists as I've actually seen Rust evangelists, and they've been significantly more hostile and toxic. The majority of Rust programmers I know were C++ programmers in the past. Many, including myself, use both professionally.
  - humanrebar 4y ago
    The headlines these days are definitely hostile, though. C++ is unsafe, a legacy language, etc.
  - lelanthran 4y ago
    > I've seen an order of magnitude more people complaining about Rust evangelists as I've actually seen Rust evangelists, and they've been significantly more hostile and toxic. Does this, to you, look like a response that is welcoming to C++ devs? I gave careful and impassive reasons, and you strike back with literal ad hominem attacks. This sort of reply is what makes Rust, as a community, look shallow-minded and toxic.
yagni_dev 4y ago
the fact we might have got sick of your evangelism and lies hasn't crossed your mind I guess >It’s basically a compiler enforced set of C++ best practices Its not, its a whole different programming paradigm where the compiler dictates how you write your software. You dont let your creative facilities to drive you or whats best for your domain - you just obey the language rules...

ThePhysicist 4y ago

What's often neglected is the excellent tooling around Rust, that alone is reason enough to switch from C++, where you have a really fragmented tooling ecosystem and you're often required to deal with complex Makefiles or build config as well as download dependencies manually. Most of that just disappears with Rust, unless you're developing code that interfaces with existing C/C++ libraries.

humanrebar 4y ago
C++ could improve its tooling but systems programming is inherently a bit of a fragmented ecosystem. I'm not sure we can shoehorn cargo into all the situations where C, C++, and Fortran have adapted to already.
qalmakka 4y ago
This, also. C and C++'s build model is a train wreck, especially in C++ code gets rebuilt dozens of times due to headers being simply plopped into whatever translation unit you are building. Rust efficiently solves this by defining building abstractions in the language itself, instead of relying on hacks such as #include, PCH, ...
- pjmlp 4y ago
  C++ modules....
  - runevault 4y ago
    Has support for these gotten any better? Last time I was reading about it compiler support was under baked at best. If they get it fully working and stable I agree modules should go a long way to getting rid of it (entirely if you are willing to wrap old header files in modules yourself).
    - selfmodruntime 4y ago
      No. GCC still produces weird output at times.

dizhn 4y ago

"So, using C++ was a no-brain decision." Oooff.

eminence32 4y ago
The expression "no brainer" means that some decision is so obvious as to not require any thought. I think this article is saying that using C++ with a team of seasoned C++ developers was such an obvious thing to do that they didn't give it much thought.
- dizhn 4y ago
  It doesn't say no-brainer. I think it should have said "The decision was a no brainer".

heisenbit 4y ago

> We deleted 276,406 lines Seven months, 10+ team but let’s average that over time to 10. Let‘s take 20 days/month and we arrive at roughly 200LOC per day. C++, database server code. Interesting…

blinkingled 4y ago

> So, using C++ was a no-brain decision Yes

lr1970 4y ago

From the article: > But as more and more engineers joined us, some shortcomings of C++ came to bite us: unreadable coding style, memory leak, segmentation fault, and more. One advantage of Rust of C/C++ that is underreported is the that Rust makes it easier to operate a team with diverse levels of experience/expertise. One or two senior engineers set up the tone, design and coding standards. Junior folks try to follow but if they screw up Rust offers more checks and guard rails limiting the blast radius. Many bugs simply do not make past the borrow checker and type system. This also helps when churn and attrition happens in the team. I witnessed quite a few horror stories when aspiring but not-yet competent C++ coders wrecked havoc in C++ code-base when seniors were busy with something else.

StreamBright 4y ago

As a hobby software engineer who mostly writes ETL jobs in Python the biggest selling point of Rust is Cargo. I usually use a lot of .clone() in my code and most of my fields are Strings which would make a seasoned Rust/C++ laugh at the code. However, the performance of novice Rust beats 10+ years of experience Python by a factor of 10 (favouring Rust). With Cargo I can build code that just runs. With Python it is always a gamble. Different arch? You need to install different packages to compile the C/C++/Fortran code if the library author did not care about WHL. Starting up an application always a gamble, do we have all the deps? Did we miss some? And so on. With Rust + Cargo I have confidence that the executable runs. Yes, the compilation is an extra step, but I would have that trade every single time for extra safety and reliability.

ghostwriter 4y ago
> As a hobby software engineer who mostly writes ETL jobs in Python the biggest selling point of Rust is Cargo. I usually use a lot of .clone() in my code and most of my fields are Strings which would make a seasoned Rust/C++ laugh at the code. [...] Yes, the compilation is an extra step, but I would have that trade every single time for extra safety and reliability. It sounds like you'd be better off with higher-level Haskell for these tasks.
- StreamBright 4y ago
  Absolutely not. The last time I have tried Haskell it was failing some pretty basic tasks. The JSON library required to be re-compiled for some reason and it used 20+G of RAM when the build crashed. The community is flat out hostile towards Mac users and a basic request was closed with a comment that Mac is a broken platform and it should not be used. I do not have time for these, Rust offers a much better experience and the community is very helpful even if you are asking silly questions.
  - ghostwriter 4y ago
    > The community is flat out hostile towards Mac users and a basic request was closed with a comment that Mac is a broken platform and it should not be used. Have you got a link to that request?

blub 4y ago

Please do yourself a favor and don’t waste 5m of your life reading this article like I did. It’s about a team of allegedly experienced C++ developers at a start-up that spent seven months building a project only to spend another two months rewriting it in Rust and then a few more months moving from async to Tokio. But those lost months have not been in vain, because they have 1600 stars on Github. They actually bragged about their Github stars. It seems the Rust euphoria has reached such levels that people have stopped thinking critically both about the inexplicably bad business decisions and the near-zero technical content described in the article.

devinthenull 4y ago
sounds like a nice lifestyle business to enjoy ones hobbies

tw1984 4y ago

You can't revert this trend - C++ is a legacy language being depreciated by more and more projects. C++ and Rust are simply not competing on the same level. You don't need a complicated "enterprise" building farm/cluster to build your project, dependencies are managed by the guys who also write your compiler, there is a growing community that actually listen to your feedback and you don't need to wait 40 years to get networking support into the standard library. What C++ can offer? Languages like Rust, Golang and Zig all want their users to focus on their applications, while C++ just wants you to focus on C++. OO was a cool concept back in the 90s, but that was 30 years ago when you have 8MBytes of RAM if you are lucky - who wants to model every program using OO in 2023? It has never been a better time to depreciate the C++ language and codebases written in C++. Museum is a much better place for C++.

blub 4y ago
I’ve noticed that the tone is becoming rougher with each new Rust story. I’d be curious to know what kind of professional would choose to express themselves in this way, it’s pretty embarrassing. First of all, that blog post - as many Rust-themed blogs from start-ups - is a submarine article attempting to increase the notoriety of their product. Not enough to disqualify it, but a hint that it won’t necessarily be technically solid. And indeed, on a technical level it’s superficial and doesn’t give any code examples to substantiate its claims. Code is critically important, because as another recent blog showed, some people don’t know why it’s a bad idea to pass non-zero-terminated char buffers to C string functions and then turn this into a big story about the dangers of C++. The story also has some red flags, such as not being able to enforce coding standards. Code review is a near-mandatory quality control and teaching method that easily handles this. Clang-format and clang-tidy take care of the low-hanging fruit. Having segfaults is a really bad sign. So is not mentioning the words address sanitizer, valgrind or similar. Having memory leaks in 2021 is incomprehensible. This is an amateur mistake of the worst kind in modern C++. It means they’re writing C. I don’t have time to dig more in depth, but there’s a disturbing lack of critical thinking and skepticism in these comments.
- tw1984 4y ago
  > I’d be curious to know what kind of professional would choose to express themselves in this way, it’s pretty embarrassing. Embarrassing for pointing out that OO is dead and it is not acceptable to ask users to wait almost 40 years to get networking support into the standard library? No, C++ standard committee members should be embarrassed, not those disappointed users. > that blog post - as many Rust-themed blogs from start-ups - is a submarine article attempting to increase the notoriety of their product. As any article published by an commercial entity, it is always about increasing its economic benefits. Personally, I don't know anyone from the company but their blog is just one more success story of depreciating C++ and I applaud for that. > Having memory leaks in 2021 is incomprehensible. This is an amateur mistake of the worst kind in modern C++. Sorry but I stopped reading after this. I've been writing software for the past 20 years, worked for the biggest names in industrial, had my PhD in CS about 15 years ago, I write code on almost daily basis (Linux kernel modules, database kernel, infrastructure kind of code). I make the above mentioned amateur mistakes pretty often. Not saying I am highly skilled, just saying if I can make such mistakes after working 20 years in the field, it wouldn't be rare for other people especially those young developers. btw, I recently had a nasty memory leak bug in golang that took me a couple days to reproduce and debug, yes, in a GC based language.

Dowwie 4y ago

Rust asyncio is still in its minimum-viable-product stage. My impression is that it will no long be considered MVP when it has structured concurrency. It would be great if authors posted honest insights about the limitations and work-arounds, if such exist. I would like to know what the core authors of Materialize or InfluxDB-IOX have to say about using Rust for building a database. They've found the dead bodies, if there are any.

fasteo 4y ago

>>> As an early-stage database startup, we entirely deleted our C++ codebase after 7 months of development I would love to hear how you explained this to your investors.

TheDesolate0 4y ago

[dead]

sidlls 4y ago

The more I read the "cons" in this article's C++ section the more I scratch my head. Code style is something that should be uniform and enforced. It's bikeshedding, but unless you're using a language with opinionated formatting (e.g. Go), it's absolutely a must have, and should be one of the first things done even at a startup. This really looks like a case of not seasoned developers throwing crap at the wall and seeing what sticks. I doubt their Rust implementation will go any better.

Shorel 4y ago
I totally agree with your first paragraph. There should have been code reviews and new developers should have been coached. About Rust... I'm open-minded, let's see what happens.

bayesian_horse 4y ago

10 seasoned C++ devs could probably make a killing in the finance sector right now.

dilippkumar 4y ago
Can you elaborate?
- ghostwriter 4y ago
  They (finances, hft, hedges) pay significantly more than FAANG, mostly with plain cash too.
- bayesian_horse 4y ago
  The finance sector has lots of bespoke C++ code driving for example hedge funds, even real time trading and C++ coders are aging and retiring... They certainly pay better, upfront, than a startup does.

andrewstuart 4y ago

Building a database is brave or stupid or both. Over the last 15 or 20 years I’ve tried lots of databases of all flavors and every time come back to Postgres. What can’t it do. And like Linux it’s written in C. Not that means much. “Never bet against Postgres”.

jen20 4y ago
> What can’t it do. Fail over without downtime or operator intervention.
ly3xqhl8g9 4y ago
However, you didn't return to postgres:6.0 (first formal release in 1997 [1]), you returned to postgres:latest, which included the lessons learned from lots of other databases, thinking here of postgres:9.2 (in 2012) adding native JSON support making Postgres basically NoSQL with extra steps. Contrary to popular wit, reinventing the wheel is always beneficial, if it will not bring something new to the world, it will at least make you understand and appreciate wheels better. [1] https://en.wikipedia.org/wiki/PostgreSQL#Release_history https://en.wikipedia.org/wiki/PostgreSQL#Release_history
nhourcard 4y ago
It depends on your use case. For example, Postgres will hit limitations for large streams of time series data on the ingestion side, and the standard SQL language of Postgres may make it challenging to navigate a dataset by time: sampling the data, time intervals, time-series joins (ASOF join), and these kinds of things are not easy or possible to do on Postgres. Why not combine Postgres for OLTP workloads alongside another database for fast analytics/time series, optimised to deal with high throughput ingestion and low latency queries? NB: I am one of the co-founder of an open-source time series database (questdb)
- sgt 4y ago
  TimescaleDB
- jjgreen 4y ago

hdhrufjdi 4y ago

> Rust is easy to learn. For seasoned C++ programmers, Rust is easy to learn. When they first start out, Rust learners usually spend most of their time making sense of ownership and lifetime. Even if they don't explicitly express these concepts in code, experienced C++ engineers always keep these two concepts in mind when programming in C++. Finally somebody understands this.

lubesGordi 4y ago
I was happy to see this as well. As a 'seasoned' C++ dev myself, when I learned Rust, I was pleasantly surprised to see those implicit concepts made explicit in code.
- monocasa 4y ago
  Same. I think that I didn't go through the Rc<RefCell> phase that so many new to Rust go through because of C++ teaching me the same lessons in a much more brutal way. By the time I got to Rust I had learned to structure my code to sort of use pointers/references as capabilities, such that simply having knowledge of the bit pattern of the pointer was implicit access through that pointer since as a codebase evolves someone will keep pointers around or not go through.whatever song and dance you need to do to keep accesses through that pointer valid. That thought process translates very well to Rust.
  - techdragon 4y ago
    What I love about Rust is what I love about Python… “explicit is better than implicit” When I open something up to read it, to try and understand it, to dig in and fix something broken… having everything be exposed in the explicitly written code… is vastly more useful to me than trying to mentally interpret a language as I read it and remember the rules, the meta-programming rules… and all the implicit stuff that can affect how the program will behave. Lot of programming languages aren’t good at this explicitness, and I’ll be honest, it’s a genuine struggle for me picking up the languages like this, only made worse when the documentation is very prose like and fails to convey how anything actually works, eschewing that in favour of, at length, demonstrating the feel of a programming language through countless demonstration example that lack mechanical “what is this doing under the hood” explanation.

foxbee 4y ago

"The STL library lacks support for some modern programming tools, for example, native co-routine support. As a result, developers must rely on many community projects, and most lack long-term support." I couldn't agree more with this comment. As great as the community is, there's a lot of projects that are just not maintained. I wish this was different.

crabbone 4y ago
Rust doesn't have support for native co-routines. You rely on community project for that (Tokyo). At first I even thought it was an argument against Rust, not in favor...
- Tamschi 4y ago
  It's "Tokio". Coroutines in Rust are a native language feature and their API is part of `std`, but the executors indeed come from community libraries. Tokio is one option, but as long as you don't contextually fork (for parallelism) you can in theory run the same coroutines on any other executor, like the ones from async-std. Spawning a sibling task (as opposed to mixed child task polling, which can be macroed inline) unfortunately isn't available through the common API so far, which is where the bulk of the mentioned incompatibilities stems from.
  - RedPanda250 4y ago
    Sorry if I'm missing something important as I have experience with C++ but not rust, but is async/await the right abstraction in Rust vs. the underlying generators which I assume is lower level and closer to C++20 Co-routines in terms of performance ? [1] In other words, it seems Rust co-routines at the level of Tokio are not as performant as C++ native Co-routines [2] and further down the thread, it's mentioned that once generators are stabilized, it might help build lower cost and performant stackless coroutines in Rust.[3] [1] https://github.com/rust-lang/rust/issues/43122#issuecomment-716871690 https://github.com/rust-lang/rust/issues/43122#issuecomment-... [2] https://github.com/rust-lang/rust/issues/43122#issuecomment-716256938 https://github.com/rust-lang/rust/issues/43122#issuecomment-... [3] https://github.com/rust-lang/rust/issues/43122#issuecomment-716263636 https://github.com/rust-lang/rust/issues/43122#issuecomment-...

ReflectedImage 4y ago

After 7 months of writing a database, you probably learned what your early bad decisions were, so in the rewrite you got to fix them.

mzimbres 4y ago

> But as more and more engineers joined us, some shortcomings > of C++ came to bite us: unreadable coding style, memory leak, > segmentation fault, and more. * unreadable coding style: This is not a C++ problem. * memory leak: Memory leak is an old C++ problem, since C++11 there is no reason for not using smart pointers. The only point that could be attributed to C++ is the segfaults perhaps, due to its lack of safety. But your other two points made me skeptical about Rust bringing you much benefit. I am specially concerned about throwing away months worth of code. Rewriting everything from scratch is what newbies love to do and will most likely get you in the same mess you were in. Try to learn with the error of others [1]. And btw, the problem is usually who writes the code and not which language is used. [1] https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...

flohofwoe 4y ago
> since C++11 there is no reason for not using smart pointers. Smart pointers lure you into a dark alley where each tiny object lives in its own tiny heap allocation, and before you know it you have so much memory management sprinkled decentralized all over your code base that the memory management overhead becomes a performance problem, but at the same time it's too late to do anything about it because it would mean rewriting everything (this is then usually when the desperate search for a silver bullet like a "high performance memory allocator" starts). (not that Rust is necessarily better in that regard when people start to work around borrow checker restrictions with Box and Rc...)
- gpderetta 4y ago
  I don't understand. Smart pointers replace dumb pointers not normal stack/inline object placement.
  - flohofwoe 4y ago
    Yes, but people need to be aware that "auto obj = make_unique..." or "auto obj = make_shared..." should be a very rare thing and not the norm (and I've seen plenty code bases like that). There needs to be a proper memory management strategy with the goal of minimizing heap allocation in random places in the code. E.g. automatic memory management doesn't resolve you from thinking just as much about memory management than doing it manually in the first place.

ahachete 4y ago

Interesting journey. > C/C++ is undoubtedly one of the most popular programming languages for building database systems. Most well-known database systems, including MySQL, PostgreSQL, Oracle, and IBM Db2, are created in C/C++. Considering C and C++ to be the same language is quite a stretch. They are fundamentally different.

dzogchen 4y ago
It is not a stretch it is just plain wrong. C++ has good C interoperability (most C is even valid C++) and most compilers that support compiling one also support the other, but I can't take seriously anyone that refers to "C/C++" as one language.
- ahachete 4y ago
  Relationship is at best one-way only. C++ can mix and use C code. A C++ compiler will do fine with C. A C++ programmer will do more or less reasonable C code. C code will not accept C++. A C compiler will not support C++ (unless it's explicitly designed as a C++ compiler too). A C-only programmer will be as lost on a C++ codebase as on a Rust, Go or Java codebase.
  - l33t233372 4y ago
    This just makes sense since C++ is supposed to be an extension of C.

smallstepforman 4y ago

Author quoted C/C++, I immediately knew what conclusions I’d find. If on the other hand they had seasoned C++ devs (instead of C/C++), they might have had a different outcome.

epage 4y ago
What is with the modern C++ community treating "C/C++" as a shibboleth to gatekeep over? I'm a seasoned C++ dev (using it before C++98 was available, kept up with trends) and I've never seen this before until the last year or so. I've never had a problem with it as I see my C++ skills being tranferrable to C.
pjmlp 4y ago
That is the current issue with most C++ codebases nowadays, I only see modern C++ on conference slides, when I look into codebases even from ISO C++ members, it is always C++ full of C idioms no matter what. I bet most candidates to C++ job offers end up discovering the hardly reality of existing code.
- bobajeff 4y ago
  That's a fundamental C++ problem. That and all the other idioms it supports. C++ is really like 10 languages and you always have find a mentor to show you the specific one you're working with in a codebase.
  - pjmlp 4y ago
    While I agree, note that it isn't exclusive of C++, most languages of similar age present similar issues. How many styles do we now have on C#11, Java 20, Python 3.11, PHP 8.2,....? This is why static analysis and linters are so relevant for keeping teams into a specific way of doing things, it is not like we can always justify start from scratch into simpler languages.
- karterk 4y ago
  Can you give some examples of problematic C idioms in C++?

gizmo 4y ago

The popular databases we use today such as Postgres and Mysql are pretty terrible. Backups are extremely complicated to set up, and restoring from backups is hard. Multimaster setups almost work, but not completely. You want a geo-distributed high availability database? Forget about it. The popular databases today just don't do the things every web startup wants from a database. So I totally get why people want to make better database software, but at the same time, I don't understand how startups think the "move fast break things" approach will work here. The value of a buggy, incomplete, poorly documented database is 0, and the value of a database system that actually works is infinite. It's totally discrete. How this is not going to end up like every other database startup? Funding gets cut eventually because they can't demonstrate traction, but they can't demonstrate traction until the database product is actually way way better than the old dogs. Which will take 10 years if not 20. MongoDB is the only new database that managed to break through, and it was a single-threaded document store without proper ACID properties. For a database, it was simple, but it would still regularly eat your data. Today, something like MongoDB would never get traction. I wish these rising wave guys luck, but I don't see any indication they understand the obstacles they're facing.

threeseed 4y ago
> MongoDB is the only new database that managed to break through Snowflake, Cassandra, DuckDB, Clickhouse, Redis, InfluxDB, DynamoDB, BigQuery etc. There are many databases that have broken through they just are far more niche focused. > but it would still regularly eat your data No it didn't which is why it became so popular. The whole fsync saga was always overblown because (a) every client that shipped set it a safe default anyway and (b) it was fixed almost immediately.
- crabbone 4y ago
  I would love to hear more about "fsync saga" if you have any references. I know that PostgreSQL had "problems" with fsync(), but they never ended. They just accepted defeat, but so would everyone else who relies on filesystems to store their data. It's even more ingrained than that. The way hardware works, and, especially, the communication protocols around it (eg. SCSI or NVMe) are structured, fsync() is always going to be a problem.

andreistan26 4y ago

To be honest golang is amazing for dbs.

habibur 4y ago

Every time I read of memory leaks in C++ codebase, I get no idea why RAII didn't work in those cases. Even if the codebase was huge and complex. Circular references? Some special cases why one can't use RAII? Forget unique pointer or smart pointers. People had been coding in C++ far before that without fearing memory leaks by following RAII principals.

sidlls 4y ago
RAII isn't a perfect safeguard against memory leaks. Rust doesn't provide 100% protection against them, either, especially when `unsafe` gets involved. It's better than C++, though, in this regard. There is no fool-proof way to avoid them, other than by never dynamically allocating memory.
- Shorel 4y ago
  If everything is stack allocated, then the never dynamically allocating memory part will be true.
crabbone 4y ago
Unless you are very deep in system / embedded, most of the code that constitutes your program is not your code -- it's various libraries, frameworks, code generated by utilities etc. Often times this is the source of both unsafe interfaces and misunderstanding of the functionality on the part of the developer writing the code. Second to it is, probably, concurrency. RAII isn't going to save you here either. Then there's also input processing where you may run into a situation that causes resources not being de-allocated because you never anticipated a particular shape of the input. RAII works for very local and very well-defined things, it's not a good answer to dynamic situations or resources with complicated ownership.

yagni_dev 4y ago

>But as more and more engineers joined us, some shortcomings of C++ came to bite us: unreadable coding style, memory leak, segmentation fault, and more. - Seasoned devs but they never heard of clang tidy/format - Memory leaks on a new db codebase where each component should have clear ownership of data or pass to the next pipeline/stage - you dont even need smart poirters here, just a half decent design. - I am not even going to mention sanitizers, probably rocket science for them

jasonhansel 4y ago
Ownership is great! Which is why you should use a language that actually supports it, instead of one where it is just a semi-enforced convention.
Shorel 4y ago
From what I understood, clangd was not enough to make new developers write good enough code. The problem was not the seasoned core of developers, but the fresh hires. Now, this would be an interesting problem. Just as Python has PEP8 and some other programmable coding guidelines, we should have different sets of modern programmable C++ coding guidelines, which can be checked by clangd. And I mean modern, logical, Rust challenging, C++20 coding guidelines. Something this particular team would follow. Nonsense like Orthodox C++ doesn't cut it.
crabbone 4y ago
In my experience, segfaults are often a result of using C libraries in C++. In the project I work on, OpenSSL integration supplies a constant trickle of segfaults (it's not my general area, I'm just witnessing those and report to whoever works on it). This is especially true if you are trying to use those libraries in multi-threaded environment.
- yagni_dev 4y ago
  Segfaults happen, you run valgrind/asan/tsan etc. As long as the codebase is not horrendous its easy to fix. Third party packages complicate things, have to choose/use wisely/appropriately

flumpcakes 4y ago

I really like Rust. I like C, and C++ too. I find a lot of C++ and C detractors are nit-picking and over-promising the-new-shiny-thing, or are probably not great engineers to begin with and are blaming the tool. In this instance, I'm leaning towards the latter. Although Rust is an improvement in many ways, I actually think it is becoming more C++ like with every release. I think it has a good chance of over-taking C++'s complexity, even if it has a much better user story around build & package/module/library management.

pdimitar 4y ago
As a guy who is writing Rust and found it a true value-add in his career, I agree with this and it's my #1 worry. Rust can already be very arcane to decipher sometimes, especially when you start getting errors from async libraries. My entirely egotistical opinion is that the maintainers of the language have to take a very good and critical look of the current status quo and start systematically killing off any complexity they can. Personally I don't want C++ 2.0. Tire people enough and they'll just move to another language again. I know I have, several times in my career, and will do so again if I have to.
- spoiler 4y ago
  > Rust can already be very arcane to decipher sometimes, especially when you start getting errors from async libraries. For what it's worth, the community, library authors, and most importantly the language design teams are all very aware of this. And there's efforts being made to make all these pain points things more ergonomics and approachable. I think it just a slower process than we'd like. I think in part it's so slow might be due to how the decision making and work is structured; it's very open, but also slow. Stuff like GATs took ages to design and implement, and yeet and parts of async are similar. These are done very carefully and they also let decisions marinate for a while before further action. It's not a critique, since I think it's good and wouldn't know a recommendation to speed it up. So, it's getting better, but slowly
  - flumpcakes 4y ago

jayp1418 4y ago

I wonder why people don't use Ada Programming Language also ? It has Spark subset for formal verification also

peoplefromibiza 4y ago
[flagged]
deleted 4y ago
[deleted]
dist1ll 4y ago
Does Ada have strong concurrency support and compile-time data race freedom?
- ghostwriter 4y ago
  Haskell does have both
  - dist1ll 4y ago
    Haskell is not a high-performance systems programming language, not sure how it's relevant in a discussion about Rust, C++ and Ada
    - ghostwriter 4y ago
      It's relevant because many people using Rust are using it for building web backends where high-performance systems language is not needed, yet the safe concurrency is required.