2 ms·
> the chance you can atomically update a system, whatever kind of database you're using, isn't something you necessarily get Postgres supports fully atomic DM
by ahachete 7y ago
> the chance you can atomically update a system, whatever kind of database you're using, isn't something you necessarily get
Postgres supports fully atomic DML and DDL operations. You can do transactions as big as you want (I have seen single transactions of multi-TB of changes).
> I don't think that's generally true
When you do a data migration of the kind we're talking here, let's say you are reversing how embeds who (you had employees embed departments, and now you want departments embed employees), you need to do a whole collection rewrite.
Since you want this to be incremental (without downtime) you necessarily need to deal with, among others:
* Change all your queries to do an "union" between the results you get from the $old model (those documents not migrated yet) and the $new model (those already migrated). This can be really hard.
* You need to change all your updates. They are not trivial neither, as they need to support now cases in which the pre-updated document is in two possible forms. Worse, depending on your migration code, you probably need to update the document in the $old form it is in the $old form, but in the $new form if it is in the $new form.
* Once migration is done, you need to undone all these "duplicated" code paths.
* At all times, you are dealing with potentially significant consistency issues. Due to the limitations to ACID operations in MongoDB --and the lack of them in other NoSQL systems--, you might see spurious effects of document in one form appearing or disappearing while being migrated.
> So relational is usually the default/general choice if you don't need performance at scale.
Define scale. One of my customers is gitlab.com. It handles millions of users, with dozens to hundreds of thousands of queries per second, and some TBs of daily changes, with a single Postgres cluster. Which still has significant headroom for vertical scaling.
Not everybody needs to get significantly bigger than this.