4 ms·
Distributed computing and networks, amirite? I'm sorry, but if you can't see that ACID transactions don't scale over multiple machines, then of course you will
by astral303 6y ago
Distributed computing and networks, amirite?
I'm sorry, but if you can't see that ACID transactions don't scale over multiple machines, then of course you will bemoan about how a large database gives up your prized feature.
Except what are you doing that is SO LARGE that you must have ACID semantics with it and are willing to go through all the hoopla to have cross-network transactions and joins? Because this hoopla is elusive and rare (impossible to do performantly, on a pragmatic scale). You are much better off designing your large volume loads to not require ACID, and to take all that is ACID-critical and keep it on so-called small databases.
This was the tradeoff that was long figured-out by NoSQL offerings, but somehow many engineers continue to insist on using 1 database for every problem, and then wonder why one solution is not reliable enough for critical data, or another doesn't scale for non-transaction-requiring data.
This is also why people writing the applications must also be the people operating the databases or at least be aware of the operational characteristics of the underlying databases. The choices you make in the application affect that. For many use cases, you can design reasonable concurrency with data without transactions, as long as you have atomic updates. But it takes sitting down, realizing that limitation, and then designing for it.
Another complaint I am reading about is nullalble fields. Fact of life is you are kidding that your database will enforce you schema and do all your schema migrations. You need to write application code that can tolerate an evolving schema. ALTER TABLE stops working at a certain scale. You cannot go back to every record and update it with new guarantees at the data store, it is simply not practical and honestly a waste of resources.
Even with SQL databases and the ALTER TABLE business, I've inevitably found our organizations writing custom schema migration logic in the application. So you still end up with a forced mess of application schema migration logic and database schema migration logic. I think long-term you should put you data migration logic into your app and stop trying to do that in your store. I've done that and have been much happier for it. All the devs know where the migration code goes (the app). The flexibility is considerable and not restricted to what the db layer might support.
Referential integrity and lack of joins is why also we denormalize data. Are your data structures overly normalized? (I'm sorry, uhhh ... "fully normalized" according to your databases professor?) Well just how normalized do they really have to be? How much can you denormalize without taking real risks or giving up real performance? I bet a lot more than you think. And how many ways can you come up with dealing with the denormalized approach, when you need to re-normalize? I bet more than zero. And how often do you need to do it? Likely never, or infrequently enough that so-called "bandaid fixes" to your denormalization are actually practical solutions, bemoaned only in classrooms devoid of production experience.