4 ms·

> The NoSQL movement was born, in part, out of the frustration with the perceived stagnation and inadequacy of SQL databases I'm a little bit young, but isn't

by gen220 6y ago

> The NoSQL movement was born, in part, out of the frustration with the perceived stagnation and inadequacy of SQL databases I'm a little bit young, but isn't this a bit of a revisionist take, by the author? I thought that Amazon, Google, FB et al moved away from relational databases because the sharding logic they needed to build on top of these databases was approaching the complexity of a RDBMS. They didn't need strong consistency or support for complex queries, on the kind of data they were storing at scale, and so made compromises in those areas while engineering their purpose-built alternatives (Dynamo, BigTable, Cassandra). It's not that SQL didn't work, but that the persistence layer was too strong and therefore too slow for their very particular needs. It's like comparing a minivan/suv (mysql/postgres) to a drag racer (nosql databases). You don't want to drive your kids to soccer practice in a two-seater with no airbags, and a 5* crash safety rating isn't as important to the pink-slippers as horsepower and 0-60. Or am I missing something?

jldugger 6y ago
> I thought that Amazon, Google, FB et al moved away from relational databases because the sharding logic they needed to build Note that these companies did not move away from relational DBs until long after the "NoSQL is Web Scale" video. Yes, Google invented Big Table to help power search (and others), but their revenue system, AdWords, didn't move off MySQL until like 2015. And last I checked, Facebook is still a heavy user of MySQL with sharding. The original NoSQL software had two major value adds: you didn't have to learn a new language, and were faster (typically via disabling fsync -- the DBA equivalent of running with scissors). If you knew SQL or an ORM already, you were really just hoping mongoDB was faster magically. These days you can even just tune pgsql to support kv store formats: https://www.postgresql.org/docs/9.1/hstore.html https://www.postgresql.org/docs/9.1/hstore.html. Yes, you'll have to pay someone to know how to DBA pgsql, or pay AWS to pay someone, but I'm comfortable paying that price.
- uhoh-itsmaciek 6y ago
  >These days you can even just tune pgsql to support kv store formats And you can turn off fsync! Though if you do, disable synchronous_commit instead for most of the performance but none of the potential data corruption (you're still risking data loss, of course, just not corruption).
barrkel 6y ago
SQL RDBMSes solve a lot of problems / provide a lot of services: locking & concurrency control, computation, consistency, replication, transactions, etc. Not all of those services can scale horizontally. Concurrency control most especially, but other things which assert global invariants are often too expensive. Most NoSQL systems remove some of these features in order to scale horizontally. But the problems they solved remain, and need new solutions. This has two effects: it forces clients to do more which hopefully means doing less complex stuff (you can write mega expensive computations in SQL where the nested loops might offend you in handwritten code); and it means higher risk of bugs and more engineering effort for correctness (e.g. transactions in application, eventual consistency, reimplementation of transaction log in queuing systems, etc.). In transport analogies, RDBMS is like a lift helicopter, NoSQL is like a fleet of container ships. Or RDBMS is like a car, and NoSQL is like a train network. NoSQL is inflexible and needs lots of extra attention at the edges, while RDBMS needs a careful operator and doesn't work well beyond a certain scale unless you give everyone (or subgroups) their own instance (which could be sharding, it can work). And if you don't have a really big problem, NoSQL is probably the wrong choice, not because it's fast with few safety checks, it's because most of them do very little for you, they can just do a lot of that scaled out. If you're not scaling out, stuffing JSON into Postgres will give you a better experience even if you hate relational algebra.
oblio 6y ago
Isn't NoSQL at this point used in 99% of the cases for basically analytics on logs, especially for ads?
- peter_mcrae 6y ago
  That is probably true. But, I'd also suspect that in 99% of cases some form of that data (lots of post processing) ends back in data store where all kinds of people use SQL to analyze it. Two main reasons: 1) join it with other data 2) SQL is so widely understood across functions / roles
riku_iki 6y ago
> I thought that Amazon, Google, FB et al moved away from relational databases Surprisingly, Spanner has tables with columns and you can run SQL on top of it.
- dodobirdlord 6y ago
  Though notably the SQL interface was not present initially and was added later.

by gen220 6y ago

jldugger 6y ago

> I thought that Amazon, Google, FB et al moved away from relational databases because the sharding logic they needed to build Note that these companies did not move away from relational DBs until long after the "NoSQL is Web Scale" video. Yes, Google invented Big Table to help power search (and others), but their revenue system, AdWords, didn't move off MySQL until like 2015. And last I checked, Facebook is still a heavy user of MySQL with sharding. The original NoSQL software had two major value adds: you didn't have to learn a new language, and were faster (typically via disabling fsync -- the DBA equivalent of running with scissors). If you knew SQL or an ORM already, you were really just hoping mongoDB was faster magically. These days you can even just tune pgsql to support kv store formats: https://www.postgresql.org/docs/9.1/hstore.html https://www.postgresql.org/docs/9.1/hstore.html. Yes, you'll have to pay someone to know how to DBA pgsql, or pay AWS to pay someone, but I'm comfortable paying that price.

uhoh-itsmaciek 6y ago
>These days you can even just tune pgsql to support kv store formats And you can turn off fsync! Though if you do, disable synchronous_commit instead for most of the performance but none of the potential data corruption (you're still risking data loss, of course, just not corruption).

barrkel 6y ago

SQL RDBMSes solve a lot of problems / provide a lot of services: locking & concurrency control, computation, consistency, replication, transactions, etc. Not all of those services can scale horizontally. Concurrency control most especially, but other things which assert global invariants are often too expensive. Most NoSQL systems remove some of these features in order to scale horizontally. But the problems they solved remain, and need new solutions. This has two effects: it forces clients to do more which hopefully means doing less complex stuff (you can write mega expensive computations in SQL where the nested loops might offend you in handwritten code); and it means higher risk of bugs and more engineering effort for correctness (e.g. transactions in application, eventual consistency, reimplementation of transaction log in queuing systems, etc.). In transport analogies, RDBMS is like a lift helicopter, NoSQL is like a fleet of container ships. Or RDBMS is like a car, and NoSQL is like a train network. NoSQL is inflexible and needs lots of extra attention at the edges, while RDBMS needs a careful operator and doesn't work well beyond a certain scale unless you give everyone (or subgroups) their own instance (which could be sharding, it can work). And if you don't have a really big problem, NoSQL is probably the wrong choice, not because it's fast with few safety checks, it's because most of them do very little for you, they can just do a lot of that scaled out. If you're not scaling out, stuffing JSON into Postgres will give you a better experience even if you hate relational algebra.

oblio 6y ago

Isn't NoSQL at this point used in 99% of the cases for basically analytics on logs, especially for ads?

peter_mcrae 6y ago
That is probably true. But, I'd also suspect that in 99% of cases some form of that data (lots of post processing) ends back in data store where all kinds of people use SQL to analyze it. Two main reasons: 1) join it with other data 2) SQL is so widely understood across functions / roles

riku_iki 6y ago

> I thought that Amazon, Google, FB et al moved away from relational databases Surprisingly, Spanner has tables with columns and you can run SQL on top of it.

dodobirdlord 6y ago
Though notably the SQL interface was not present initially and was added later.