3 ms·

To be fair, Vector databases does sound more official as "new and important technology" compared to the last db hype of NOSQL.

Guvante 3y ago
I mean NOSQL was hype with no substance but "you can scale more if you deal with not having ACID" is just generally true. Of course ACID scales to well into the Fortune 500 scale so...
- smitty1e 3y ago
  "No substance" seems a bit harsh. They mostly seem a tarted up associative array, sure, but a key-value store is a thing.
  - mnky9800n 3y ago
    But don't you prefer your key value stores to be wearing red lipstick and a pushup bra?
  - threeseed 3y ago
    > but a key-value store is a thing DynamoDB underpins much of AWS which in turn underpins a ridiculous number of web services. So definitely more than just a thing.
  - rakoo 3y ago
    A naturally distributed key/value store. NoSQL wasn't a product, it was a radical rethinking of the balance between what we wanted and what we needed. Turns out some absolute necessities of using a RDBMS or even using SQL are not that important, and relaxing those actually-not-requirements allows massive scalability, something we desperately needed, or some evolved data structures and computation, like what redis provides.
- deleted 3y ago
  [deleted]
threeseed 3y ago
> compared to the last db hype of NOSQL NoSQL has been around for over 20+ years. Since then Cassandra, DynamoDB, FoundationDB, MongoDB, Neo4J, Redis etc are not only still around but widely used and powering many of the services you use today.
- da_chicken 3y ago
  That's true, but 13 or 14 years ago, the NoSQL hype was calling for widespread abandonment of RDBMSs. They were claiming key-value stores and document databases would replace traditional relational databases the same way Reddit replaced Digg. That "MongoDB is web scale" video was lampooning the very real hype that people had, and how difficult it was to pick through it. It was as bad as the more recent blockchain hype. Even looking at HN's reactions to that video show a few comments that did not age very well (although most of them did): https://news.ycombinator.com/item?id=1636198 https://news.ycombinator.com/item?id=1636198 The sensible takes were that NoSQL would supplement and enhance RDBMSs, but the hype was much more than that.
- dagw 3y ago
  The difference is that today those DBs are generally considered as complements to SQL databases targeting specific use cases. Back in the peak NoSQL days people were pushing the narrative that you'd never need to use a SQL database for anything ever again.
  - AtlasBarfed 3y ago
    IMO this is the deal: RDBMS is such a mature and powerful technology, and with the vast power in modern single-node hardware, they will scale to considerable sizes. But there is a limit. Once a table or set of tables hit a certain scale, it needs to be distributed. Once you get to those scales, you are likely dealing with the threat of exponential data growth, so doing the "distributed Postgres" will bandaid the problem ... but you are starting to run into the CAP theorem's problems. You'll need to AP-scale the biggest data tables, and since that almost always means a big coding change lift and introduction of an AP-scaling database, it will almost always be a six months to a year transition, and possibly adding an entirely new DB technology (Cassandra / DynamoDB / maybe FoundationDB). I have yet to have anyone explain how joins scale on an AP distributed database except in limited situations where the joined data is somehow node-local to the other tables, usually some hierarchical situation. Otherwise you are pulling data from lots of nodes and aggregating and comparing the different sets to account for node drift / partitions / network failures. Cassandra and Dynamo basically say "you are scaling a single table/query/update/pre-joined data table". Which really isn't fun for RDBMS folks. Because it is a shitload of denormalization on top of all the AP headaches and distributed transactions / updates. As I said, megahuge machines really make that an outside case unless your use case really emphasizes the "A" in CAP, where write speed can be tolerated to be low and you want to tolerate entire cloud or datacenter outages. But yeah, nosql was never going to kill the rdbms. The functionality/power of rdbms/SQL is so so so so much higher. Just be aware when you're about to shoot over the limit and prepare for the code changes to handle true scale in the (unlikely) event you're going to need it.

Guvante 3y ago

I mean NOSQL was hype with no substance but "you can scale more if you deal with not having ACID" is just generally true. Of course ACID scales to well into the Fortune 500 scale so...

smitty1e 3y ago
"No substance" seems a bit harsh. They mostly seem a tarted up associative array, sure, but a key-value store is a thing.
- mnky9800n 3y ago
  But don't you prefer your key value stores to be wearing red lipstick and a pushup bra?
- threeseed 3y ago
  > but a key-value store is a thing DynamoDB underpins much of AWS which in turn underpins a ridiculous number of web services. So definitely more than just a thing.
- rakoo 3y ago
  A naturally distributed key/value store. NoSQL wasn't a product, it was a radical rethinking of the balance between what we wanted and what we needed. Turns out some absolute necessities of using a RDBMS or even using SQL are not that important, and relaxing those actually-not-requirements allows massive scalability, something we desperately needed, or some evolved data structures and computation, like what redis provides.
deleted 3y ago
[deleted]

threeseed 3y ago

> compared to the last db hype of NOSQL NoSQL has been around for over 20+ years. Since then Cassandra, DynamoDB, FoundationDB, MongoDB, Neo4J, Redis etc are not only still around but widely used and powering many of the services you use today.

da_chicken 3y ago
That's true, but 13 or 14 years ago, the NoSQL hype was calling for widespread abandonment of RDBMSs. They were claiming key-value stores and document databases would replace traditional relational databases the same way Reddit replaced Digg. That "MongoDB is web scale" video was lampooning the very real hype that people had, and how difficult it was to pick through it. It was as bad as the more recent blockchain hype. Even looking at HN's reactions to that video show a few comments that did not age very well (although most of them did): https://news.ycombinator.com/item?id=1636198 https://news.ycombinator.com/item?id=1636198 The sensible takes were that NoSQL would supplement and enhance RDBMSs, but the hype was much more than that.
dagw 3y ago
The difference is that today those DBs are generally considered as complements to SQL databases targeting specific use cases. Back in the peak NoSQL days people were pushing the narrative that you'd never need to use a SQL database for anything ever again.
- AtlasBarfed 3y ago
  IMO this is the deal: RDBMS is such a mature and powerful technology, and with the vast power in modern single-node hardware, they will scale to considerable sizes. But there is a limit. Once a table or set of tables hit a certain scale, it needs to be distributed. Once you get to those scales, you are likely dealing with the threat of exponential data growth, so doing the "distributed Postgres" will bandaid the problem ... but you are starting to run into the CAP theorem's problems. You'll need to AP-scale the biggest data tables, and since that almost always means a big coding change lift and introduction of an AP-scaling database, it will almost always be a six months to a year transition, and possibly adding an entirely new DB technology (Cassandra / DynamoDB / maybe FoundationDB). I have yet to have anyone explain how joins scale on an AP distributed database except in limited situations where the joined data is somehow node-local to the other tables, usually some hierarchical situation. Otherwise you are pulling data from lots of nodes and aggregating and comparing the different sets to account for node drift / partitions / network failures. Cassandra and Dynamo basically say "you are scaling a single table/query/update/pre-joined data table". Which really isn't fun for RDBMS folks. Because it is a shitload of denormalization on top of all the AP headaches and distributed transactions / updates. As I said, megahuge machines really make that an outside case unless your use case really emphasizes the "A" in CAP, where write speed can be tolerated to be low and you want to tolerate entire cloud or datacenter outages. But yeah, nosql was never going to kill the rdbms. The functionality/power of rdbms/SQL is so so so so much higher. Just be aware when you're about to shoot over the limit and prepare for the code changes to handle true scale in the (unlikely) event you're going to need it.