3 ms·

On a side note, I find it a bit ironic how NoSQL was all the rage back in the last decade or so but in 2022, NoSQL DBs are racing to add SQL querying back to ke

On a side note, I find it a bit ironic how NoSQL was all the rage back in the last decade or so but in 2022, NoSQL DBs are racing to add SQL querying back to key value stores. It turns out that SQL is crucial to non-app-developers such as business analysts, data scientists. Trying to setup an ETL to pull data from your MongoDB datastore to a Postgres DB so the analysts could generate reports is such a waste of time and resources. Or worse, the analysts have to request devs to write Javascript code to query the data they need. For this reason alone, I will always start with a DB that supports SQL out of the box.

deleted 4y ago
[deleted]
ralusek 4y ago
People didn't leave RDBMS for NoSQL because they wanted to get rid of RDBMS features, they did it because they couldn't scale their applications on single instances anymore. So they had to give up transactions and foreign keys and schemas, etc, because those concepts just didn't exist across instances. Not because they wanted to, but because they had to. The entire database world has been working ever since trying to get back those RDBMS features into their horizontally scalable solutions, it's just taken a while because the problems are exponentially harder in a network. It's not like people are using eventual consistency and a loss of referential integrity because they prefer it.
- senttoschool 4y ago
  >People didn't leave RDBMS for NoSQL because they wanted to get rid of RDBMS features, they did it because they couldn't scale their applications on single instances anymore. I agree that this should have been the reason for choosing something like MongoDB over MySQL/Postgres. In reality, I saw a lot about the companies switching/starting with because so their devs didn't have to learn SQL and they can just stay in Javascript/whatever language they were using.
  - Tainnor 4y ago
    > In reality, I saw a lot about the companies switching/starting with because so their devs didn't have to learn SQL and they can just stay in Javascript/whatever language they were using. Or because they wanted their data to be schemaless. Now, a valid reason for wanting data to be schemaless is that you store free form documents, generic cache entries (Redis) or other such things where it really makes no sense to specify the shape of the document/entry, because at any point in time, different entries will have different shapes. Another can be scalability concerns or other reasons for why you consciously choose to have your data fully denormalised. But invalid reasons, IMHO, include: - "It's faster." It is, until you get data corruption and have to manually fix data on prod. - "We don't know the schema yet." That's not "schemaless", because your application will still assume a schema, the schema is now just implicit. If you want to iterate on your schema, you can easily do so using DB migrations; this is actually easier because the DB will help you verify that you migrated your data properly when you changed its schema, as opposed to changing the schema, but still keeping data around that is invalid under the new schema. - "Not all of our data has a fixed schema." This is now easily solvable in e.g. Postgres with JSONB columns, so you only need to fully specify the data that does have a clearly defined shape.
    - kcartlidge 4y ago
      > Or because they wanted their data to be schemaless. I once led a team creating a public-facing site based on its own custom forms/wizards. We had half a dozen internal departments wanting dozens of wildly varying forms defined, and decided we didn't want fixed schemas as future field requirements were unknown. Rather than the corporate C#/SQL Server we went for Node/MongoDB, and we (successfully) ramped up a new system that transformed the power and cost-base of our public sector employer. We were well chuffed, and it was deemed revolutionary (it also did cloud integration whilst pulling submissions, parsing/verifying, communicating, managing, and pushing into legacy IT systems, so not hyperbole). And yet, despite the outcome, privately the team regretted the choice from about half-way through. I think for me the clue was when we introduced Mongoose to ... enforce a schema onto our schema-less Mongo collections and help out with the relational querying. (Edit: I'm not saying schema-less is bad, I'm saying you need to be clear about when it's good.)
nprateem 4y ago
100%. You build something cool and want to get investment or understand how your marketing efforts are going so need to run some analytics. As soon as even the most basic queries require custom code it's obvious you've wasted a load of dev and analyst time.
machiaweliczny 4y ago
We seem to be happy using Hevo + BigQuery + Metabase for analytics need. Hevo can pull from many sources.

deleted 4y ago

[deleted]

ralusek 4y ago

People didn't leave RDBMS for NoSQL because they wanted to get rid of RDBMS features, they did it because they couldn't scale their applications on single instances anymore. So they had to give up transactions and foreign keys and schemas, etc, because those concepts just didn't exist across instances. Not because they wanted to, but because they had to. The entire database world has been working ever since trying to get back those RDBMS features into their horizontally scalable solutions, it's just taken a while because the problems are exponentially harder in a network. It's not like people are using eventual consistency and a loss of referential integrity because they prefer it.

senttoschool 4y ago
>People didn't leave RDBMS for NoSQL because they wanted to get rid of RDBMS features, they did it because they couldn't scale their applications on single instances anymore. I agree that this should have been the reason for choosing something like MongoDB over MySQL/Postgres. In reality, I saw a lot about the companies switching/starting with because so their devs didn't have to learn SQL and they can just stay in Javascript/whatever language they were using.
- Tainnor 4y ago
  > In reality, I saw a lot about the companies switching/starting with because so their devs didn't have to learn SQL and they can just stay in Javascript/whatever language they were using. Or because they wanted their data to be schemaless. Now, a valid reason for wanting data to be schemaless is that you store free form documents, generic cache entries (Redis) or other such things where it really makes no sense to specify the shape of the document/entry, because at any point in time, different entries will have different shapes. Another can be scalability concerns or other reasons for why you consciously choose to have your data fully denormalised. But invalid reasons, IMHO, include: - "It's faster." It is, until you get data corruption and have to manually fix data on prod. - "We don't know the schema yet." That's not "schemaless", because your application will still assume a schema, the schema is now just implicit. If you want to iterate on your schema, you can easily do so using DB migrations; this is actually easier because the DB will help you verify that you migrated your data properly when you changed its schema, as opposed to changing the schema, but still keeping data around that is invalid under the new schema. - "Not all of our data has a fixed schema." This is now easily solvable in e.g. Postgres with JSONB columns, so you only need to fully specify the data that does have a clearly defined shape.
  - kcartlidge 4y ago
    > Or because they wanted their data to be schemaless. I once led a team creating a public-facing site based on its own custom forms/wizards. We had half a dozen internal departments wanting dozens of wildly varying forms defined, and decided we didn't want fixed schemas as future field requirements were unknown. Rather than the corporate C#/SQL Server we went for Node/MongoDB, and we (successfully) ramped up a new system that transformed the power and cost-base of our public sector employer. We were well chuffed, and it was deemed revolutionary (it also did cloud integration whilst pulling submissions, parsing/verifying, communicating, managing, and pushing into legacy IT systems, so not hyperbole). And yet, despite the outcome, privately the team regretted the choice from about half-way through. I think for me the clue was when we introduced Mongoose to ... enforce a schema onto our schema-less Mongo collections and help out with the relational querying. (Edit: I'm not saying schema-less is bad, I'm saying you need to be clear about when it's good.)

nprateem 4y ago

100%. You build something cool and want to get investment or understand how your marketing efforts are going so need to run some analytics. As soon as even the most basic queries require custom code it's obvious you've wasted a load of dev and analyst time.

machiaweliczny 4y ago

We seem to be happy using Hevo + BigQuery + Metabase for analytics need. Hevo can pull from many sources.