4 ms·

Maybe not Mongo itself but schema-less NoSQL databases are very useful for Data Hubs that integrate many different data sources. If you work as I do on a team t

Maybe not Mongo itself but schema-less NoSQL databases are very useful for Data Hubs that integrate many different data sources. If you work as I do on a team that has 100+ members, trying to integrate 25 billion documents with 30+ datasources and 60+ APIs, then a schema-less database where you can more flexibly handle changes to the document model (which are happening every day) is ideal. Trying to build that in MySQL would be a disaster.

arcticfox 8y ago
Right, but you could just build that in Postgres with JSONB
- shoo 8y ago
  So the question might be: when would building it as postgres + jsonb not be appropriate?
- ruiquelhas 8y ago
  MySQL also has JSON support (https://dev.mysql.com/doc/refman/8.0/en/json.html https://dev.mysql.com/doc/refman/8.0/en/json.html) and even a Document Store plugin (https://dev.mysql.com/doc/refman/8.0/en/document-store.html https://dev.mysql.com/doc/refman/8.0/en/document-store.html).
overcast 8y ago
The vast majority of projects are basic CRUD operations, using relational data. Not something you should ever do in a NoSQL database. That's the problem, people were using it for EVERYTHING. Your use case is very niche.
sixdimensional 8y ago
I like using tools like Apache Drill for the schemaless use case. I apply that in use cases where you’re dropping files with maybe similar schemes into a folder and doing exploratory analysis on the way to a structured solution, or for quick throwaways, or even for data integration ELT type solutions, as you mentioned. This use case works well with data vault-like architecture (or “data lakes”). Actually, anything that can read and perform a SQL query against a standards based file in a file system (e.g. CSV, JSON, Parquet, etc. on HDFS or regular FS). Apache Hive, PrestoDB, Spark, or even some RDBMS which support Foreign Data Wrappers can do this, although some of those require a schema to be specified for querying even if the underlying files can be of different schemas. Drill doesn’t have that limitation and can literally query without metadata. Schema / data types matter so true schemaless can be difficult unless you manually map data types, but it has a place.

arcticfox 8y ago

Right, but you could just build that in Postgres with JSONB

shoo 8y ago
So the question might be: when would building it as postgres + jsonb not be appropriate?
ruiquelhas 8y ago
MySQL also has JSON support (https://dev.mysql.com/doc/refman/8.0/en/json.html https://dev.mysql.com/doc/refman/8.0/en/json.html) and even a Document Store plugin (https://dev.mysql.com/doc/refman/8.0/en/document-store.html https://dev.mysql.com/doc/refman/8.0/en/document-store.html).

overcast 8y ago

The vast majority of projects are basic CRUD operations, using relational data. Not something you should ever do in a NoSQL database. That's the problem, people were using it for EVERYTHING. Your use case is very niche.

sixdimensional 8y ago

I like using tools like Apache Drill for the schemaless use case. I apply that in use cases where you’re dropping files with maybe similar schemes into a folder and doing exploratory analysis on the way to a structured solution, or for quick throwaways, or even for data integration ELT type solutions, as you mentioned. This use case works well with data vault-like architecture (or “data lakes”). Actually, anything that can read and perform a SQL query against a standards based file in a file system (e.g. CSV, JSON, Parquet, etc. on HDFS or regular FS). Apache Hive, PrestoDB, Spark, or even some RDBMS which support Foreign Data Wrappers can do this, although some of those require a schema to be specified for querying even if the underlying files can be of different schemas. Drill doesn’t have that limitation and can literally query without metadata. Schema / data types matter so true schemaless can be difficult unless you manually map data types, but it has a place.