12 ms·
Goodbye MongoDB, Hello PostgreSQL (2015)
- anticristi 6y agoI used both NoSQL and SQL databases. Proper replication topologies (e.g. multi-master configurations) and JSON columns killed the NoSQL movement. I'm glad the NoSQL movement existed and forced the SQL camp into out-of-the-box thinking, but I wouldn't go for any NoSQL solution today on a greenfield project.
- menzoic 6y agoWhat if you didn't need relational queries and was dealing with a large volume of data that needed to be sharded?
- sascha_sl 6y agoIf you just need a key-value store, a database like MongoDB is rather oversized. Redis or and a bunch of object storage system (now that they actually include locking) are much easier to operate. As soon as you're back to needing complex queries (with or without relations, Postgres it is again.
- petersellers 6y agoRedis is an in-memory datastore first and foremost (with optional persistence). With that said, it's probably not the right tool if you need to store and persist TB-worth of data even if it is key-value. Something like Cassandra is probably a better fit if you have a lot of non-relational data that needs to be persisted and you need it to scale.
- funcDropShadow 6y agoBut you need more than a few TiB of stored data for Cassandra to be a net win. I've been part of two inhouse software projects with database in the single digit TiB scale based on Cassandra. Both projects where switched to PostgreSQL and the performance improved for almost everything. Plus all the other advantages of mature relational database. Hosting Cassandra is a whole different league of complication compared to PostgreSQL. So you should be very sure that you actually gain something before choosing it over PostgreSQL.
- petersellers 6y agoAbsolutely - pick the right tool for the job. Generally speaking I'd go for PostgreSQL in most cases but there are definitely scenarios where Cassandra would perform better and be more appropriate.
- bildung 6y agoIn what scenarios does one work on data without any relations? Honest question - perhaps I'm too used to think in SQL but even with e.g. a completely flat bunch of documents one usually needs at least access control (i.e. document owners, which is a relation).
- dheera 6y agoWhen you get piles of JSON data from somewhere else (e.g. an API response from a provider you subscribe to) and want to store all of it intact, query it, and that somewhere else may insert new fields at any time in the future without telling you. When you have multiple types of objects with differently populated fields that you want to store in a single collection and query across them in currently unknown ways, and where you need flexibility for the unknown.
- drodgers 6y agoThat sounds like a great use case for OLAP and fuzzy search solutions. Eg. Dump all of the data to S3 and search it in parallel via AWS Athena. If you want fast fuzzy search, then elasticsearch is built for this. If the data is small enough that you want B-tree indexes (with the accompanying slower writes and lower query latencies), then any SQL database with a JSON column will work.
- dheera 6y ago> Eg. Dump all of the data to S3 and search it in parallel via AWS Athena. Too proprietary. Can't be run locally, on robots, etc. Can't be transferred to other cloud providers. Also S3 is horribly slow to write or delete thousands of records at once.
- kornish 6y ago> Too proprietary. Every major cloud provider has a blob store. Most are S3-interface compatible, or can be fronted by something which is, like Minio. > Can't be run locally, on robots, etc. If the contents of your S3 keys are just line-delineated JSON files, you can easily download those files and run scripts or process them locally. > Can't be transferred to other cloud providers. Again, not true — a tool like rclone handles this case in a single command. Depends on the amount of data, but if anything it's easier to move a bunch of flat files between providers than copying a database backup around. You have to pay for egress bandwidth, though, of course. > Also S3 is horribly slow to write or delete thousands of records at once. If you're keeping one JSON record per S3 key, that is a blatant misuse of the tool and performance will be terrible. On the other hand, if you batch records into files of appropriate sizes, it's very cost-effective to store and query via Athena, Spark on EMR, etc. If you need individual record-level access as identifiable by a key, then you'd probably be better off with Redis or Memcached. (though, those will not be as good for bulk offline processing) It's all about your access patterns.
- AmericanChopper 6y agoAs a former Oracle DBA, and general skeptic of most of the use cases the NoSQL ends up getting chosen for, I’ll gladly say that document databases are pretty good at storing documents. I’ve seen them used very well in content management applications, and the document-database-as-a-materialized-view use case can make a lot of sense. Just like graph databases can be very good at storing graphs. As far as the bandwagoning goes though, I think Mongo has taught a lot of people about the strengths of the relational model.
- srich36 6y agoThere are plenty of modern, distributed RDBMSes that make sharding transparent to the user (E.g. cockroachDB, yugabyte, vitess, many cloud offerings, etc.). Most NoSQL databases end up adding transactions because they are important, and thus the scale advantages for NoSQL systems over relational databases are diminishing, if remaining at all.
- bushbaba 6y agoLatency at p99 is substantially better at scale with a key value lookup in nosql over postgresql. Thing is most people don’t have scale these days. You can get a single box with hundreds of logical cores and many hundreds of TiB of locally attached ssd. Until you exceed that you don’t necessarily have scale.
- ZephyrBlu 6y agoDistributed databases like cockroachDB are KV stores.
- collyw 6y agoMySQL had a Berkley DB engine underneath, that's a KV store too. I wouldn't say that makes /made MySQL a key value store.
- threeseed 6y ago> You can get a single box with hundreds of logical cores and many hundreds of TiB of locally attached ssd And then that box falls over because the entire region fails like just happened yesterday with OVH. Or it just randomly fails like has happened to me with AWS dozens of times. Vertically scaling a database on your own cloud instances is amateur hour. Either use a cloud-managed database or one that is highly available.
- bushbaba 6y agoSure. And those services support some fairly beefy instance types. Such as https://aws.amazon.com/about-aws/whats-new/2019/11/amazon-rds-for-sql-server-now-supports-x1-and-x1e-instance-types/ https://aws.amazon.com/about-aws/whats-new/2019/11/amazon-rd...
- anticristi 6y agoTo me, "warm" (not hot) data falls into one of two boxes: - schema on write: You know what future queries you will ask. Use SQL. - schema on read: You don't know what future queries you will ask. Use object storage. Of course, in practice, part of your data will be schema on write and part on read.
- aidos 6y agoCan you explain this more? I’d argue that if you don’t know the patterns you’ll use to access your data, relational is the way to go. That’s the point, no? Flexibility over how you access your data. With an object store you mostly have to have decided upfront which data is collocated together.
- anticristi 6y agoMaybe I explained it badly. :) SQL essentially forces you to first define your data schema, then write data according to that schema. This works great for reading a user profile given an email; or read all comments of all users who are born before 1995. Since you have a schema on write, you can add indexes to optimise these queries. In contrast, if you need to do "anomaly detection" over an opaque dataset (where the definition of "anomaly" moves faster than you can rewrite your schemas), then it's better to just do a full dataset scan and create a schema on read. My 2 cents.
- aidos 6y agoStill not sure I buy it. I can extract a document representation from my relational data that matches your document store at read time. I can effectively choose my schema when I read, there’s no need to rewrite it at the storage layer.
- rwoerz 6y agoMaybe I do not get your point but I have a complete opposite opinion. If you do not know about your clients' queries, you are better off with SQL since it enables clients to query exactly the data they need. If you know about future queries, you can shape your object or document storage or db beforehand to match these queries.
- amenod 6y agoI had a project where I just needed to be saving a constant flow of data (id + 100 bytes records) somewhere, no persistence needed, no sharding (single machine) - but lots of it. I first turned to Redis and it worked nicely, but since I had PostgreSQL running, I made a performance test with it too. To my surprise, once I optimized both (memory only, pg WAL disabled, persistent connections,...) there was almost no difference in insert rate, so I ended up using PgSQL only. This is anecdotal, of course, and it is possible that I simply didn't optimize Redis enough (though I did give it my best!). Let me also say that Redis is great otherwise, it was just in this specific case that PostgreSQL surprised me and fulfilled the need itself.
- HelloNurse 6y agoYour data-saving test was probably limited by how fast you could save to disk a roughly equal amount of data with the two databases; queries might work and perform more differently.
- amenod 6y agoThat's what I meant by "no persistence needed". In both cases, I wasn't actually saving anything to disk. I needed an in-memory storage for a never-ending stream of incoming data that would be read (and purged) by a separate process, no biggie if the data got lost occasionally. I also simply inserted and read the data, no complex queries or anything. Key-value store is all I needed and for this particular case PostgreSQL was as good at it as Redis.
- kinos 6y agoWhy did you use a database instead of just creating a global map and running in memory? Sharding?
- amenod 6y agoNo - different processes were reading and writing. And once you implement a global map which is accessible from multiple processes, you might as well use an existing solution, like Redis. Or, as I've learned, pgSQL - which gave me the same performance with less moving parts (because I was already using it) when I disabled WAL.
- YZF 6y agoIs there a proper replication topology that guarantees all writes survive single node failures e.g. in a 3 node cluster and and reads also as expected? i.e. any sequence of reads and writes execute correctly in any single node failure scenario?
- solidasparagus 6y agoI'm less sure about this. One advantage of NoSQL is that the serverless offerings blow away any SQL DB-as-a-service offering that I've seen. I think that's because NoSQL databases give you minimal, but easy-to-reason-about scaling properties whereas SQL databases do more for you, but at the cost of greatly increased complexity. I think pretty highly of AWS Aurora Serverless and the UX is still garbage in comparison to S3 or DynamoDB. Now if you start trying to add transactions to a key-value store based architecture, NoSQL gets pretty ugly, but that's misusing the tool IMO. Most of my projects still rely on NoSQL DBs (usually S3) because they have really excellent UX and I have plenty of data where I don't need relational properties.
- remus 6y agoFrankly it sounds like you just don't need a relational database. Their whole selling point is the strong guarantees they provide about your data, and if you aren't using those then you're basically just writing files to disk in a very convoluted way.
- solidasparagus 6y agoYou're absolutely right, I don't need a relational database. There are many use cases that don't need relations and avoiding the complexity introduced by RDBMS features I don't need is where NoSQL shines.
- valzam 6y agoI think when most people say NoSQL they don't mean S3 (even though it is like a database). In general I think - S3 - Redis/Memcached - Elasticsearch aren't really NoSQL databases in the Marketing sense since they fulfil such a specific purpose.
- schoolornot 6y agoPostgres doesn't have multi-master. That's the problem.
- collyw 6y agoI am stuck maintaining a mongo monstrosity.
- psychometry 6y agoNo SQL database has the anywhere near the search capabilities of Elasticsearch/Lucene. I also wouldn't choose Postgres for storing logs, either, but I would use a document store (i.e. ELK stack).
- citrin_ru 6y agoClickHouse suits well for string logs, especially if a log message have some fixed structure which can be mapped to separate columns. And most logs have some number of predefined fields which are easy to map to columns (e. g. timestamp, IP, request time, e. t. c.). You can store free-form JSON which doesn't have per-defined schema in a string column, but ELK would be probably better in this case. And then you can query logs using all power of SQL.
- fhunt 6y ago@dang, I found this in the guidelines: Please don't post shallow dismissals, especially of other people's work. I see good critical comments here but also many shallow dismissals from people who just sneer at the work of all software engineers who created MongoDB. Why is such behavior tolerated in any Mongo thread and why do these people never face any consequences?
- swebs 6y agoThat doesn't work here. You have to send an email to hn@ycombinator.com
- neogodless 6y ago> Why is such behavior tolerated in any Mongo thread and why do these people never face any consequences? I think you answered your own question: "guidelines." Those that believe in this community, and want adherence to the guidelines will vote on submissions and comments so as to reflect that belief. Shallow dismissals tend to get voted down, and they may be refuted if someone is inspired to put the effort in to explain why such a shallow dismissal is either unwelcome or missing important information. Seeing as these are guidelines, there is no swift and absolute retribution for light infractions, when the community can largely self-correct such things.
- _ozde 6y agoThis is from 2015. I wonder if it is the same in 2021. Anyone running high throughput/low latency services using MongoDB?
- menzoic 6y agoMongoDB and Postgres have very different strengths and weaknesses. It sounds like they didn't consider the needs of their read and write patterns before they decided to go with MongoDB and then blamed it for not being a good fit for their use case. The article doesn't give any insight at all into the cause of the performance issues, just that they happened. It could've been that they were doing something inefficiently. They also don't give any reasons for the performance increase when switching to Postgres. It could've been anything like they were doing application layer joins with Mongo and things got faster when switching to a relational database because they could do joins in one process without serially fetching data over the network. They also touch on schema on read vs schema on write but instead of talking about the tradeoffs and use cases of both patterns, they just claim schema on read is bad.
- chris_wot 6y agoThe article doesn't give any insight into the cause of the performance issue because they coudn't work out what that cause was: In another instance we noticed degraded performance of our applications and managed to trace it to our MongoDB cluster. However, upon further inspection we were unable to find the actual cause of the problem. No matter what metrics we installed, tools we used or commands we ran we couldn’t find the cause.
- menzoic 6y agoTrue, but it seems like they blamed it on MongoDB just being bad without justification and proposed Postgres as the solution.
- chris_wot 6y agoNo, it seems like the diagnostics tools they were provided didn't help them get to the bottom of the very bad performance issue they were actually having with MongoDB, and then they proposed Postgres as the solution. As most companies do, eventually.
- 6y ago
- anonytrary 6y agoI learned a framework several years ago and its default db was MongoDB. So I worked with MongoDB for a few years. Then at my job I used PostgreSQL for a several months. I will probably never use MongoDB again.
- ekvilibrist 6y agoI'm sorry you had to go through that.
- pietrovismara 6y agoOh please, quit the bullshit elitism. I think postgres is a way more robust solution than mongo, but I've had my worst db experiences having to deal with a postgres db with almost all the business logic in triggers. That was a huge pain in the ass to maintain and understand. Mongo is defnitely a limited solution, but for certain cases it just works pretty well, with less complexity than postgres. At one of my jobs, I've setupped and maintained a Mongo cluster that did its job without causing any fuss for years.
- urthor 6y agoYou suffered because of idiot co-workers. Let me tell you, your idiot co-workers can also do some evil things in Mongo pipelines. Very, very evil things.
- voiper1 6y agoPrevious discussion from 2015: https://news.ycombinator.com/item?id=9178773 https://news.ycombinator.com/item?id=9178773
- Twisell 6y agoThe top comments of this old thread are retrospectively hilarious because sone frontend guys argues about wether SQL is not easily composable and exchanges trick about how you can securely and dynamically concatenate SQL statements. But nowhere to be found is the concept of prepared statements that would gracefully help then write nicer code. https://www.postgresql.org/docs/current/sql-prepare.html https://www.postgresql.org/docs/current/sql-prepare.html And that is a lightweight in-code solution for those who are repelled by database procedures which is yet another way to solve theirs issue with SQL.
- amenod 6y agoCorrect me if I'm wrong, but I think you missed the point of the discussion? It is not connected to frontend at all. It talks about the difference between using an ORM versus constructing SQL statements directly, as strings. In that light I don't think `PREPARE` and stored procedures would help much (or at all). I also don't find them hilarious, though they do represent very extreme views on the subject (imho) while to me it is essentially a tradeoff (I use both direct SQL and ORMs, depending on the needs of the project).
- Twisell 6y agoMy point is precisely that they compare two subpar approach of SQL while prepared or stored procedure are the real contender here. And as far as I understand depending on your use case prepared or stored is better. Prepared mean less code base inside the db and still more composability with less sql injection risks. Stored add even more composability and security but at the cost of managing code inside the db.
- aidos 6y agoI was involved in those original discussions and it was just the standard “how to manipulate sql” discussion. Prepared statements don’t help you here, right? My understanding is that they’re purely for performance as the parser doesn’t need to be rerun each time. You need to recreate them for every dB session.
- soulchild37 6y agoMongoDB is like snapchat for database
- isubasinghe 6y agothis was funny, but I think its a lot better now. Imo rolling their own consensus protocol was a terrible idea, but they are using Raft now I believe.
- threeseed 6y agoWould've been funnier and timely about a decade ago.
- dijit 6y agoMake a joke similar to this a decade ago (as I did) and developers would jump on you for being "old" or "not getting it". So, not really. But I agree that this joke is an old one and doesn't really have a place in a constructive discussion on hackernews.
- twic 6y agoMongoDB would have been timely about a decade ago.
- menzoic 6y agoI'm starting to realize all the people complaining about data loss in Mongodb just didn't understand the tradeoffs between weak consistency and strong consistency.
- qaq 6y agoWhat surprises me is that so far none shoehorned Postgres on top of FoundationDB.
- whitepoplar 6y agoI'm not familiar with FoundationDB. What benefit would that give Postgres?
- qaq 6y agoIt would become distributed DB on top of super solid foundation that powers snowflake, icloud etc.
- pedrogpimenta 6y agoI'm a designer and front-end developer. I'm building a fullstack project by myself in Node and I'm using MongoDB. Can someone tell me why is it so bad? I've searched for concrete answers but I got none for the moment yet from time to time people say these kind of things and I worry.
- Doxin 6y agoThese days MongoDB is mostly a subset of what SQL servers do. If all you need is a document store you can simply make a table with a JSONB column and get that. If you later discover you do need relations you can do that with e.g. Postgres but not with MongoDB. Historically MongoDB has been known for dropping or mangling data in some unfortunate ways, I'm not entirely sure what the story there is these days but personally I wouldn't trust it ever again. Data integrity is too important.
- YouthfulMarxism 6y agoA single point for Mongo over Postgre JsonB is WiredTiger is very good at compression. In times of Mongo:4.0 vs Postgre:10.5 it was like 20Gb vs 50Gb. Though could depend on documents structure. Not sure about current state. Querries were also somewhat faster in Mongo.
- Doxin 6y ago> A single point for Mongo over Postgre JsonB is WiredTiger is very good at compression Granted, the compression story in postgres isn't currently amazing. > Querries were also somewhat faster in Mongo. If you really want fast queries it's not super hard to tune postgres to be fast and drop data if it crashes. It's a bit trickier to tune it to be quite that fast if you want to keep all your data, but for a single table of json documents it should be doable if you're okay with not having transactions.
- amenod 6y agoIt's not that bad. I have used both and while I wouldn't choose MongoDB anymore, it did its job as needed. For me, the main problem with MongoDB is that it is "schemaless" - which doesn't mean that there is no schema, just that it is not explicitly stated and enforced. This leads to problems later when schema changes and you need to be extra careful not to end up with inconsistent data (good luck with that). But this is solveable, it just takes more effort than necessary. If you are using MongoDB for a small-ish project you should be ok. And if your project becomes big, you will need people who know the difference and will be able to migrate (if needed) anyway. So no worries, and good luck with your project! :)
- hasa 6y agoI still feel somewhat that Mongo is blamed for things that it is not supposed to do. Removing millions of records surely is not a usecase that Mongo is comfortable with. Or if you absolutely need a schema to protect developer doing silly things, then traditional relational database is surely way to go. Mongo is superbly quick to serve read-heavy applications. I did one project with Mongo & Sinatra and the app was able to reach high transaction/second rates compared to the hardware stack it was running on. Of course we faced issues to design trustworthy design for very high scaling. I think it is one culprit with Mongo, sharding and things. But anyway, we as developers and architects, just need to have right tools for the task we try to solve!
- smt88 6y ago> Or if you absolutely need a schema to protect developer doing silly things, then traditional relational database is surely way to go. Then you should use a relational database every time, because all developers "do silly things". I've seen this same argument against strong types: I don't need them because I don't make mistakes. You do. So does everyone else.
- StreamBright 6y agoThese articles never get old. Hey we picked solution A and realized that it has some cons not only pros and now we are moving to solution B that only has pros. Few years later there is a new article, hey we are moving to solution A|C because...
- smt88 6y agoI've never seen an article about switching from Postgres to something else. Obviously Citus, Timescale, and other Postgres forks don't count. Uber did one a while ago, but tl;dr they didn't really understand Postgres very well and were dealing with unique problems.
- philliphaydon 6y agoI'm not sure they understood the problem they were trying to solve let alone understood Postgres.
- StreamBright 6y agoI have seen Uber ping-ponging between Postgres and MySQL. My approach to this problem is, can we use Postgres? If the answer is yes (90% of the cases) we use Postgres. If the answer if no investigate what is good out there. For k:v I have used Riak with much satisfaction. For distributed databases we are experimenting with TiDB. It looks very promising so far. There is only one problem with Postgres that I run into. Client connection scalability. This is why many users use a connection proxy in the front of it.
- jspash 6y agoI find them quite useful. Especially if I've never used technology A. After a while you start to see a pattern of people with similar problems to yours moving away from a certain tech. If it weren't for articles like these, we'd all be making the same silly mistakes over and over again. Collective hindsight is a powerful thing and we should use it to our advantage to short-circuit certain problems. If only politics worked that way :/
- kumarvvr 6y ago
- javchz 6y agoI use MongoDB as a front-end that does full-stack prototypes from time to time in Node, and it's great using javascript from start to finish, you don't have to switch mental gears, so creating an MVP it's fast... but. Sometimes I feel like I'm just avoiding SQL for no reason, not using any of the advantages of NoSQL, and only using Mongo as a SQL-wannabe with a shoehorn like mongoose. I wonder how many MongoDB projects out there are just using Mongo because it's the trend (something like MEAN / MERN) and not because makes sense. Like there are out there a lot of SQL ORMs for node that, do the same job and better. Don't get me wrong, I love MongoDB for a lot of weird stuff (like web scraping, CSV files, and archive data), but I feel a lot of the hate, it's because a lot of devs (myself included) have developed projects, that could be working better with a old fashion relational DB.
- deepGem 6y agoWhen you are creating MVPs time matters the most. You are anyway going to throw it away and build something better. I was using Mongo and NodeJS a lot but then switched to Python. Now I find the world of ORMs to be onerous. I just want to define tables in MySQL and just run raw SQLs. The best compromise I have found so far. I mean it does involve boiler plate code but ultimately it's an MVP. And you are right, most of MongoDBs uptick was because of the reasons you state. E2E Javascript and in addition the ability to query nested JSON natively.
- arcturus17 6y agoI also moved away from NoSQL for prototyping; as I recount in the sibling comment, I think the touted increase in dev speed is a myth, as the only penalties in dev speed for SQL come from (a) learning the tech and (b) thinking of your data model ahead of time, both fixed costs which dilute over time and eventually make you faster. > Now I find the world of ORMs to be onerous Funnily enough, I've also moved to Python, Django in particular. However, I'm having no problem whatsoever with the Django ORM and it's serving me extremely well for prototyping - I feel it is very easy to operate and that I can move at the same speed that I ever could with NoSQL after just a couple of months of practice. Have you tried the Django ORM in particular? And what about the ORMs you've tried makes them onerous?
- cies 6y agoJust did some POCs with Hasura, a service on top of PG that provides: * GraphQL interface (I think even a REST one in the latest release) * authorization (role based) * API gateway (put other API behind your GraphQL interface) * a UI for managing the schema * a way to do mirgations Now GraphQL provides a schema (that includes the types of of the values), that can be used to generate clients libs in many languages. This effectively provides me a type safe interface + authorization to my db, while I have not programmed anything yet. Slowly SPAs start to make sense (if there barely is any self written server-side code running).
- diveanon 6y agoI've been using hasura in production for about a year now and am very happy with it. Little recommendation, use their typescript metadata sdk instead of the client ui if you want to be able to build your metadata as part of your ci pipeline.
- fhunt 6y agoWhile Hasura is nice, it's too much blackbox, too much all-in, too much risk, we decided against it. 2021 is great because of TypeScript, types and decorators (just put here any other typed lang with decoraters if you do not like TS). If your db is your leading system for your schema, everything else is derived and "code-generated" with all the restrictions SQL has. Which is not great if you want to decorate and enhance. Yes, you could use interfaces but it's not the same, you can't decorate code-generated type definitions but your architecture wouldn't stay DRY either. If you think TS' types as the leading schema you do not need a type checked db like Postgres anymore, anything which can store data works just fine you have all schema definitions at one place which is not the DB. Good examples enabling such an architecture are TypeORM and TypeGraphQL. However most stick to SQL and 'my schema must be in my DB, there is no other way to enforce a schema' (but there is!) you just need a good architecture (types and decorators!) and not another mainstream-from-the-shelf-solution.
- cies 6y agoInteresting. My db and it's types have whole migration procedures to ensure all works on all environments, Hasura also does this. How would that work with just-TS? The schema changes but the documents in my KV-store (I assume that what you mean by "no types on the storage") have not changed accordingly. How to prevent this obvious trouble?
- mkl95 6y agoEarly in my career, I worked for a company that stored tons of data with Mongo, and their databases looked like poor man's SQL (lots of collections with foreign key relationships between them, etc.). Please don't do this, Mongo doesn't care about your imaginary SQL system actually working.
- distalx 6y agoPrevious threads on HN https://news.ycombinator.com/item?id=9178773 https://news.ycombinator.com/item?id=9178773 https://news.ycombinator.com/item?id=13389196 https://news.ycombinator.com/item?id=13389196
- kumarvvr 6y agoFor most use cases, PostgreSQL will suffice. And I mean, you can solve a lot of problems with beefing the hardware and RAM. For issues beyond that, you can exploit latest sharding features in it. For issues, even beyond that, you can do for Cloud offerings.
- timewarrior 6y agoThere were many questions asking about recent experience with MongoDB. I have some recent experience so wanted to share with everyone. My background: 2007-2010: Used MySQL to run a social network for 50M+ users. Would send 2B messages on peak days. All powered by Mysql. 2015-2020: Used Mongo to power a top 5k site. Zero downtime and dataloss in 5 years. Used Postgres for internal non user facing database So I have familiarity with all of these databases at a decent scale. I am going to list pros and cons of each and why I would use. Again a tool is a tool. Just because I love a wrench, doesn't mean that I am going to use it instead of a hammer when I need to put some nails. MongoDB: They have come a long way since I started using them in 2012. If your use case is CRUD, you would be fine. There is still a lot of marketing fluff about sharding, multi document transactions etc which might not be fully reliable. I stay away from those things. But if you just use CRUD and you data can fit in a single db you are going to be fine at decent scale. Following are some reasons I prefer Mongodb. It's possible that Postgres/MySQL has some of them even though I couldn't find them: 1. I don't like to manage schema migrations. During development you constantly add columns and it's a pain to make sure that this column is added to the sql databases on development, testing, staging, prod etc. With mongo, you just add a column and you are done. 2. No downtime on migrations: You can change you server type or mongo version with zero downtime. Did that since 6 years. Couldn't figure out how to do that with postgres (even with Aurora). 3. If you data structure fits document and subdocument schema, mongo is really easy to use. I tried jsonb with postgres and didn't find it as easy to use at mongo. If you data fits this paradigm, you can essentially get atomicity for multiple updates because they are all in a single document. 4. The admin interface of MongoDB Atlas is really development friendly. You can easily add read only replicas. You can easily add replicas in different regions. I shudder at the thought of managing Postgres myself and even Amazon Aurora is not as easy to admin as Atlas. At this point I would consider Postgres only in the following case: If my data structure is like a non-tree graph. In that case I would expect to do multiple complex join and expect transaction consistency between tables. If I am building a financial product which directly handles money or money instruments (bank, stock trading) I would definitely not use mongo for that. Happy to answer followup questions.
- threeseed 6y ago> Happy to answer followup questions. This thread is like a piñata that gets pulled out every 6 months or so take a wack at MongoDB and talk about how everyone should just use PostgreSQL for every database problem no matter whether it's relational or not. No-one cares about whether MongoDB is useful or not.
- colesantiago 6y agoIs there a situation where I SHOULDN'T use Postgres?
- nathanwallace 6y agoPostgres is incredibly flexible and scales beyond what most projects will ever need. We use it both as a very large scale CMDB store for https://turbot.com https://turbot.com and as an small embedded DB for https://steampipe.io https://steampipe.io. Use cases where we hit road bumps earlier were: 1. High churn data. Postgres copies data for each rite update and then vacuums to clean the old versions. These use cases dominated our DB load and moving them out helped a lot. 2. Large JSON blobs. Postgres automatically compresses and will pull large (~>8kB) columns into a separate “toast” table. This makes sense but also means they are often a real bottleneck if you expect to work with larger JSON blobs as you core model. pg is great for JSON, but if you are using it at scale do pre-work and testing of this factor and how it plays with your model. 3. Of course, long term archive of data objects is much more cost efficient and scalable in an object store like S3. Bet on Postgres. Whatever you use case it will work great until you are large enough to really understand your data and optimize for your business.
- colesantiago 6y agoEven in cases for regional replication? Because this is the use case I am looking for, something like Spanner but cheaper.
- nathanwallace 6y agoCross-region replication of data is always significantly harder, definitely consider if cross-AZ would be sufficient for your case (which it often is). Postgres can definitely be configured for a cross-region scenario, but the approach varies a lot depending on your recovery needs.
- vorticalbox 6y agoMongoDB does have support for schemas, but didn't in 2015 https://docs.mongodb.com/realm/mongodb/document-schemas/ https://docs.mongodb.com/realm/mongodb/document-schemas/
- jb3689 6y ago"I don't know what was wrong" articles like this come up really often with Mongo. My team manages thousands of Mongo nodes and we have issues, but few are directly due to Mongo. Mongo's documentation could be friendlier to the operator/maintainer, however the tools for debugging Mongo tend to be sufficient (metric counters, various status/state dumps, general Linux tooling). You need to understand databases folks. There is no way around it. "Porting to Postgres" does not replace understanding of database internals and figuring out how to debug bad access patterns and explain why they are bad. Blindly migrating data seems like one of the worst things you could ever do - you're not fixing the problem that you don't understand how databases work. I get it - ain't nobody got time for that - but it will come back and bite you with your new shiny fancy tool too if you aren't careful Anyone who brings up something like master-master hopefully realizes they are talking about a completely different problem now with completely different constraints. Expecting the same access pattern will perform just as well in master-master is wishful thinking - it requires careful design to avoid coordination problems. The meat of the story with databases is: coordination is expensive, distributed coordination is even more expensive
- dalu 6y agoWhat the article fails to mention is that you can use as a disk backed in memory database/cluster. In a replicaset you can have e.g. 2 in memory nodes and 3 disk backed nodes. Postgres doesn't have that. I wish it did. Mysql has in memory tables. Postgres doesn't. If I'm wrong please correct me.
- fraud 6y agoI personally really like using Mongo for my side projects. Most of the time my side projects are just little utility things like link shorteners, paste bins etc. and I find it really easy to just make a basic schema and use something like mongoose to do all the queries for me. SQL is nice when i’m doing things that require a relational model, but in node.js I find it takes a lot more setup than mongo.
- jscheel 6y agoIf you find yourself reaching for EAV because you have schema-less data (custom fields, for example), nosql can make your life so much easier, as long as you understand the trade-offs.