11 ms·
A Postgres Perspective on MongoDB
- bojo 10y ago> MongoDB is basically a native JavaScript filesystem That is probably the most apt description of MongoDB I've ever read.
- threeseed 10y agoFilesystems don't intrisincally support searching the contents of the files to satisfy predicates. And that is the most important part of a database. I think I understand what the author means by the statement. But technically it doesn't make much sense.
- raverbashing 10y agoYeah, technically you could bring grep to your fs instead of having it as a userspace tool, but 90% of the work is on the fs
- bonesmoses 10y agoIt may or may not be apt, but it just felt like I was dumping JSON objects wherever, with a loosely applied path of db.collection to keep things together.
- pbreit 10y agoIs NoSQL really all that much different than relational? At the end of the day isn't a collection pretty much like a table? And you have to tie two collections together with a common ID (ie, join)? NoSQL is less rigid but your code pretty much has to be as rigid, right?
- lastofus 10y agoThis is often what happens when Mongo is used, except now the task of joining and aggregating is pushed up to the application, instead of getting it for [mostly] free from the DB itself. In the process of pushing it up to the app, you lose ACID guarantees. Without ACID guarantees, your ability to view/update a snapshot in time of all your data becomes a very difficult problem to solve. As such, it's hard to say the two methodologies are equivalent. When you start putting relational data into a non-relational DB, you are going to have a bad time.
- eyelidlessness 10y agoIn addition to losing ACID guarantees, you lose single-query joins. They're performed in memory, sequentially, with all the incurred costs of a waterfall series of requests and whatever network communication is involved.
- eyelidlessness 10y ago(Much like the default behavior of some ORMs I work with...)
- toasterlovin 10y agoDatabase views have been incredibly useful for working around limitations of the ORM I work with on a daily basis.
- zihotki 10y agoTo add to that, MongoDB has a plenty of issues with consistency and documentation has some statements which are not proven and even disproven - https://aphyr.com/posts/322-jepsen-mongodb-stale-reads https://aphyr.com/posts/322-jepsen-mongodb-stale-reads
- asimuvPR 10y agoMy experience says that the issue is not differences, but bad architectural decisions. Most applications out there are glorified CRUDs. They are perfectly serviced by SQL databases. NoSQL fits a smaller set of systems whose scope is mostly passing data around. PostgreSQL now supports JSON as a datatype and allows querying it. Works beautifully. No need for MongoDb at all.
- chucky_z 10y agoI will argue _against_ MongoDB all day, but there is most certainly a time and place for it. If you have a real firehose of data, don't care too much about the accurateness, and really need good speed... MongoDB is not an awful choice.
- eyelidlessness 10y agoHow well does it stand up against other "NoSQL" solutions that compete in the same space? My impression has been that it tends compete poorly with a number of other solutions for data integrity, and my intuition is that its appeal is correlated to the proliferation of JavaScript and the convenience of interoperating with it.
- chucky_z 10y agoFrom my experience, the problem is not how does any NoSQL solution stand up vs. any other NoSQL solution. The question to ask is "What caveats affect me the least?" and then go with that NoSQL solution. In the case of MongoDB, the biggest issue (at least in the past) is that the caveats simply aren't known well, or at all! Compare this to something like Elasticsearch, which is well-known (and well-documented) to have data loss in certain situations.
- omginternets 10y ago>MongoDB is not an awful choice Frankly I think it's still an awful choice given it's broken replication model.
- Illniyar 10y agoWell, there are a few other massive differences - RDBMS have basically its own version control built in (which is what allows ACID) and by containing the logic for joins it allows performance based on statistics (which can't be done in an upper layer)
- eyelidlessness 10y agoWouldn't it be great if those facilities for version control provided something akin to event logging as a first-class API out of the box?
- Illniyar 10y agoWell oracle has an eventlog (which you use to synchornize databases). I'm guessing that other databases have similar implementations. Though no API to hook into it, as far as I know.
- pritambaral 10y agoPostgres has logical replication. I'd guess MySQL and MariaDB also have something of this sort, because Galera uses that.
- ibotty 10y agoWell, NoSQL usually does not have much to do with relational algebra. That means, it usually is _very_ different.
- tremon 10y agoThe entire premise of NoSQL is based on two (perceived) shortcomings of SQL databases: its rigid structure limits flexibility and its transactional safety limits speed and scale. So it really depends on what you mean. No, NoSQL isn't that different from regular SQL in that it still provides an object-based data storage and retrieval facility. Yes, NoSQL is different from regular SQL because it does away with the most fundamental properties of SQL. isn't a collection pretty much like a table No, it isn't. A table (or relation) has predefined columns, and each row adheres to the data type and constraints for each column. If you were to model a NoSQL "collection" as a table, you'd end up with a sparse matrix where the number of columns could be equal to number_of_rows * number_of_distinct_attributes of the original collection. you have to tie two collections together with a common ID Nothing in SQL requires you to join collections based on a common ID (though it is the most useful). But still, an SQL join has a predefined result structure regardless of the actual table contents or join condition. There is no predicting the data structure of a NoSQL join.
- xorcist 10y agoIs a relational database really all that much different from a file system? At the end of the day isn't a directory pretty much like a table? And you tie data together with their common id, or file name? I think there is a difference, especially in practice. Relations are a first class citizen in relational databases. Integrity is enforced. Most types of operations can be bundled up in transactions. That abstraction differs from non relational data stores and you have to design your application to fit it.
- Corrado 10y agoWell, the predecessor to DB2, IMS, is basically a database in a file system. Sure, IMS is hierarchical and not relational but it is a full on database with fields and indexs. In fact, Mongo (and other NoSQL solutions) seem to have a lot in common with IMS including the lack of a proper schema and referential integrity.
- Kiro 10y agoI'm building a Node.js app where I was thinking of using MongoDB but I'm reconsidering it after reading the comments here. The database's only job is to keep persistence between server restarts. On server start I fetch the a collection from the database and populate a variable with it, which I use for any direct access. When a user updates something it will update the variable as well as updating the collection in the database. Something like this: var foo = readFromDB(); socket.on('connection', function() { this.emit(foo); }); socket.on('update', function(data) { foo[data.id] = data.val; updateDB(data.id, data.val); }); (In my real application each collection has sub-arrays etc and not just key-value like in the example.) What should I use instead of MongoDB for this? Or is my approach fundamentally flawed?
- Fannon 10y agoBasically any database should do for this usecase. A simple key-value store probably best. I'd aim for keeping the technological complexity as low as possible. What is the simplest tool to do the job? Or maybe better yet: Use something that's already (or will be), in your technology stack to avoid having one more thing in it.
- Kiro 10y agoThanks! > What is the simplest tool to do the job? For me it's definitely MongoDB but I'm afraid of data loss etc. My example was simplified. In reality I want to update a sub-array within the collection, like this: collection['some_identifier']['items'][3] = 123. Overwriting the whole 'some_identifier' document seems wasteful. With MongoDB it's easy though to update nested sub-data.
- elmigranto 10y agoIf you ever only need to read or update single document in a single collection, Mongo's fine. When you need anything more (which is probably "soon" if not "already"), it'll bite one way or the other.
- twblalock 10y agoI think the NoSQL trend is something similar to what is going on with programming language trends (in industry, not in academia). Many commonly used languages were strictly typed and static, such as C, C++, and Java. Then, dynamically typed languages became popular because they are easy to use, although they give up a lot of the safety and static analysis capability of the older languages. Let's call that the second phase, and it is exemplified by Ruby and Python. Now, newer languages with type inference are coming out that combine the safety of the older languages with the ease of use of the dynamic languages -- Scala, Go, etc. Let's call that the third phase. In the database world, we are still in the second phase, which is exemplified by databases such as Mongo, which have started from scratch and thrown out many of the good ideas embodied in older databases. I suppose that the new JSON capabilities of Postgres are an attempt to move database design into the third phase, which combines the ease of use of NoSQL with the many superior qualities of traditional relational databases.
- mtrycz 10y agoThere's also additional considerations that weren't touched in the article, and they're not functional but architectural. Sometimes you have payloads that can't fit in a traditional RDBMS, and SQL/Relational don't cluster/shard well. NoSQL bring convenient clustering/sharding to your data. Anecdote: an old team chose Cassandra solely for its admin-tools (which are really cool), even if we didn't actually need a NoSQL; I lost that battle, but I least we all learned something.
- BurningFrog 10y agoMy pet theory is that the rise of automated testing was/is a big factor in the rise of dynamic languages. Test suites guard against most things type safety does, so if you have a test suite anyway, the type safety becomes a net burden.
- matt4077 10y agoI don't remember much automated testing being done in those early days of PHP, and I don't see how your theory would explain the current trend back towards typed languages.
- graffitici 10y agoI wish the OP had also drawn a comparison between Postgres and Cassandra (which he mentions early on). Based on the schema he writes at the beginning, it looks as though he works mostly with time-series. My understanding is that Cassandra is particularly well-suited for these types of workloads (definitely more so than MongoDB?). If the write throughput is such that a single server can handle it, how would Postgres compare with Cassandra? What are the distinctions in read-latencies? Storage space? Perhaps for the next article..?
- zihotki 10y agoSo true, I often see people choose a database without considering pros and cons, they often considering only what they want as a developer to work with or to try and don't consider the nature of the data. Time series data fits the best in Cassandra and some other column-oriented DBs but definitely not in MongoDB.
- threeseed 10y agoTime series data is actually a great fit for MongoDB. You create a collection for each measurement, JSON document for each day and then preallocate the fields for each hour/minute/second. With batching you can update 100K documents a second pretty easily off one node. For example: https://blog.serverdensity.com/using-mongodb-as-a-time-series-database/ https://blog.serverdensity.com/using-mongodb-as-a-time-serie...
- eddieroger 10y ago
- petetnt 10y ago> While it’s in our best interests to keep similar documents restricted to certain collections, there’s no kind of constraint that enforces this. However, if you need such a thing, there's always things such as Mongoose[1] that can do this trivially for you. [1] https://github.com/Automattic/mongoose https://github.com/Automattic/mongoose
- elmigranto 10y agoBut you still can insert non-consitent data into the database with CLI, or otherwise. And `default` values defined inside Mongoose is no good when filtering or searching documents. There are plenty more problems with emulating schema in app code (Mongoose or anything else), so it's not that trivial.
- raverbashing 10y agoGood to see an article with a fair comparison and not just "anything is worse than PostgreSQL" mentality
- elmigranto 10y agoArticle is more an introduction of Mongo to Postgres users, rather than comparison. And yeah, anything's better than Mongo :)
- raverbashing 10y ago> And yeah, anything's better than Mongo :) Yeah I guess keeping to their comfort zone while keeping a sense of superiority is easier :)
- elmigranto 10y agoI started with Mongo, used it for little bit over 2 years. Then new project used Postgres (2 more years). After that I got a chance to work with Couch. Whenever I encounter Mongo nowadays, it's either simulating multi table joins on app level, or inventing locking with Redis. Part of the issue is people storing relations in NoSQL just because, but I can't think of any use cases where Mongo wins over other NoSQL solutions. Perhaps you have some?
- raverbashing 10y ago> but I can't think of any use cases where Mongo wins over other NoSQL solutions For small deployments, and when you don't have to worry about (many) relations between tables it should be ok to use Mongo Maybe Cassandra is better in some cases, or PostgreSQL with JSON And of course Redis in a lot of different situations I don't know what you mean by "inventing locking with Redis" but it doesn't seem like anything I would advise doing either with Mongo or Redis
- lloyd-christmas 10y agoImage storage * When working on any type of image-based data science, it's incredibly easy to manage, it's fast, and meta data is incredibly convenient. The drivers are pretty consistent and readable across languages. Assuming that you know javascript, it's a pretty quick and easy set up to add a light weight node server with an image cropper on the front end, then set up a python app to do your data processing once you're done munging. I set that up any time there is an image-related competition on kaggle or setting up HAL-like robots on my raspberry pi to mess with my roomates: "I'm sorry Dave, I can't let you cook that". * all you "OMGZORS USE A FILE SYSTEM!" can keep your mouth shut. This is drastically faster and much more manageable for the use-case.
- eljulio 10y agoOne missing point Is the scalability. Postgres is awesome until you need to setup a db cluster.
- scarlac 10y agoAgreed. Scaling out databases is never something you want to do, particularly for Postgres. But it's something you're less likely to need if you structure your data, in an RDBMS. If you know your SQL, then optimizations should be preferred before scaling out. Not just because you need to, but because it can improve user experience (server response latency).
- collyw 10y agoIs it actually any better in Mongo? I know their marketing says it is, but from what I have read the reality isn't so.
- code_research 10y agowould you like to describe the problems you had with the existing solutions?
- bonesmoses 10y agoWell, we ran a billion queries per day on a 1.4TB system I was in charge of a couple years ago. Now we're building a 500+TB reporting system and probably putting it on Postgres-XL. Seems to scale fine to me. ;)
- IshKebab 10y agoWhy does it take 110ms for Postgres to count the rows in a table? That seems like an insanely long time.
- JimmyAustin 10y agoPostgres can require a full table scan in order to count the number of rows. See: https://wiki.postgresql.org/wiki/Slow_Counting https://wiki.postgresql.org/wiki/Slow_Counting
- cpkpad 10y agoI would guess two issues: 1. In many cases, there is a ~30ms floor on operations if a server is non-local. That's true of virtually all services on AWS -- whether SQS, SNS, SES, RDS (MySQL/PostgreSQL), etc. 2. In many cases, rows are missing. A table has a million rows. Ten are deleted. It's not compacted. So it's not a simple size lookup. A lot of this depends on how the data is stored/indexed. That's something which is naively pretty slow, but easily optimized.
- bonesmoses 10y agoIn Postgres, because it uses MVCC, every count requires verifying the table row in the data segment. This means a sequential scan over the contents of the table to verify the rows exist and are visible to the current session. Few operations have this particular limitation, and it's been a sticking point in the Postgres world for years. Unfortunately there's no easy way around it.
- patsplat 10y agoFrom my experience building and deploying production systems with mongodb the primary benefit is not needing a migration or ORM framework. - Deployments get much simpler when migrations for additive changes are unnecessary. Hundreds of deployments a week! - JSON queries easy to pass around and decorate in application code. Hydration from a dictionary is straightforward. Combined, an ORM isn't really necessary.
- mrweasel 10y ago>primary benefit is not needing a migration or ORM framework You don't need an ORM, but they are handy. I'm pretty sure it wouldn't be an issue to load rows from a RDBMS, using a language like Python, as a dictionary, and just decorate the snot out of that dictionary, to make it look like an object. Anyway, regarding migrations, I sort of agree. Migrations can be terrifying and slow things, but at least you're allow to make certain assumptions about your data afterwards. Assumptions like "this field will always be there". With MongoDB, and any other NoSQL database I've used, you need to handle your "migrations" in the application code. So now you have block of code that checks if a given attribute exists in your JSON document. It's not necessarily a problem, but it is something you need to consider. Personally I much prefer RDBMS, because I actually like SQL and the ability to make ad hoc queries easily. If JavaScript is advertised as the query language for a database I'm not terribly interested, unless the pay off is extremely high.
- franzwong 10y agoI tried to scale Postgres few months ago but I found that it is quite tedious.
- forgotpwtomain 10y ago> I tried to scale Postgres few months ago but I found that it is quite tedious. If your application is big enough that you really need to scale postgres, I assure you, you will have a whole lot of tedious problems - not just sharding.