8 ms·

Jsonb: Stories about performance

132 points by erthalion 9y ago

larrykwg 9y ago
There is absolutely no good reason for anyone to use MongoDB ever, its just such an amazingly rotten database that its astounding how much hype it generated.
- Goopplesoft 9y ago
  I can easily think of a few good reasons (it is actually a pretty good database for the right use-cases). Can you actually not find a good reason or are you just continuing the exaggerating echo chamber around mongodb's earlier failures for 'points'? Plus this has very little to do with the article which is a very very well done technical analysis of different solutions with very little opining.
  - overcast 9y ago
    I mean, the JSON support in PostgreSQL is pretty freaking awesome now. Beyond the sharding, what are your few good reasons?
    - CodesInChaos 9y ago
      One thing I'd like to see in postgres is support for a superset of json, such as timestamps or binary data inside a json document. Replication and failover don't look so great in postgres either.
      - hiram112 9y ago
        Whenever I store binary data as text, I just base64 encode it and convert on the client.
  - larrykwg 9y ago
    I think you baiting a rant, I count myself among the myriad developers who had to learn it the hard way to never use mongo. My point would be that there are no "right" use-cases, I pointed it out because the article was talking about a specific thing: as the article points out in the outlined scenario: "It’s clear now, that throughput for PostgreSQL and MongoDB now is almost the same before the spinlock performance degradation hits MongoDB." So the article points out there is a significant performance degradation after a certain number of connections in MongoDB, such things won't surprise me anymore I'll add it to the heap of things wrong with MongoDB, thats why I was making the comment. > "We’ve got so far that throughput of PostgreSQL and MongoDB is the same for read workloads. Is it surprising? Not at all - I’m going to show you, that under the hood all our databases use more or less the same data structures and the same approach to store documents." I'd argue that there is no reason to use MongoDB because there are no use-cases other more mature databases won't be equally or better suited for. MongoDB is horrid, you won't know all its deficiencies when you get started with it, but over time you will come to despise it, it is really far away from being a mature database and I would advice anyone against using it for any task, and I'm not the only one who was burned using it and who adopted this attitude towards it.
- wyldfire 9y ago
  I used it once when it first came out just to kick the tires. What a remarkably convenient API it had. NoSQL/"Document store" stuff was relatively new (to me, at least). Did the convenient API contribute to whatever design problems they had? No idea. But if they had problems with durability then it really is all for naught. I didn't get beyond prototyping so I never saw whatever issues caused them to get so much ire.
  - skywhopper 9y ago
    Yeah, agreed. It does have a much better API than other JSON document stores I've tried. For the right use case, it would be fine.
- 013a 9y ago
  *Today. The thing you're forgetting is that Mongo was actually somewhat novel at the time it was released. I've heard at least two stories of engineering teams who felt that their specific product couldn't have scaled the way it did and they wouldn't here today without Mongo, for whatever reason. I've heard the words "it was the best engineering decision we ever made" from one team who's company ended up selling for 8 figures. You're being hyperbolic. Yes, there are so many better choices for databases out there. There are also many worse choices. Mongo is very hard to recommend nowadays, but unless you go in completely ignoring its problems its not like your company is going to go up in flames.
  - user5994461 9y ago
    MongoDB doesn't scale. Write are handled by a single node.
    - gaius 9y ago
      No, it's web scale.
      - user5994461 9y ago
        It's not webscale if there is only one node doing all the work.
    - weddpros 9y ago
      a single node per shard will handle writes. If you have 20 shards, that's 20x more writes than a single node...
  - cookiecaper 9y ago
    > I've heard at least two stories of engineering teams who felt that their specific product couldn't have scaled the way it did and they wouldn't here today without Mongo, for whatever reason. I've heard the words "it was the best engineering decision we ever made" from one team who's company ended up selling for 8 figures. Do you expect to hear them say, "Yes, choosing MongoDB was a bad decision and our production infrastructure is currently on an unwieldy, slow, dangerous piece of crap" and then go on to sell the company for tens of millions of dollars? Do you expect people to admit the real reason they're using MongoDB, which, in 99% of cases, is "we don't know how to use SQL and are really perturbed that anyone thinks we should have to learn"? It is true that MongoDB only spontaneously combusts occasionally, but that doesn't mean the choice is of negligible importance.
- dang 9y ago
  Maybe so, but please don't post unsubstantive comments to HN, or rant here in flamebaity ways. We're looking for thoughtful comments. That doesn't require changing your views at all, just presenting them with more information, and an orientation to good conversation rather than venting.
- SkyPuncher 9y ago
  On the contrary, It's great when you have relatively unformed data. I've head of a lot of logging (i.e. application logging) companies have great success with Mongo because they can store their entire client's set of data without need to know the schema ahead of time.
dominotw 9y ago
>MongoDB has some problems with spinlocks,...Interesting enough is that this situation is getting better between releases (but it’s still quite significant), for example here is comparison of throughput for 3.2 and 3.4: Is there an image missing or something?
- erthalion 9y ago
  Hm...no, it's right there, below this phrase https://imgur.com/a/vN0Q3 https://imgur.com/a/vN0Q3
faragon 9y ago
TL;DR: Use PostgreSQL.
- jack9 9y ago
  Tweak your tests till PostgreSQL looks like the best choice. FTFY
  - solidsnack9000 9y ago
    After reading this comment I went ahead and read the article. I am having trouble finding a case where it seems like the tests are biased in Postgres’s favor but I don’t know very much about MongoDB so could be missing something. Large numbers of clients — really, any more than there are cores — are notably not a Postgres strength.
    - jack9 9y ago
      It doesn't seem like you read it. It's spelled out as directly as it can be. > I assume this one is most important questions for you now. Why PostgreSQL is underperforms so significantly? The answer is simple, we’re doing an unfair benchmark. If PG doesn't perform well, it's unfair.
      - ibotty 9y ago
        > Why PostgreSQL is underperforms so significantly? The answer is simple, we’re doing an unfair benchmark. As you may notice, we were indexing only an ID in a document for MySQL and MongoDB, and an entire document for PostgreSQL. That's hardly biased after and very much before.
        jack9 9y ago
        Three systems treated the same, have different profiles. PG profile was low enough, the benching had to be altered, to get the results that they were expecting (or the reader to be expecting). If it was a technical implementation issue, why wasn't it done right the first time? Probably hubris, in an obvious case of tweak the testing methodology until you get the result you wanted.
ris 9y ago
A note to plot-drawers everywhere. Please don't hide your actual data-points by drawing a bezier curve over them.
- zmmmmm 9y ago
  And PLEASE just label the damn axes. Why do people go to all the effort to draw nice plots, write a giant blog post, but leave of the 1 minute it would take to make the plots interpretable without reading giant slabs of text to reverse engineer out what is actually in the plot. /rant
ris 9y ago
> how to apply document-oriented approach in the world of relational databases I've increasingly found that the number of situations where this is actually a good idea are quite, quite rare.
- HumanDrivenDev 9y ago
  I'll play devils advocate. In any real world system I've worked there are always entities that are purely self contained, but are represented by many different database tables. When they're saved, the entity is decomposed into its constituent parts and written to those tables. When they're retrieved, those tables are joined together to re-create the entity we care about. And that's all that ever happens, it's just serialised and de-serialised. Isn't it premature optimisation to normalise that entity right out of the gate? Why not stuff it in a JSONB column. When - and only when - you find yourself needing to dig inside of it for queries in other parts of the system, then you normalize it.
  - clhodapp 9y ago
    One word: constraints. The problem with actually storing things in a document-oriented manner is that it requires extreme vigilance and no mistakes to actually keep the shape of your documents consistent. Further, it can be extremely costly to discover all the shapes of data you actually have if you initially didn't treat it as important. More often than not, it is useful to use tools that guide you into being disciplined along the way.
    - HumanDrivenDev 9y ago
      Yes, the issue you run into is that then your server has to make sure your json is the right shape. CouchDB actually solves that problem really well IMO. You upload a validation function in javascript to the database that will validate every document that comes in. I wonder if the same thing can't be done with trigger functions in postgres.
      - saltcured 9y ago
pier25 9y ago
Surprising that PG beat Mongo at its own game. We've been recently using Jsonb with PG as a nice-to-have but always assumed Mongo would be much faster with schemaless data.
- gaius 9y ago
  Well, PG is written by people who know what they're doing.
  - Klonoar 9y ago
    ...Mongo isn't run by a bunch of morons, contrary to the Hacker News and /r/programming hate circlejerk boner for them. The level of vitriol on these communities about anything new is really getting old.
  - kevin_thibedeau 9y ago
    LibBSON somewhat humorously has no error handling for OOM and just kills the whole process if a malloc fails. Bit of an annoyance when porting to embedded hardware.
  - Klathmon 9y ago
    Come on can we keep this kind of stuff off HN? If you have concrete criticisms say them so we can have a discussion, don't write snarky lines like this is some kind of "us vs them" game.
    - draw_down 9y ago
      Try being a JavaScript developer on here ;)
    - gaius 9y ago
      In an earlier comment yesterday I remarked that it was good to avoid platform wars, but the MongoDB crew squandered the right to converse as peers with all the lies they told. Remember when they compared their speed writing to memory with Postgres speed writing to disk, as if this made them faster? Either they have fundamentally misunderstood what a database is, or they have deliberately acted in bad faith, which is it?
zmmmmm 9y ago
This is a great post but sad that they addressed everything thoroughly except the most important question for me - query performance. I don't do all this mass updating of document oriented entities. They are stored and they stay stored. But we query the hell out of them so I would love to know how well that actually performs with various indexes on various data types within the document.
- nojvek 9y ago
  Should take a look at flatbuffers. It’s an evolution of protobuf that’s super optimized for querying sunsets of data. Basically lookup of offsets and mmap all the way.
Thaxll 9y ago
That post fails to mention something critical, Postgres doesn't scale and is not HA out of the box so you're comparing a single server solution vs another one that as built-in cluster feature ... What do you do when you need HA? or your data is too big or you need a lot write/sec ect ... The hate on MongoDB is strong on Hacker News / Reddit but I suspect most of you never had to design a solution when master/slave is not enough. Also: https://www.mongodb.com/mongodb-3.4-passes-jepsen-test https://www.mongodb.com/mongodb-3.4-passes-jepsen-test
- WhitneyLand 9y ago
  Just a suggestion that may be worth considering. Sometimes hate is confused with disagreement, and sometimes assuming people you’ve never met have less wisdom can cause hate before you’ve had a fair chance to make your point.
- tgtweak 9y ago
  While json fields in traditional SQL are useful in many ways (avoiding that infamous "extra" column) it really doesn't compare to mongodb. The aggregation framework in mongo for working with these documents is very powerful and if you run pg json fields to do anything other than retrieve extra fields (ie: if you try to query/aggregate it's fields meaningfully as you would a standard field) you'll find they fall over pretty fast with modest amounts of records. The simple scalability of mongo (sharding/replication) is something that, while the recipient of much mocking initially, now at 3.6, gives enterprise level availability and scale with minimal oversight or expertise. I'm really liking the features in 3.6 (over the wire compression, replication hardening, streaming query subscriptions to name a few) and every release packs a ton of features and improvements without breaking anything. Postgre is a great relational database (one of my favorites) but its json field (which even mssql sports nowadays) is not a stand-in replacement for mongo in anyway way.
- itsjustme2 9y ago
  I'm curious whether the parent poster knows about the most recent HA features in Postgres, such as those introduced in v10, and if so, what is still missing for comparable HA. Here's a link to the HA docs for out-of-the-box postgres: https://www.postgresql.org/docs/10/static/high-availability.html https://www.postgresql.org/docs/10/static/high-availability.... As for scaling, it certainly scales out of the box with partitioning, but admittedly a partitioned table doesn't offer all the features that a non-partitioned sql table offers. A great assessment of the current state of postgres scaling can be found here: https://blog.timescale.com/scaling-partitioning-data-postgresql-10-explained-cd48a712a9a1 https://blog.timescale.com/scaling-partitioning-data-postgre...
deleted 9y ago
[deleted]
hermanradtke 9y ago
A bummer that VM’s were used. I have used packet.net to provision bare metal servers for the purposes of benchmarking. The server specs are close to the same servers techempower uses as well.

132 points by erthalion 9y ago

There is absolutely no good reason for anyone to use MongoDB ever, its just such an amazingly rotten database that its astounding how much hype it generated.

Goopplesoft 9y ago
I can easily think of a few good reasons (it is actually a pretty good database for the right use-cases). Can you actually not find a good reason or are you just continuing the exaggerating echo chamber around mongodb's earlier failures for 'points'? Plus this has very little to do with the article which is a very very well done technical analysis of different solutions with very little opining.
- overcast 9y ago
  I mean, the JSON support in PostgreSQL is pretty freaking awesome now. Beyond the sharding, what are your few good reasons?
  - CodesInChaos 9y ago
    One thing I'd like to see in postgres is support for a superset of json, such as timestamps or binary data inside a json document. Replication and failover don't look so great in postgres either.
    - hiram112 9y ago
      Whenever I store binary data as text, I just base64 encode it and convert on the client.
- larrykwg 9y ago
  I think you baiting a rant, I count myself among the myriad developers who had to learn it the hard way to never use mongo. My point would be that there are no "right" use-cases, I pointed it out because the article was talking about a specific thing: as the article points out in the outlined scenario: "It’s clear now, that throughput for PostgreSQL and MongoDB now is almost the same before the spinlock performance degradation hits MongoDB." So the article points out there is a significant performance degradation after a certain number of connections in MongoDB, such things won't surprise me anymore I'll add it to the heap of things wrong with MongoDB, thats why I was making the comment. > "We’ve got so far that throughput of PostgreSQL and MongoDB is the same for read workloads. Is it surprising? Not at all - I’m going to show you, that under the hood all our databases use more or less the same data structures and the same approach to store documents." I'd argue that there is no reason to use MongoDB because there are no use-cases other more mature databases won't be equally or better suited for. MongoDB is horrid, you won't know all its deficiencies when you get started with it, but over time you will come to despise it, it is really far away from being a mature database and I would advice anyone against using it for any task, and I'm not the only one who was burned using it and who adopted this attitude towards it.
wyldfire 9y ago
I used it once when it first came out just to kick the tires. What a remarkably convenient API it had. NoSQL/"Document store" stuff was relatively new (to me, at least). Did the convenient API contribute to whatever design problems they had? No idea. But if they had problems with durability then it really is all for naught. I didn't get beyond prototyping so I never saw whatever issues caused them to get so much ire.
- skywhopper 9y ago
  Yeah, agreed. It does have a much better API than other JSON document stores I've tried. For the right use case, it would be fine.
013a 9y ago
*Today. The thing you're forgetting is that Mongo was actually somewhat novel at the time it was released. I've heard at least two stories of engineering teams who felt that their specific product couldn't have scaled the way it did and they wouldn't here today without Mongo, for whatever reason. I've heard the words "it was the best engineering decision we ever made" from one team who's company ended up selling for 8 figures. You're being hyperbolic. Yes, there are so many better choices for databases out there. There are also many worse choices. Mongo is very hard to recommend nowadays, but unless you go in completely ignoring its problems its not like your company is going to go up in flames.
- user5994461 9y ago
  MongoDB doesn't scale. Write are handled by a single node.
  - gaius 9y ago
    No, it's web scale.
    - user5994461 9y ago
      It's not webscale if there is only one node doing all the work.
  - weddpros 9y ago
    a single node per shard will handle writes. If you have 20 shards, that's 20x more writes than a single node...
- cookiecaper 9y ago
  > I've heard at least two stories of engineering teams who felt that their specific product couldn't have scaled the way it did and they wouldn't here today without Mongo, for whatever reason. I've heard the words "it was the best engineering decision we ever made" from one team who's company ended up selling for 8 figures. Do you expect to hear them say, "Yes, choosing MongoDB was a bad decision and our production infrastructure is currently on an unwieldy, slow, dangerous piece of crap" and then go on to sell the company for tens of millions of dollars? Do you expect people to admit the real reason they're using MongoDB, which, in 99% of cases, is "we don't know how to use SQL and are really perturbed that anyone thinks we should have to learn"? It is true that MongoDB only spontaneously combusts occasionally, but that doesn't mean the choice is of negligible importance.
dang 9y ago
Maybe so, but please don't post unsubstantive comments to HN, or rant here in flamebaity ways. We're looking for thoughtful comments. That doesn't require changing your views at all, just presenting them with more information, and an orientation to good conversation rather than venting.
SkyPuncher 9y ago
On the contrary, It's great when you have relatively unformed data. I've head of a lot of logging (i.e. application logging) companies have great success with Mongo because they can store their entire client's set of data without need to know the schema ahead of time.

dominotw 9y ago

>MongoDB has some problems with spinlocks,...Interesting enough is that this situation is getting better between releases (but it’s still quite significant), for example here is comparison of throughput for 3.2 and 3.4: Is there an image missing or something?

erthalion 9y ago
Hm...no, it's right there, below this phrase https://imgur.com/a/vN0Q3 https://imgur.com/a/vN0Q3

faragon 9y ago

TL;DR: Use PostgreSQL.

jack9 9y ago
Tweak your tests till PostgreSQL looks like the best choice. FTFY
- solidsnack9000 9y ago
  After reading this comment I went ahead and read the article. I am having trouble finding a case where it seems like the tests are biased in Postgres’s favor but I don’t know very much about MongoDB so could be missing something. Large numbers of clients — really, any more than there are cores — are notably not a Postgres strength.
  - jack9 9y ago
    It doesn't seem like you read it. It's spelled out as directly as it can be. > I assume this one is most important questions for you now. Why PostgreSQL is underperforms so significantly? The answer is simple, we’re doing an unfair benchmark. If PG doesn't perform well, it's unfair.
    - ibotty 9y ago
      > Why PostgreSQL is underperforms so significantly? The answer is simple, we’re doing an unfair benchmark. As you may notice, we were indexing only an ID in a document for MySQL and MongoDB, and an entire document for PostgreSQL. That's hardly biased after and very much before.
      - jack9 9y ago
        Three systems treated the same, have different profiles. PG profile was low enough, the benching had to be altered, to get the results that they were expecting (or the reader to be expecting). If it was a technical implementation issue, why wasn't it done right the first time? Probably hubris, in an obvious case of tweak the testing methodology until you get the result you wanted.

ris 9y ago

A note to plot-drawers everywhere. Please don't hide your actual data-points by drawing a bezier curve over them.

zmmmmm 9y ago
And PLEASE just label the damn axes. Why do people go to all the effort to draw nice plots, write a giant blog post, but leave of the 1 minute it would take to make the plots interpretable without reading giant slabs of text to reverse engineer out what is actually in the plot. /rant

ris 9y ago

> how to apply document-oriented approach in the world of relational databases I've increasingly found that the number of situations where this is actually a good idea are quite, quite rare.

HumanDrivenDev 9y ago
I'll play devils advocate. In any real world system I've worked there are always entities that are purely self contained, but are represented by many different database tables. When they're saved, the entity is decomposed into its constituent parts and written to those tables. When they're retrieved, those tables are joined together to re-create the entity we care about. And that's all that ever happens, it's just serialised and de-serialised. Isn't it premature optimisation to normalise that entity right out of the gate? Why not stuff it in a JSONB column. When - and only when - you find yourself needing to dig inside of it for queries in other parts of the system, then you normalize it.
- clhodapp 9y ago
  One word: constraints. The problem with actually storing things in a document-oriented manner is that it requires extreme vigilance and no mistakes to actually keep the shape of your documents consistent. Further, it can be extremely costly to discover all the shapes of data you actually have if you initially didn't treat it as important. More often than not, it is useful to use tools that guide you into being disciplined along the way.
  - HumanDrivenDev 9y ago
    Yes, the issue you run into is that then your server has to make sure your json is the right shape. CouchDB actually solves that problem really well IMO. You upload a validation function in javascript to the database that will validate every document that comes in. I wonder if the same thing can't be done with trigger functions in postgres.
    - saltcured 9y ago

pier25 9y ago

Surprising that PG beat Mongo at its own game. We've been recently using Jsonb with PG as a nice-to-have but always assumed Mongo would be much faster with schemaless data.

gaius 9y ago
Well, PG is written by people who know what they're doing.
- Klonoar 9y ago
  ...Mongo isn't run by a bunch of morons, contrary to the Hacker News and /r/programming hate circlejerk boner for them. The level of vitriol on these communities about anything new is really getting old.
- kevin_thibedeau 9y ago
  LibBSON somewhat humorously has no error handling for OOM and just kills the whole process if a malloc fails. Bit of an annoyance when porting to embedded hardware.
- Klathmon 9y ago
  Come on can we keep this kind of stuff off HN? If you have concrete criticisms say them so we can have a discussion, don't write snarky lines like this is some kind of "us vs them" game.
  - draw_down 9y ago
    Try being a JavaScript developer on here ;)
  - gaius 9y ago
    In an earlier comment yesterday I remarked that it was good to avoid platform wars, but the MongoDB crew squandered the right to converse as peers with all the lies they told. Remember when they compared their speed writing to memory with Postgres speed writing to disk, as if this made them faster? Either they have fundamentally misunderstood what a database is, or they have deliberately acted in bad faith, which is it?

zmmmmm 9y ago

This is a great post but sad that they addressed everything thoroughly except the most important question for me - query performance. I don't do all this mass updating of document oriented entities. They are stored and they stay stored. But we query the hell out of them so I would love to know how well that actually performs with various indexes on various data types within the document.

nojvek 9y ago
Should take a look at flatbuffers. It’s an evolution of protobuf that’s super optimized for querying sunsets of data. Basically lookup of offsets and mmap all the way.

Thaxll 9y ago

That post fails to mention something critical, Postgres doesn't scale and is not HA out of the box so you're comparing a single server solution vs another one that as built-in cluster feature ... What do you do when you need HA? or your data is too big or you need a lot write/sec ect ... The hate on MongoDB is strong on Hacker News / Reddit but I suspect most of you never had to design a solution when master/slave is not enough. Also: https://www.mongodb.com/mongodb-3.4-passes-jepsen-test https://www.mongodb.com/mongodb-3.4-passes-jepsen-test

WhitneyLand 9y ago
Just a suggestion that may be worth considering. Sometimes hate is confused with disagreement, and sometimes assuming people you’ve never met have less wisdom can cause hate before you’ve had a fair chance to make your point.
tgtweak 9y ago
While json fields in traditional SQL are useful in many ways (avoiding that infamous "extra" column) it really doesn't compare to mongodb. The aggregation framework in mongo for working with these documents is very powerful and if you run pg json fields to do anything other than retrieve extra fields (ie: if you try to query/aggregate it's fields meaningfully as you would a standard field) you'll find they fall over pretty fast with modest amounts of records. The simple scalability of mongo (sharding/replication) is something that, while the recipient of much mocking initially, now at 3.6, gives enterprise level availability and scale with minimal oversight or expertise. I'm really liking the features in 3.6 (over the wire compression, replication hardening, streaming query subscriptions to name a few) and every release packs a ton of features and improvements without breaking anything. Postgre is a great relational database (one of my favorites) but its json field (which even mssql sports nowadays) is not a stand-in replacement for mongo in anyway way.
itsjustme2 9y ago
I'm curious whether the parent poster knows about the most recent HA features in Postgres, such as those introduced in v10, and if so, what is still missing for comparable HA. Here's a link to the HA docs for out-of-the-box postgres: https://www.postgresql.org/docs/10/static/high-availability.html https://www.postgresql.org/docs/10/static/high-availability.... As for scaling, it certainly scales out of the box with partitioning, but admittedly a partitioned table doesn't offer all the features that a non-partitioned sql table offers. A great assessment of the current state of postgres scaling can be found here: https://blog.timescale.com/scaling-partitioning-data-postgresql-10-explained-cd48a712a9a1 https://blog.timescale.com/scaling-partitioning-data-postgre...

deleted 9y ago

[deleted]

hermanradtke 9y ago

A bummer that VM’s were used. I have used packet.net to provision bare metal servers for the purposes of benchmarking. The server specs are close to the same servers techempower uses as well.