20 ms·

Building a MongoDB Clone in Postgres

161 points by jerrysievert 14y ago

NewtonsFolly 14y ago
Sounds legit.
nsanch 14y ago
Unless he's planning to build sharding on postgres too, I think he's missing the point.
- jerrysievert 14y ago
  while sharding is an important aspect of mongodb, i don't consider it the most important feature.
  - jshen 14y ago
    that doesn't mean that others don't find it to be an extremely important feature.
  - nsanch 14y ago
    I don't know if it's the _most_ important feature, but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story. With postgres you have to roll your own. If you want to bridge the gap from postgres to mongo, I think that's where you have to start.
    - jshen 14y ago
      "but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story." There are many serious sites that don't need sharding.
      - nsanch 14y ago
        Fair, my statement was overly broad. Sites that are read-only or store blob data in something like S3 can often avoid sharding for quite a while and rely on machines to just get bigger over time. That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad.
        jeffdavis 14y ago
        "That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad." I think you're being too absolute. For instance, Instagram used sharding in postgres, and they didn't have to throw anything away or dedicate any huge engineering team to solve it.
        leothekim 14y ago
        They had to put engineering effort into it. With Mongo, you don't.
        moe 14y ago
        With Mongo, you don't. Bullshit. The sharding impl in MongoDB still[1] crumbles pitifully[2] under load. Regardless of sharding MongoDB still halts the world[3] under write-load. Their map/reduce impl is a joke[4][5]. If you had done the slightest research you'd know that every single aspect that you need to scale out Mongo is either broken by design or so immature that you can't rely on it. MongoDB may be fine as long as your working set fits into RAM on a single server. If you plan to go beyond that then you'd better start with a more stable foundation - or brace yourself for some serious pain. [1] https://groups.google.com/group/mongodb-user/browse_thread/thread/131720fe4fc73f4c https://groups.google.com/group/mongodb-user/browse_thread/t... [2] http://highscalability.com/blog/2010/10/15/troubles-with-sharding-what-can-we-learn-from-the-foursquare.html http://highscalability.com/blog/2010/10/15/troubles-with-sha... [3] http://2.bp.blogspot.com/_VHQJkYQ5-dY/TUO3RAn8SNI/AAAAAAAABqs/FJKgl_HgBWA/s1600/mongo-rw-fail.png http://2.bp.blogspot.com/_VHQJkYQ5-dY/TUO3RAn8SNI/AAAAAAAABq... [4] http://stackoverflow.com/a/3951871 http://stackoverflow.com/a/3951871 [5] http://steveeichert.com/2010/03/31/data-analysis-using-mongodb-map-reduce.html/ http://steveeichert.com/2010/03/31/data-analysis-using-mongo...
    - ibotty 14y ago
      "With postgres you have to roll your own [sharding]" but it will be trivial if you do not do joins.
    - einhverfr 14y ago
      Postgres-XC is also in beta. Basically it's sharded PostgreSQL with full RI enforcement between shards, and seamless query integration. I assume they are working off the 9.2 codebase (hence the beta being the same time as Pg 9.2) but maybe it's only 9.1 (i.e. no JSON). Postgres-XC is probably the most exciting PostgreSQL-related project out there. It promises full write-extensibility across the cluster without sacrificing consistency.
    - cbsmith 14y ago
      > I don't know if it's the _most_ important feature, but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story. You know people say this, but in practice I find that simple hash bucketing with a redundant pair works surprisingly well, particularly in the cloud. Yes it isn't fancy, but it is trivial to manage and debug, and you can do a lot of optimizations given such a clear cut set of partitioning rules. Your problems have to get really big before a more sophisticated mechanism really pays off in terms of avoiding headaches, and often the more sophisticated mechanisms actually cause more headaches before you get there.
  - dickeytk 14y ago
    It also seems to be the flakiest feature
- deleted 14y ago
  [deleted]
- jeffdavis 14y ago
  "Unless he's planning to build sharding on postgres too, I think he's missing the point." NoSQL seems to have three general claims (I'm not saying whether these are correct or not): (1) ease of administration in some cases; (2) different data model; (3) better performance or availability in some situations. The author is clearly addressing the second, and you are clearly talking about the third.
  - nsanch 14y ago
    Good point.
- bri3d 14y ago
  Should be pretty trivial for him to shard collections across multiple databases with the same level of intelligence as MongoDB's automated sharding simply using the primary key, since MongoDB doesn't join. Not sure how sharding is the point of MongoDB, though - in most of the universe, sharding is a database architecture/schema-level thing, not a database-server level thing, and for good reason - it's pretty darn hard for a database server to shard effectively without some knowledge of the app layer (and which keys are likely to become hot). Personally, if I really wanted a flexible-schema "document based" database, I'd have implemented this using the FriendFeed K/V + Index model ( http://backchannel.org/blog/friendfeed-schemaless-mysql http://backchannel.org/blog/friendfeed-schemaless-mysql ) plus Postgres's HStore functionality, storing K/V per document in an HStore rather than in one giant K/V table like FriendFeed. That way I wouldn't need to use V8 and JSON parsing to run queries, and the mythic MongoDB-style "sharding" would be just as easy (just distribute the document -> hstore table across shards keyed on ID again).
  - nsanch 14y ago
    I disagree that sharding can ever be a trivial problem if you're going to try to tackle moving data between shards while staying online. I'm not saying it's impossible, just that it's not trivial.
    - bri3d 14y ago
      MongoDB's relatively simple approach involves continuing to use the old shard as an authoritative source (and committing updates to it) while shipping data in the background, then pushing the additional changes across and marking the new shard as "master." Such an approach wouldn't be horribly difficult to implement in SQL using a copy table and write triggers - almost identically to how SoundCloud's Large Hadron Migrator allows writes to occur over a MySQL InnoDB table that's locked for migration (but even simpler because the table schema can't conflict afterwords). The entire problem is admittedly nontrivial, since if the application happens to be writing data to the shard under migration too quickly (or the shard being migrated to dies), the server can end up in a situation where the new shard is never able to catch up and become a master. However, the easy solution (give up and retry later) is Good Enough for most situations (and is pretty much how MongoDB works).
- mark_l_watson 14y ago
  You have a good point. I must admit that after many years of using and loving PostgreSQL I have never had to scale it out: that is outside of my experiences. On the other hand, old fashion MongoDB master slave configurations and now replica sets have been easy for me to set up when on just a few occasions I had to use a scaled out MongoDB setup.
- einhverfr 14y ago
  Easily enough done on Postgres-XC, and you still get full ACID compliance across shards as well!
baq 14y ago
so, um, yeah... http://www.postgresql.org/docs/9.1/static/hstore.html http://www.postgresql.org/docs/9.1/static/hstore.html
- jerrysievert 14y ago
  still key/value, no deep packet inspection.
- deleted 14y ago
  [deleted]
StavrosK 14y ago
This is (almost) exactly what Goatfish[1] does, except it uses SQLite, and can create SQL indexes on any arbitrary field. I needed a schemaless, embeddable store, so I decided to go with that, since Postgres already has the hstore. It's still very simple and preliminary, but it works, and it's very useful for prototyping. I'd like to develop it some more, if people found it useful and started using it. [1] https://github.com/stochastic-technologies/goatfish https://github.com/stochastic-technologies/goatfish
DEinspanjer 14y ago
It is interesting, but what I really need in the JSON functionality of PG is some internal representation that will allow fast and efficient exploration of the JSON blob within the query. i.e. being able to refer to a single attribute within the select/where/groupby clauses without having to pay the toll of serde every time.
- jerrysievert 14y ago
  exactly. calling the find_in_obj() function can get fairly expensive, especially given the need to execute JSON.parse() for every call. there is definitely a lot of room for improvement in the postgres native JSON toolkit. i am hoping that building this exposes more of those issues and helps move it forward.
  - disbelief 14y ago
    But the same is true for MongoDB. If you're querying an attribute of a document that isn't indexed, Mongo has to scan all of the Documents in the collection for it. The solution to this is to have additional indexes on any attributes you want to be able to query by. This holds true both in Mongo and this Postgres implementation.
  - rarrrrrr 14y ago
    I haven't tried it, but I think you would be able to use a PostgreSQL functional index (i.e. indexing the result of a function call to extract a particular bit of data from the JSON.) http://www.postgresql.org/docs/9.1/interactive/indexes-expressional.html http://www.postgresql.org/docs/9.1/interactive/indexes-expre...
    - rosser 14y ago
      I was just coming here to say exactly this. You could also use a partial index (that is, an index with a WHERE clause; so, "WHERE json_field LIKE '%foo.bar = baz%'" or whatever).
    - jerrysievert 14y ago
jeffdavis 14y ago
Relational databases have a history of absorbing the advantages of other systems when they come along, particularly changes to the model (cf. object databases and XML databases). As the author shows, a similar thing will happen quite quickly for document models. Architectural changes are slower, but you can also start to see this happening in postgres with features like unlogged tables (i.e. don't write to the recovery log for changes to this table) and transaction-controlled async commit (i.e. don't wait for this transaction to hit disk). More changes are in the works. MongoDB will have a chance to have an impact, and will be successful if they are able to keep innovating. If they just stand still or incrementally improve, they will be marginalized. Now that Mongo has center stage in the NoSQL movement, it will be interesting to see what they do next.
- autarch 14y ago
  I'd phrase this differently. Once you implement Mongo in Postgres, what you have isn't a relational database. What you are taking advantage of is the incredibly solid underlying infrastructure of Pg, which provides reliability, scalability, transactions, replication, etc. But the data model itself is no longer relational from the application's perspective.
  - jeffdavis 14y ago
    Every data model is just a degenerate special case of every other data model ;)
mtkd 14y ago
I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (in a good way) - and frictionless for development. It really fits what I want to do with web apps in particular. So far I've not needed to scale it, but I've less apprehension about that than I used to have about scaling SQL back in the day - the large denormalised tables I used to build from SQL for performance are now the default. SQL still has a place and MongoDB is no replacement for complex models, but give it a try before buying that you can do it all with Postgres tweaks.
- bad_user 14y ago
  Experimenting with new technologies is encouraged, especially since cross-pollination of ideas happen this way. However I would actually advise developers to stop and think if they really need MongoDB or the latest fad, because their current relational database, such as PostgreSQL, does a mighty fine job for most of their needs. Why? Because I have never seen angry opinions about PostgreSQL losing people's data. Or about how the modeling tools and architecture exposed by PostgreSQL are insufficient for certain problems ... not until you're operating at Google's scale, and MongoDB won't save you there ;) give it a try before buying that you can do it all with Postgres tweaks My main problem with most NoSQL solutions is that I have to tweak the problems I have to fit the solution, instead of the other way around. Technologies that can be tweaked simply rock.
  - taligent 14y ago
    I guess if only I "woke up" I would realise that PostgreSQL is the answer to all my problems. Guess what. PostgreSQL is just another SQL database. It's definitely one of the best ones but it still stuffers from all the same issues, limitations and frustrations. Many of which stem not from the database itself but from the relational modelling and tools to support it. I use Java + MongoDB and life is significantly better now that I don't have to worry about the domain model so much. I can have lists, maps etc and can add classes or make changes seamlessly. It's worth the tradeoffs for me.
roncohen 14y ago
If you just need a simple key/value, a good way to go is HSTORE in PostgreSQL. It allows only string key/values, and not complex structures. Use the JSON datatype if you need lists, nested objects etc. With HSTORE you get indexes on keys.
- k_bx 14y ago
  I wonder if postgres will adopt BSON instead of JSON and implement all the querying -- that would be so much better. p.s.: guys who disagree -- why would you do that? I mean, BSON lets you quickly skip (embedded) documents you're not interested in, since it stores their size. Also it has type info and some additional types. So it's just "better JSON" for storing and navigating, and if postgres wants to have querying in JSON, they will either take BSON or invent the wheel for something similar.
peterhunt 14y ago
This is the right way to think about building a "NoSQL" datastore. I think that schemaless, eventually-consistent data stores have a place and are useful. I just think that most of the current efforts are throwing away years of investment in SQL datastores. Rather than thinking of NoSQL as a brand-new paradigm shift that requires a ground-up reimplementation, we need to think of it as layer of abstraction on top of MySQL and memcache (or your preferred setup). Re-implementing all of the work that has gone into these projects is a bad idea and is contrary to The Unix Way. Thinking of SQL databases as a storage engine rather than the kitchen sink is the key to building a scalable system.
abarales 14y ago
LEARN PHOTOGRAPHY EASILY AND MAKE MONEY WIHTOUTSWEATING.BARALESFHOTO.BLOGSPOT.COM

161 points by jerrysievert 14y ago

NewtonsFolly 14y ago

Sounds legit.

nsanch 14y ago

Unless he's planning to build sharding on postgres too, I think he's missing the point.

jerrysievert 14y ago
while sharding is an important aspect of mongodb, i don't consider it the most important feature.
- jshen 14y ago
  that doesn't mean that others don't find it to be an extremely important feature.
- nsanch 14y ago
  I don't know if it's the _most_ important feature, but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story. With postgres you have to roll your own. If you want to bridge the gap from postgres to mongo, I think that's where you have to start.
  - jshen 14y ago
    "but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story." There are many serious sites that don't need sharding.
    - nsanch 14y ago
      Fair, my statement was overly broad. Sites that are read-only or store blob data in something like S3 can often avoid sharding for quite a while and rely on machines to just get bigger over time. That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad.
      - jeffdavis 14y ago
        "That said, if your site grows in some way you didn't originally anticipate and you get to a point where you need to shard, but can only do so by changing data stores, then it's sad." I think you're being too absolute. For instance, Instagram used sharding in postgres, and they didn't have to throw anything away or dedicate any huge engineering team to solve it.
        leothekim 14y ago
        They had to put engineering effort into it. With Mongo, you don't.
        moe 14y ago
        With Mongo, you don't. Bullshit. The sharding impl in MongoDB still[1] crumbles pitifully[2] under load. Regardless of sharding MongoDB still halts the world[3] under write-load. Their map/reduce impl is a joke[4][5]. If you had done the slightest research you'd know that every single aspect that you need to scale out Mongo is either broken by design or so immature that you can't rely on it. MongoDB may be fine as long as your working set fits into RAM on a single server. If you plan to go beyond that then you'd better start with a more stable foundation - or brace yourself for some serious pain. [1] https://groups.google.com/group/mongodb-user/browse_thread/thread/131720fe4fc73f4c https://groups.google.com/group/mongodb-user/browse_thread/t... [2] http://highscalability.com/blog/2010/10/15/troubles-with-sharding-what-can-we-learn-from-the-foursquare.html http://highscalability.com/blog/2010/10/15/troubles-with-sha... [3] http://2.bp.blogspot.com/_VHQJkYQ5-dY/TUO3RAn8SNI/AAAAAAAABqs/FJKgl_HgBWA/s1600/mongo-rw-fail.png http://2.bp.blogspot.com/_VHQJkYQ5-dY/TUO3RAn8SNI/AAAAAAAABq... [4] http://stackoverflow.com/a/3951871 http://stackoverflow.com/a/3951871 [5] http://steveeichert.com/2010/03/31/data-analysis-using-mongodb-map-reduce.html/ http://steveeichert.com/2010/03/31/data-analysis-using-mongo...
  - ibotty 14y ago
    "With postgres you have to roll your own [sharding]" but it will be trivial if you do not do joins.
  - einhverfr 14y ago
    Postgres-XC is also in beta. Basically it's sharded PostgreSQL with full RI enforcement between shards, and seamless query integration. I assume they are working off the 9.2 codebase (hence the beta being the same time as Pg 9.2) but maybe it's only 9.1 (i.e. no JSON). Postgres-XC is probably the most exciting PostgreSQL-related project out there. It promises full write-extensibility across the cluster without sacrificing consistency.
  - cbsmith 14y ago
    > I don't know if it's the _most_ important feature, but I wouldn't build a serious site on top of anything that didn't have some sort of built-in sharding story. You know people say this, but in practice I find that simple hash bucketing with a redundant pair works surprisingly well, particularly in the cloud. Yes it isn't fancy, but it is trivial to manage and debug, and you can do a lot of optimizations given such a clear cut set of partitioning rules. Your problems have to get really big before a more sophisticated mechanism really pays off in terms of avoiding headaches, and often the more sophisticated mechanisms actually cause more headaches before you get there.
- dickeytk 14y ago
  It also seems to be the flakiest feature
deleted 14y ago
[deleted]
jeffdavis 14y ago
"Unless he's planning to build sharding on postgres too, I think he's missing the point." NoSQL seems to have three general claims (I'm not saying whether these are correct or not): (1) ease of administration in some cases; (2) different data model; (3) better performance or availability in some situations. The author is clearly addressing the second, and you are clearly talking about the third.
- nsanch 14y ago
  Good point.
bri3d 14y ago
Should be pretty trivial for him to shard collections across multiple databases with the same level of intelligence as MongoDB's automated sharding simply using the primary key, since MongoDB doesn't join. Not sure how sharding is the point of MongoDB, though - in most of the universe, sharding is a database architecture/schema-level thing, not a database-server level thing, and for good reason - it's pretty darn hard for a database server to shard effectively without some knowledge of the app layer (and which keys are likely to become hot). Personally, if I really wanted a flexible-schema "document based" database, I'd have implemented this using the FriendFeed K/V + Index model ( http://backchannel.org/blog/friendfeed-schemaless-mysql http://backchannel.org/blog/friendfeed-schemaless-mysql ) plus Postgres's HStore functionality, storing K/V per document in an HStore rather than in one giant K/V table like FriendFeed. That way I wouldn't need to use V8 and JSON parsing to run queries, and the mythic MongoDB-style "sharding" would be just as easy (just distribute the document -> hstore table across shards keyed on ID again).
- nsanch 14y ago
  I disagree that sharding can ever be a trivial problem if you're going to try to tackle moving data between shards while staying online. I'm not saying it's impossible, just that it's not trivial.
  - bri3d 14y ago
    MongoDB's relatively simple approach involves continuing to use the old shard as an authoritative source (and committing updates to it) while shipping data in the background, then pushing the additional changes across and marking the new shard as "master." Such an approach wouldn't be horribly difficult to implement in SQL using a copy table and write triggers - almost identically to how SoundCloud's Large Hadron Migrator allows writes to occur over a MySQL InnoDB table that's locked for migration (but even simpler because the table schema can't conflict afterwords). The entire problem is admittedly nontrivial, since if the application happens to be writing data to the shard under migration too quickly (or the shard being migrated to dies), the server can end up in a situation where the new shard is never able to catch up and become a master. However, the easy solution (give up and retry later) is Good Enough for most situations (and is pretty much how MongoDB works).
mark_l_watson 14y ago
You have a good point. I must admit that after many years of using and loving PostgreSQL I have never had to scale it out: that is outside of my experiences. On the other hand, old fashion MongoDB master slave configurations and now replica sets have been easy for me to set up when on just a few occasions I had to use a scaled out MongoDB setup.
einhverfr 14y ago
Easily enough done on Postgres-XC, and you still get full ACID compliance across shards as well!

baq 14y ago

so, um, yeah... http://www.postgresql.org/docs/9.1/static/hstore.html http://www.postgresql.org/docs/9.1/static/hstore.html

jerrysievert 14y ago
still key/value, no deep packet inspection.
deleted 14y ago
[deleted]

StavrosK 14y ago

This is (almost) exactly what Goatfish[1] does, except it uses SQLite, and can create SQL indexes on any arbitrary field. I needed a schemaless, embeddable store, so I decided to go with that, since Postgres already has the hstore. It's still very simple and preliminary, but it works, and it's very useful for prototyping. I'd like to develop it some more, if people found it useful and started using it. [1] https://github.com/stochastic-technologies/goatfish https://github.com/stochastic-technologies/goatfish

DEinspanjer 14y ago

It is interesting, but what I really need in the JSON functionality of PG is some internal representation that will allow fast and efficient exploration of the JSON blob within the query. i.e. being able to refer to a single attribute within the select/where/groupby clauses without having to pay the toll of serde every time.

jerrysievert 14y ago
exactly. calling the find_in_obj() function can get fairly expensive, especially given the need to execute JSON.parse() for every call. there is definitely a lot of room for improvement in the postgres native JSON toolkit. i am hoping that building this exposes more of those issues and helps move it forward.
- disbelief 14y ago
  But the same is true for MongoDB. If you're querying an attribute of a document that isn't indexed, Mongo has to scan all of the Documents in the collection for it. The solution to this is to have additional indexes on any attributes you want to be able to query by. This holds true both in Mongo and this Postgres implementation.
- rarrrrrr 14y ago
  I haven't tried it, but I think you would be able to use a PostgreSQL functional index (i.e. indexing the result of a function call to extract a particular bit of data from the JSON.) http://www.postgresql.org/docs/9.1/interactive/indexes-expressional.html http://www.postgresql.org/docs/9.1/interactive/indexes-expre...
  - rosser 14y ago
    I was just coming here to say exactly this. You could also use a partial index (that is, an index with a WHERE clause; so, "WHERE json_field LIKE '%foo.bar = baz%'" or whatever).
  - jerrysievert 14y ago

jeffdavis 14y ago

Relational databases have a history of absorbing the advantages of other systems when they come along, particularly changes to the model (cf. object databases and XML databases). As the author shows, a similar thing will happen quite quickly for document models. Architectural changes are slower, but you can also start to see this happening in postgres with features like unlogged tables (i.e. don't write to the recovery log for changes to this table) and transaction-controlled async commit (i.e. don't wait for this transaction to hit disk). More changes are in the works. MongoDB will have a chance to have an impact, and will be successful if they are able to keep innovating. If they just stand still or incrementally improve, they will be marginalized. Now that Mongo has center stage in the NoSQL movement, it will be interesting to see what they do next.

autarch 14y ago
I'd phrase this differently. Once you implement Mongo in Postgres, what you have isn't a relational database. What you are taking advantage of is the incredibly solid underlying infrastructure of Pg, which provides reliability, scalability, transactions, replication, etc. But the data model itself is no longer relational from the application's perspective.
- jeffdavis 14y ago
  Every data model is just a degenerate special case of every other data model ;)

mtkd 14y ago

I do encourage any SQL user, who hasn't already tried MongoDB, to fire it up and try it themselves. Mongoid in Ruby is fairly fast to get started. I've been using SQL since early 90s. For web apps and large collections I've started using MongoDB more recently. It's one of the most exciting technologies I've used in a long time. It takes a while to stop thinking SQL, but once you pass that it's really very primitive (in a good way) - and frictionless for development. It really fits what I want to do with web apps in particular. So far I've not needed to scale it, but I've less apprehension about that than I used to have about scaling SQL back in the day - the large denormalised tables I used to build from SQL for performance are now the default. SQL still has a place and MongoDB is no replacement for complex models, but give it a try before buying that you can do it all with Postgres tweaks.

bad_user 14y ago
Experimenting with new technologies is encouraged, especially since cross-pollination of ideas happen this way. However I would actually advise developers to stop and think if they really need MongoDB or the latest fad, because their current relational database, such as PostgreSQL, does a mighty fine job for most of their needs. Why? Because I have never seen angry opinions about PostgreSQL losing people's data. Or about how the modeling tools and architecture exposed by PostgreSQL are insufficient for certain problems ... not until you're operating at Google's scale, and MongoDB won't save you there ;) give it a try before buying that you can do it all with Postgres tweaks My main problem with most NoSQL solutions is that I have to tweak the problems I have to fit the solution, instead of the other way around. Technologies that can be tweaked simply rock.
- taligent 14y ago
  I guess if only I "woke up" I would realise that PostgreSQL is the answer to all my problems. Guess what. PostgreSQL is just another SQL database. It's definitely one of the best ones but it still stuffers from all the same issues, limitations and frustrations. Many of which stem not from the database itself but from the relational modelling and tools to support it. I use Java + MongoDB and life is significantly better now that I don't have to worry about the domain model so much. I can have lists, maps etc and can add classes or make changes seamlessly. It's worth the tradeoffs for me.

roncohen 14y ago

If you just need a simple key/value, a good way to go is HSTORE in PostgreSQL. It allows only string key/values, and not complex structures. Use the JSON datatype if you need lists, nested objects etc. With HSTORE you get indexes on keys.

k_bx 14y ago
I wonder if postgres will adopt BSON instead of JSON and implement all the querying -- that would be so much better. p.s.: guys who disagree -- why would you do that? I mean, BSON lets you quickly skip (embedded) documents you're not interested in, since it stores their size. Also it has type info and some additional types. So it's just "better JSON" for storing and navigating, and if postgres wants to have querying in JSON, they will either take BSON or invent the wheel for something similar.

peterhunt 14y ago

This is the right way to think about building a "NoSQL" datastore. I think that schemaless, eventually-consistent data stores have a place and are useful. I just think that most of the current efforts are throwing away years of investment in SQL datastores. Rather than thinking of NoSQL as a brand-new paradigm shift that requires a ground-up reimplementation, we need to think of it as layer of abstraction on top of MySQL and memcache (or your preferred setup). Re-implementing all of the work that has gone into these projects is a bad idea and is contrary to The Unix Way. Thinking of SQL databases as a storage engine rather than the kitchen sink is the key to building a scalable system.

abarales 14y ago

LEARN PHOTOGRAPHY EASILY AND MAKE MONEY WIHTOUTSWEATING.BARALESFHOTO.BLOGSPOT.COM