24 ms·

DynamoDB to Postgres: Why and How

116 points by orf 9y ago

paladin314159 9y ago
> Assuming you know how your data needs to be queried when you create your table, then the hash/range key combination could be the answer to your query problems. However we did not. DynamoDB's purpose is to allow you to easily scale a known query pattern. I definitely don't recommend it as a first solution when you don't know how the data is going to be accessed. Or when everything fits on one PostgreSQL box. We've had a lot of success starting out with a big PostgreSQL box and migrating use cases to DynamoDB when a table gets too big and has a query pattern that fits (essentially K/V lookup with some simple indexes).
- gamesbrainiac 9y ago
  Good news is that postgres is going to get multi-master replication soon in postgres 10, so a lot of the painpoints when it comes to scaling our writes, are hopefully going to be solved.
  - bradhe 9y ago
    > are hopefully going to be solved. Right, because multi-master replication schemes have such a long lineage of being operationally simple. /s
    - paladin314159 9y ago
      Yeah, that's the real power of DynamoDB, despite all of its shortcomings. It's operationally simple (given you have the money).
  - crypto5 9y ago
    > Good news is that postgres is going to get multi-master replication soon in postgres 10 Where did you read this?..
    - anarazel 9y ago
      I think the poster of that claim is confusing "plain" logical replication and multi-master replication. There's a lot of infrastructure in common between both, but the conflict resolution part of async mm didn't make it it into postgres 10...
  - b777 9y ago
    How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all servers? And if all writes need to happen on all servers, how does that improve write scalability as the load write load on each server is the same as it would be without multi-master replication.
    - anarazel 9y ago
      > How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all servers? It doesn't generally. You often can increase the total throughput because independent transactions can be batch committed when replicated, which can increase throughput noticeably, but that's more like a constant factor improvement than a real scalability improvement. What it does however often allow you to do is to: a) Address latency issues with writers distributed across the globe. If you have masters on each continent your total write rate can often be massively higher than if you have application servers from each of those continents write to a centralized database. b) Partially replicate your data globally, partially not. Quite often you'll want to have some data available everywhere (typically metadata, security related data, etc), but a lot of the bulk data only available in certain places.
      - darkr 9y ago
        > a) Address latency issues with writers distributed across the globe How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Same with incrementing IDs etc.
        anarazel 9y ago
        > > a) Address latency issues with writers distributed across the globe > How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Well, there's different systems out there. What you want for something like this is asynchronous multi-master. Which, as you observe, allows for conflicts to occur, which need to be resolved somehow. > Same with incrementing IDs etc. The solution for those is usually to have each node assign sequences with different start values, have a system that distributes "chunks" of values (that was what BDR, an async MM version of postgres used to do), or avoiding sequences in favor of UUIDs or similar.
  - m00x 9y ago
    Wouldn't multi-master replication be bad for write throughput? Depending on the scheme, it will have to be committed to all machines for every write, therefore increasing write operation complexity. If you want to scale up your writes, sharding would be a better solution.
  - manigandham 9y ago
    > postgres is going to get multi-master replication soon in postgres 10 No it's not. Postgres barely has decent replication as is. Logical replication is a good step beyond (while still missing major functionality) but nowhere near what it takes for multi-master replication.
    - anarazel 9y ago
      > No it's not. Postgres barely has decent replication as is. You're overly negative whenever postgres comes up on HN. Obviously the v10 stuff is not multi-master and the project definitely is not claiming it has, but saying that Postgres' replication "barely has decent replication as is" is a bit absurd given that it works quite well in a lot of high-throughput scenarios in a lot of companies, and has so for 5+ years. There's a lot of valid complaints about it you can make (especially around flexibility and operational issues), but it's definitely not just "barely decent". I'm not saying that you have to be fan. It's fine to say the replication isn't good enough for everything, or just ignore postgres. > but nowhere near what it takes for multi-master replication. FWIW, there's async MM built on top of the relevant pieces, and it's used in production. It's not ready for wide audiences, but arguably (and I think one can very validly disagree about that) it's closer than "nowhere near".
      - anarazel 9y ago
        https://news.ycombinator.com/item?id=14578213 https://news.ycombinator.com/item?id=14578213 https://news.ycombinator.com/item?id=14373424 https://news.ycombinator.com/item?id=14373424 https://news.ycombinator.com/item?id=13653708 https://news.ycombinator.com/item?id=13653708 https://news.ycombinator.com/item?id=13423951 https://news.ycombinator.com/item?id=13423951 ...
      - manigandham 9y ago
        What is the point of this comment? And then linking to my past comments? Is it wrong to have perspective and criticism or to express it multiple times? I get that you're a committer but "overly negative" is as much your opinion as what I say, and my comments are the truth as I see it. I've even spent an hour talking to Ozgun about databases and moving to Citus before so my statements come from being a user, operator and maintainer of dozens of different database technologies in production for years, including postgres right now, and from all that experience it is definitely the most behind when it comes to replication/HA compared to anything else I use. All of those threads have plenty of upvotes and other user comments so it's clear I don't stand alone here. Perhaps instead of policing my statements, time would be better spent on improving the features and experience of the product.
aluskuiuc 9y ago
I'm surprised the author didn't consider using DynamoDB Streams to replicate to a PGSQL database for the relational queries while keeping the primary key-value lookups on DDB.
- manigandham 9y ago
  What does that gain other than more expense and overhead? SQL can do key/value just fine.
williamstein 9y ago
This closely mirrors my experience a few months ago: https://news.ycombinator.com/item?id=12649712 https://news.ycombinator.com/item?id=12649712
linkmotif 9y ago
> I’m not a database guy, I’m a node guy. Is this really a distinction your "scrappy startup type" can afford to make? > Although since version 3.4.x Mongo passes all Jepsen tests, when our original database was chosen the current version of Mongo at the time did not and was deemed unreliable by the team Does not pass all Jepsen tests => deeemed unreliable :( Wow. Of all possible reasons, THAT'S the reason MongoDB wasn't chosen? Jepsen, man. Pretty sad so many people just look at Jepsen results like this. Jepsen, really, is about the tests more than the results. It highlights various failure modes that may be difficult or uncommon to replicate in production. These failure modes are important to know about to understand the product you are using, but are not really the end-all when it comes to deciding whether that product is worth using! Anyway, show me a database that passes Jepsen and I'll show you a database that only accepts writes to a single node.
- andreimatei1 9y ago
  > Anyway, show me a database that passes Jepsen and I'll show you a database that only accepts writes to a single node. I know one - CockroachDB :). Relational, strongly-consistent, SQL, elastic, multi-active replication - the dream! (I work on it) https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-jepsen-testing/ https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-j... http://jepsen.io/analyses/cockroachdb-beta-20160829 http://jepsen.io/analyses/cockroachdb-beta-20160829
  - hmottestad 9y ago
    RethinkDB also passes jepsen.
    - linkmotif 9y ago
      It's also defunct so... :)
      - jjuel 9y ago
        Open Sourced is definitely different than defunct...
neovintage 9y ago
The really important part of this post is the section articulating why not Dynamo. It uncovers a big question I ask of all of my customers when they're trying to choose data stores at any point during the life of their application. Do you know your query patterns for your application? If you know how your application / product need to pull data from the data store, you way more information that will allow you to pick the right data store to fit the problem at hand. When you don't have any prior query patterns to go off of, picking a relational database to start your application is a good choice. You need the flexibility to allow your schema to change over time. The schema and query patterns will eventually reach a point where changes don't happen all that often and then you can look to other datastores if you need to scale. Plus, are you really going to have THAT much data when you start? Disclosure: I work on Heroku Postgres. Could be a little biased....
- lilbobbytables 9y ago
  Similar to the article author, many just don't have experience with relational db's and have only heard that they aren't flexible and don't scale. They don't realize just how easily performant Postgres can be - and often so easily by sticking basic indexes where they are needed. Similarly they may not realize the amount of data and load PG or MySQL can handle just fine.
orf 9y ago
I submitted this article because of a single paragraph: > When it came to the different types of NoSQL databases, it was pretty clear that a document store best suited our needs. We needed to be able to query based on the data so key/value was out, there was no need to have the ability to make all of the complex connections between data of a graph db, and nobody actually knows what a column db is (jk but our data is not the right use case for it). So document it was. We would have to change our data layout a bit to really work well with most document databases, our current data structure was similar to what you would expect for a relational db with tables etc since dynamo also uses a table structure. I wanted something that was ACID, none of that eventual consistency nonsense. That took a lot of them out of the race right there. I don't get it. He needed arbitrary querying, relations, ACID, tables. And yet... his first thought was to go to a document-based no-sql database and is only later hit with the sudden realization that his relational data is bet suited in a relational database. What has gone so wrong here that an otherwise smart and educated engineer makes these kinds of fundamental mistakes? Is it the success of no-sql marketing departments, and the apparent un-sexiness of traditional relational databases? Is it education? The job market? A fear of SQL (that you never really have to write anyway...)? Is it the quick-fix non-relational databases give you? As far as I can see there are very very very few nails that a PostgreSQL hammer cannot hit. I guess that's part of the point of the article, but why does it need an article to explain this? How can we stop these kinds of articles from needing to be written? > The change drastically increased the speed of our API, some requests are up to 6x faster than they were with Dynamo. The account details request, our previously slowest call, took around 12 seconds with Dynamo and was reduced to 2 seconds with Postgres! sigh Edit: Not sure why this has been moved to the bottom of the thread. I wasn't trying to single this particular developer out, I was merely commenting on this troubling trend. Apologies if this came across in the wrong way.
- misterbowfinger 9y ago
  When I hear "brogrammer", I'm usually a little put-off because it's kind of a troll term. But this article, wow. You can call me on it - brogrammers are ruining the industry.
  - 9y ago
hesdeadjim 9y ago
DynamoDB was a constant source of anxiety for me and my team. We fit the usage pattern perfectly, so the author's points didn't apply to us. Our primary problem was having to constantly manage and change our provisioned capacity so that we could respond to usage spikes and keep our bill from becoming stratospheric. Worse, anytime we'd add new functionality -- such as nighttime analytics jobs that scanned the table or daily backups, it meant revisiting the parameters for our auto-scaling or risk getting throttled and having our application randomly start spewing errors. DynamoDB's billing and provisioning model is awful to deal with. When your team is having serious conversations about the repercussions of your data values going from 4kb to 8kb, you very quickly begin questioning your decision to use DynamoDB in the first place.
- eropple 9y ago
  The high score I ever saw at a company I worked for was about $85K in DynamoDB spend. That's an eight, a five, and three zeroes after it. It was a feature so otherwise silly and simple that I don't want to flame the developers thereof by describing it, but it might have been the most gratuitous burning of cash I've ever seen at a startup.
  - hesdeadjim 9y ago
    Ha, yea I've heard of numbers like that. I remember talking to someone at re:Invent who described their high five figures monthly bill in almost reverent terms -- it made sense after he described their 200+ MySQL server catastrophe DynamoDB replaced. We actually ended up engineering our system to use a variant of event sourcing. This let us dump DynamoDB and use RedisLabs and S3 in such a way that we were able to handle 1mil DAU with a 10:1 write/read ratio for around $5k/mo -- including front end/backend instances.
    - kornish 9y ago
      > This let us dump DynamoDB and use RedisLabs and S3 in such a way that we were able to handle 1mil DAU with a 10:1 write/read ratio This sounds really interesting – do you have a writeup of the system architecture anywhere?
idbehold 9y ago
I'd also like to remind anyone using DynamoDB: it cannot store empty strings. Let that sink in for a second. If you think you're going to be able to store JSON documents you'll be in for a nasty surprise when you try to insert a valid JSON document and either the request fails or it converts any empty strings in your JSON document into nulls.
- koolba 9y ago
  An empty string (zero bytes) isn't valid JSON. '""' vs '' (single quotes to illustrate the wrapper for the total content).
  - idbehold 9y ago
    {"foo":""} That's valid JSON that cannot be stored in DynamoDB because the value of "foo" is an empty string. See previous discussion: https://news.ycombinator.com/item?id=13170746 https://news.ycombinator.com/item?id=13170746
    - koolba 9y ago
      Ah I didn't notice the distinction between storing the serialzied form (which would be non empty) and storing the record itself. Yes that does suck quite a bit and seems like a pretty bad decision.
mark242 9y ago
Dismissing CouchDB because it only supports pre-sharding is extremely shortsighted. It takes very little time to replicate from one db with 2 shards to another db with 3 shards, even on N nodes. Copying the *.couch files is the absolute wrong way to manage this especially in a production environment when those files will be constantly changing. It is far, far easier to add nodes to a Cloudant/Couch2 cluster, set up realtime replication from db1 -> db2, then once the replication has caught up, at your leisure you switch your database connection in your app.
dchuk 9y ago
It doesn't really seem like this guy was qualified to make such a pivotal decision like choosing the database your entire system will run on given his self-submitted breadth of experience in the opening paragraphs...
- takeda 9y ago
  You could say I'm a Postgres fanboy, but even to me he did not sound convincing. The guy did not seem to know what he was talking about. Almost felt like he picked Postgres by throwing darts at possible options. Yes, he picked the right choice :), but I feel like he will misuse it[1] and few months later write article how bad it was. [1] Heh, I've seen Postgres tables that had 4 columns: - id - date added - date updated - thrift encoded blob containing the data (that was before JSONB, but even then the data was highly relational and that was silly)
luord 9y ago
My own experience working with Postgres, discovering nifty features or trying out new ones, and articles like this make it clear for me that I'm unlikely to ever need anything else. By the time any of the applications I work on could grow to the size in which Postgres wouldn't be a good option, I'd hopefully be in the process of selling it anyway. I'm quite ok with that. It's such an awesome piece of software.
pedalpete 9y ago
I've just started using DynamoDB on a project. Before I made the choice, I discussed some architecture issues with a friend who works at a big fast growing unicorn start-up. They use MongoDB for the majority of the app the user sees, they use ElasticSearch for search, postgres for reporting. Data gets exported or updated in the data-sources that best suit the need of that data, rather than trying to find a single data solution that fits every use of the data.
- jungturk 9y ago
  This (polyglot persistence in Fowler-speak) is typical in situations when you have access patterns that aren't well-solved by a single data store. It will likely introduce some serious consistency issues that need to be attended to, but that pain might be better than the alternatives.
tdurden 9y ago
The author will hopefully look back on this post and cringe a little. All the requirements point to a relational database, but that is not realized quickly because "if I need to use a database, I go straight to NoSQL"

116 points by orf 9y ago

> Assuming you know how your data needs to be queried when you create your table, then the hash/range key combination could be the answer to your query problems. However we did not. DynamoDB's purpose is to allow you to easily scale a known query pattern. I definitely don't recommend it as a first solution when you don't know how the data is going to be accessed. Or when everything fits on one PostgreSQL box. We've had a lot of success starting out with a big PostgreSQL box and migrating use cases to DynamoDB when a table gets too big and has a query pattern that fits (essentially K/V lookup with some simple indexes).

gamesbrainiac 9y ago
Good news is that postgres is going to get multi-master replication soon in postgres 10, so a lot of the painpoints when it comes to scaling our writes, are hopefully going to be solved.
- bradhe 9y ago
  > are hopefully going to be solved. Right, because multi-master replication schemes have such a long lineage of being operationally simple. /s
  - paladin314159 9y ago
    Yeah, that's the real power of DynamoDB, despite all of its shortcomings. It's operationally simple (given you have the money).
- crypto5 9y ago
  > Good news is that postgres is going to get multi-master replication soon in postgres 10 Where did you read this?..
  - anarazel 9y ago
    I think the poster of that claim is confusing "plain" logical replication and multi-master replication. There's a lot of infrastructure in common between both, but the conflict resolution part of async mm didn't make it it into postgres 10...
- b777 9y ago
  How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all servers? And if all writes need to happen on all servers, how does that improve write scalability as the load write load on each server is the same as it would be without multi-master replication.
  - anarazel 9y ago
    > How does multi-master replication solve write scalability issues? I'm not familiar with the postgres implementation, but don't all writes have to happen on all servers? It doesn't generally. You often can increase the total throughput because independent transactions can be batch committed when replicated, which can increase throughput noticeably, but that's more like a constant factor improvement than a real scalability improvement. What it does however often allow you to do is to: a) Address latency issues with writers distributed across the globe. If you have masters on each continent your total write rate can often be massively higher than if you have application servers from each of those continents write to a centralized database. b) Partially replicate your data globally, partially not. Quite often you'll want to have some data available everywhere (typically metadata, security related data, etc), but a lot of the bulk data only available in certain places.
    - darkr 9y ago
      > a) Address latency issues with writers distributed across the globe How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Same with incrementing IDs etc.
      - anarazel 9y ago
        > > a) Address latency issues with writers distributed across the globe > How? Surely to provide ACID compliance all writers need to co-ordinate a lock if they want to update a record. Well, there's different systems out there. What you want for something like this is asynchronous multi-master. Which, as you observe, allows for conflicts to occur, which need to be resolved somehow. > Same with incrementing IDs etc. The solution for those is usually to have each node assign sequences with different start values, have a system that distributes "chunks" of values (that was what BDR, an async MM version of postgres used to do), or avoiding sequences in favor of UUIDs or similar.
- m00x 9y ago
  Wouldn't multi-master replication be bad for write throughput? Depending on the scheme, it will have to be committed to all machines for every write, therefore increasing write operation complexity. If you want to scale up your writes, sharding would be a better solution.
- manigandham 9y ago
  > postgres is going to get multi-master replication soon in postgres 10 No it's not. Postgres barely has decent replication as is. Logical replication is a good step beyond (while still missing major functionality) but nowhere near what it takes for multi-master replication.
  - anarazel 9y ago
    > No it's not. Postgres barely has decent replication as is. You're overly negative whenever postgres comes up on HN. Obviously the v10 stuff is not multi-master and the project definitely is not claiming it has, but saying that Postgres' replication "barely has decent replication as is" is a bit absurd given that it works quite well in a lot of high-throughput scenarios in a lot of companies, and has so for 5+ years. There's a lot of valid complaints about it you can make (especially around flexibility and operational issues), but it's definitely not just "barely decent". I'm not saying that you have to be fan. It's fine to say the replication isn't good enough for everything, or just ignore postgres. > but nowhere near what it takes for multi-master replication. FWIW, there's async MM built on top of the relevant pieces, and it's used in production. It's not ready for wide audiences, but arguably (and I think one can very validly disagree about that) it's closer than "nowhere near".
    - anarazel 9y ago
      https://news.ycombinator.com/item?id=14578213 https://news.ycombinator.com/item?id=14578213 https://news.ycombinator.com/item?id=14373424 https://news.ycombinator.com/item?id=14373424 https://news.ycombinator.com/item?id=13653708 https://news.ycombinator.com/item?id=13653708 https://news.ycombinator.com/item?id=13423951 https://news.ycombinator.com/item?id=13423951 ...
    - manigandham 9y ago
      What is the point of this comment? And then linking to my past comments? Is it wrong to have perspective and criticism or to express it multiple times? I get that you're a committer but "overly negative" is as much your opinion as what I say, and my comments are the truth as I see it. I've even spent an hour talking to Ozgun about databases and moving to Citus before so my statements come from being a user, operator and maintainer of dozens of different database technologies in production for years, including postgres right now, and from all that experience it is definitely the most behind when it comes to replication/HA compared to anything else I use. All of those threads have plenty of upvotes and other user comments so it's clear I don't stand alone here. Perhaps instead of policing my statements, time would be better spent on improving the features and experience of the product.

aluskuiuc 9y ago

I'm surprised the author didn't consider using DynamoDB Streams to replicate to a PGSQL database for the relational queries while keeping the primary key-value lookups on DDB.

manigandham 9y ago
What does that gain other than more expense and overhead? SQL can do key/value just fine.

williamstein 9y ago

This closely mirrors my experience a few months ago: https://news.ycombinator.com/item?id=12649712 https://news.ycombinator.com/item?id=12649712

linkmotif 9y ago

> I’m not a database guy, I’m a node guy. Is this really a distinction your "scrappy startup type" can afford to make? > Although since version 3.4.x Mongo passes all Jepsen tests, when our original database was chosen the current version of Mongo at the time did not and was deemed unreliable by the team Does not pass all Jepsen tests => deeemed unreliable :( Wow. Of all possible reasons, THAT'S the reason MongoDB wasn't chosen? Jepsen, man. Pretty sad so many people just look at Jepsen results like this. Jepsen, really, is about the tests more than the results. It highlights various failure modes that may be difficult or uncommon to replicate in production. These failure modes are important to know about to understand the product you are using, but are not really the end-all when it comes to deciding whether that product is worth using! Anyway, show me a database that passes Jepsen and I'll show you a database that only accepts writes to a single node.

andreimatei1 9y ago
> Anyway, show me a database that passes Jepsen and I'll show you a database that only accepts writes to a single node. I know one - CockroachDB :). Relational, strongly-consistent, SQL, elastic, multi-active replication - the dream! (I work on it) https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-jepsen-testing/ https://www.cockroachlabs.com/blog/cockroachdb-beta-passes-j... http://jepsen.io/analyses/cockroachdb-beta-20160829 http://jepsen.io/analyses/cockroachdb-beta-20160829
- hmottestad 9y ago
  RethinkDB also passes jepsen.
  - linkmotif 9y ago
    It's also defunct so... :)
    - jjuel 9y ago
      Open Sourced is definitely different than defunct...

neovintage 9y ago

The really important part of this post is the section articulating why not Dynamo. It uncovers a big question I ask of all of my customers when they're trying to choose data stores at any point during the life of their application. Do you know your query patterns for your application? If you know how your application / product need to pull data from the data store, you way more information that will allow you to pick the right data store to fit the problem at hand. When you don't have any prior query patterns to go off of, picking a relational database to start your application is a good choice. You need the flexibility to allow your schema to change over time. The schema and query patterns will eventually reach a point where changes don't happen all that often and then you can look to other datastores if you need to scale. Plus, are you really going to have THAT much data when you start? Disclosure: I work on Heroku Postgres. Could be a little biased....

lilbobbytables 9y ago
Similar to the article author, many just don't have experience with relational db's and have only heard that they aren't flexible and don't scale. They don't realize just how easily performant Postgres can be - and often so easily by sticking basic indexes where they are needed. Similarly they may not realize the amount of data and load PG or MySQL can handle just fine.

orf 9y ago

I submitted this article because of a single paragraph: > When it came to the different types of NoSQL databases, it was pretty clear that a document store best suited our needs. We needed to be able to query based on the data so key/value was out, there was no need to have the ability to make all of the complex connections between data of a graph db, and nobody actually knows what a column db is (jk but our data is not the right use case for it). So document it was. We would have to change our data layout a bit to really work well with most document databases, our current data structure was similar to what you would expect for a relational db with tables etc since dynamo also uses a table structure. I wanted something that was ACID, none of that eventual consistency nonsense. That took a lot of them out of the race right there. I don't get it. He needed arbitrary querying, relations, ACID, tables. And yet... his first thought was to go to a document-based no-sql database and is only later hit with the sudden realization that his relational data is bet suited in a relational database. What has gone so wrong here that an otherwise smart and educated engineer makes these kinds of fundamental mistakes? Is it the success of no-sql marketing departments, and the apparent un-sexiness of traditional relational databases? Is it education? The job market? A fear of SQL (that you never really have to write anyway...)? Is it the quick-fix non-relational databases give you? As far as I can see there are very very very few nails that a PostgreSQL hammer cannot hit. I guess that's part of the point of the article, but why does it need an article to explain this? How can we stop these kinds of articles from needing to be written? > The change drastically increased the speed of our API, some requests are up to 6x faster than they were with Dynamo. The account details request, our previously slowest call, took around 12 seconds with Dynamo and was reduced to 2 seconds with Postgres! sigh Edit: Not sure why this has been moved to the bottom of the thread. I wasn't trying to single this particular developer out, I was merely commenting on this troubling trend. Apologies if this came across in the wrong way.

misterbowfinger 9y ago
When I hear "brogrammer", I'm usually a little put-off because it's kind of a troll term. But this article, wow. You can call me on it - brogrammers are ruining the industry.
- 9y ago

hesdeadjim 9y ago

DynamoDB was a constant source of anxiety for me and my team. We fit the usage pattern perfectly, so the author's points didn't apply to us. Our primary problem was having to constantly manage and change our provisioned capacity so that we could respond to usage spikes and keep our bill from becoming stratospheric. Worse, anytime we'd add new functionality -- such as nighttime analytics jobs that scanned the table or daily backups, it meant revisiting the parameters for our auto-scaling or risk getting throttled and having our application randomly start spewing errors. DynamoDB's billing and provisioning model is awful to deal with. When your team is having serious conversations about the repercussions of your data values going from 4kb to 8kb, you very quickly begin questioning your decision to use DynamoDB in the first place.

eropple 9y ago
The high score I ever saw at a company I worked for was about $85K in DynamoDB spend. That's an eight, a five, and three zeroes after it. It was a feature so otherwise silly and simple that I don't want to flame the developers thereof by describing it, but it might have been the most gratuitous burning of cash I've ever seen at a startup.
- hesdeadjim 9y ago
  Ha, yea I've heard of numbers like that. I remember talking to someone at re:Invent who described their high five figures monthly bill in almost reverent terms -- it made sense after he described their 200+ MySQL server catastrophe DynamoDB replaced. We actually ended up engineering our system to use a variant of event sourcing. This let us dump DynamoDB and use RedisLabs and S3 in such a way that we were able to handle 1mil DAU with a 10:1 write/read ratio for around $5k/mo -- including front end/backend instances.
  - kornish 9y ago
    > This let us dump DynamoDB and use RedisLabs and S3 in such a way that we were able to handle 1mil DAU with a 10:1 write/read ratio This sounds really interesting – do you have a writeup of the system architecture anywhere?

idbehold 9y ago

I'd also like to remind anyone using DynamoDB: it cannot store empty strings. Let that sink in for a second. If you think you're going to be able to store JSON documents you'll be in for a nasty surprise when you try to insert a valid JSON document and either the request fails or it converts any empty strings in your JSON document into nulls.

koolba 9y ago
An empty string (zero bytes) isn't valid JSON. '""' vs '' (single quotes to illustrate the wrapper for the total content).
- idbehold 9y ago
  {"foo":""} That's valid JSON that cannot be stored in DynamoDB because the value of "foo" is an empty string. See previous discussion: https://news.ycombinator.com/item?id=13170746 https://news.ycombinator.com/item?id=13170746
  - koolba 9y ago
    Ah I didn't notice the distinction between storing the serialzied form (which would be non empty) and storing the record itself. Yes that does suck quite a bit and seems like a pretty bad decision.

mark242 9y ago

Dismissing CouchDB because it only supports pre-sharding is extremely shortsighted. It takes very little time to replicate from one db with 2 shards to another db with 3 shards, even on N nodes. Copying the *.couch files is the absolute wrong way to manage this especially in a production environment when those files will be constantly changing. It is far, far easier to add nodes to a Cloudant/Couch2 cluster, set up realtime replication from db1 -> db2, then once the replication has caught up, at your leisure you switch your database connection in your app.

dchuk 9y ago

It doesn't really seem like this guy was qualified to make such a pivotal decision like choosing the database your entire system will run on given his self-submitted breadth of experience in the opening paragraphs...

takeda 9y ago
You could say I'm a Postgres fanboy, but even to me he did not sound convincing. The guy did not seem to know what he was talking about. Almost felt like he picked Postgres by throwing darts at possible options. Yes, he picked the right choice :), but I feel like he will misuse it[1] and few months later write article how bad it was. [1] Heh, I've seen Postgres tables that had 4 columns: - id - date added - date updated - thrift encoded blob containing the data (that was before JSONB, but even then the data was highly relational and that was silly)

luord 9y ago

My own experience working with Postgres, discovering nifty features or trying out new ones, and articles like this make it clear for me that I'm unlikely to ever need anything else. By the time any of the applications I work on could grow to the size in which Postgres wouldn't be a good option, I'd hopefully be in the process of selling it anyway. I'm quite ok with that. It's such an awesome piece of software.

pedalpete 9y ago

I've just started using DynamoDB on a project. Before I made the choice, I discussed some architecture issues with a friend who works at a big fast growing unicorn start-up. They use MongoDB for the majority of the app the user sees, they use ElasticSearch for search, postgres for reporting. Data gets exported or updated in the data-sources that best suit the need of that data, rather than trying to find a single data solution that fits every use of the data.

jungturk 9y ago
This (polyglot persistence in Fowler-speak) is typical in situations when you have access patterns that aren't well-solved by a single data store. It will likely introduce some serious consistency issues that need to be attended to, but that pain might be better than the alternatives.

tdurden 9y ago

The author will hopefully look back on this post and cringe a little. All the requirements point to a relational database, but that is not realized quickly because "if I need to use a database, I go straight to NoSQL"