3 ms·

Whatever you do, get your database off Heroku. Heroku's postgres offering has sweet options, but performance is fucking awful. We went from a 20k+ month herok

by ctide 14y ago

Whatever you do, get your database off Heroku. Heroku's postgres offering has sweet options, but performance is fucking awful. We went from a 20k+ month heroku bill to a fraction of that price on EC2 so we could get a slave that could keep up with the writes to our master. Performance being shit would be fine if it wasn't a miserable experience trying to migrate off. Not being able to setup replication to slaves off of Heroku means you have to deal with things like Bucardo and all the problems that brings along. Heroku is really awesome for little web apps that performance doesn't matter on. If you think you're building something that people will want to use, you don't want to be on Heroku.

gfodor 14y ago
Interesting -- can you go into more detail or provide references that you found helpful when trying to run Postgres on ec2?
- ctide 14y ago
  Yeah, if you run Ubuntu, don't try running any versions other than 11.04 if you're going to be under heavy i/o. We had all kinds of CPU soft lock issues with software raid on other versions. Otherwise? 2 m2.4xlarge running mostly standard postgresql installs absolutely dominate Mecha instances. Obviously, you need to deal with backups and that sort of thing, but, you can hire a full time person for the price difference between Heroku and EC2 as soon as you get to the top end of Heroku pricing.
fdr 14y ago
That's quite interesting; I don't see how one can really so soundly beat the disk configuration, but it certainly can be matched by something similar. It was posted about quite some time ago: http://orion.herokuapp.com/past/2009/7/29/io_performance_on_ebs/ http://orion.herokuapp.com/past/2009/7/29/io_performance_on_.... If this has become bad advice, inquiring minds wish to know. EBS Priops seem promising and different. Replication out of Heroku would be neat, but the WAL as-is is architecture and operating system dependent. Luck and tenaciousness willing, logical replication in Postgres may yet serve that use case, some day -- we'll have to see as to the details. A lot of effort has been expended attempting to get it in some form into 9.3, but it's far from sure thing. Also, don't forget to set up archiving, if you continue to retain fast interconnect with S3: https://github.com/wal-e/wal-e https://github.com/wal-e/wal-e, which I principally wrote on Heroku's behalf, but now is a project of multiple users and contributors. It has a focus on ease of use and reliability, although it could suffer improvement.
- ctide 14y ago
  Yeah, we're using wal-e in conjunction with streaming replication. Either of those options would have been awesome (anything but Bucardo, really. Bucardo's triggers overwhelmed an already overwhelmed database)
- zwily 14y ago
  "EBS Priops seem promising and different." We run many busy PG databases on PIOPS. The difference from normal EBS is huge. Very happy with PIOPS.

by ctide 14y ago

gfodor 14y ago

Interesting -- can you go into more detail or provide references that you found helpful when trying to run Postgres on ec2?

ctide 14y ago
Yeah, if you run Ubuntu, don't try running any versions other than 11.04 if you're going to be under heavy i/o. We had all kinds of CPU soft lock issues with software raid on other versions. Otherwise? 2 m2.4xlarge running mostly standard postgresql installs absolutely dominate Mecha instances. Obviously, you need to deal with backups and that sort of thing, but, you can hire a full time person for the price difference between Heroku and EC2 as soon as you get to the top end of Heroku pricing.

fdr 14y ago

That's quite interesting; I don't see how one can really so soundly beat the disk configuration, but it certainly can be matched by something similar. It was posted about quite some time ago: http://orion.herokuapp.com/past/2009/7/29/io_performance_on_ebs/ http://orion.herokuapp.com/past/2009/7/29/io_performance_on_.... If this has become bad advice, inquiring minds wish to know. EBS Priops seem promising and different. Replication out of Heroku would be neat, but the WAL as-is is architecture and operating system dependent. Luck and tenaciousness willing, logical replication in Postgres may yet serve that use case, some day -- we'll have to see as to the details. A lot of effort has been expended attempting to get it in some form into 9.3, but it's far from sure thing. Also, don't forget to set up archiving, if you continue to retain fast interconnect with S3: https://github.com/wal-e/wal-e https://github.com/wal-e/wal-e, which I principally wrote on Heroku's behalf, but now is a project of multiple users and contributors. It has a focus on ease of use and reliability, although it could suffer improvement.

ctide 14y ago
Yeah, we're using wal-e in conjunction with streaming replication. Either of those options would have been awesome (anything but Bucardo, really. Bucardo's triggers overwhelmed an already overwhelmed database)
zwily 14y ago
"EBS Priops seem promising and different." We run many busy PG databases on PIOPS. The difference from normal EBS is huge. Very happy with PIOPS.