5 ms·
When I was running the Postgres databases for reddit, I was constantly seeking people who could teach me better ways to tune Postgres on AWS. Most people I wou
by jedberg 14y ago
When I was running the Postgres databases for reddit, I was constantly seeking people who could teach me better ways to tune Postgres on AWS. Most people I would meet would just regurgitate what anyone could learn with a day or two of googling and reading.
At one point I got scared by the prospect that I might be an expert on Postgres on AWS, because frankly I didn't know all that much about it and thought we were doomed if that was really the case.
Then I went over had lunch with the Heroku team, and it was eye opening. These guys truly knew how to run Postgres on AWS (and presumably still do).
I can't think of another org that is moving Postgres on AWS forward better than Heroku.
- pvh 14y agoThanks a lot! We've come a long way since then - forks, followers, dataclips, extensions, new plans, new versions... Man, actually it kind of terrifies me to think of it all.
- ayanb 14y agoHey, do you have customer instances where the app is on non-aws bare metal ( like Softlayer) and the db is on heroku postgres? What are the pros and cons of this scenario?
- pvh 14y agoAbsolutely. Basically, as long as your app is running somewhere with Very Good Latency to your database, this can be a fine pattern. In practice, this means if the data center is also in Ashburn, Virginia, and better yet, has a DirectConnect deal set up with AWS, you should be in business. The best way to test this is to provision a database and see how latency from your app is. My opinion is that 5ms latency would be a ceiling on what I'd tolerate for my own apps, and I'd probably look for more like 1-2ms or better.
- judofyr 14y agoSo, is this information publicly available anywhere?
- pvh 14y agoWe give talks some times, and we're always happy to answer questions if you've got 'em. We open source stuff sometimes too, if we think it might be generally useful.
- wooster 14y agoAny insights you can share? I'm looking at this now, and it's not encouraging compared to dedicated hardware.
- sickpig 14y agoNot from heroku guys but definitely a worth reading http://thebuild.com/blog/2012/05/18/running-postgresql-on-aws/ http://thebuild.com/blog/2012/05/18/running-postgresql-on-aw...
- pvh 14y agoYou could always use Heroku Postgres. Our whole reason to exist is to ensure you can work fearlessly with your data. At this point, I think running your own data infrastructure is like having a generator in your garage instead of using the power grid.
- pestaa 14y agoIt sounds good until the power grid goes down and the only person in the street with the lights on is with the generator in his garage.
- bradgessler 14y agoThis is a bad metaphor. Utility computing isn't quite up to the same level as electrical utilities. Heroku has suffered from more downtime in the past year than I have power outages. Also, most commercial operations that tap into public utilities have some sort of backup plan for when the power goes out. A dramatic example of that is the power staying on in the Goldman Sacs building during Hurricane Sandy (http://www.inquisitr.com/381743/hurricane-sandy-rages-but-the-lights-stay-on-at-goldman-sachs/ http://www.inquisitr.com/381743/hurricane-sandy-rages-but-th...) More visibility would be greatly appreciated into what you're doing behind the scenes and increase your customers' confidence and expectations of using the Heroku pg cloud service. I hope you provide more depth to future answers as opposed to, "Just trust us". I've found that in practice, things never work out that way.
- monksy 14y agoThere is this book: http://www.amazon.com/PostgreSQL-High-Performance-Gregory-Smith/dp/184951030X http://www.amazon.com/PostgreSQL-High-Performance-Gregory-Sm... I haven't had a chance to finish it. But it does look like the author knows what he's talking about.
- saosebastiao 14y agoI don't doubt what you are saying, but I just wanted to point out one thing: Postgres is amazingly forgiving, and performs incredibly with just basic adjustments to the default configuration. Give it a good amount of memory, and it will handle the vast majority of workloads extremely well. I have been using Postgres basically out of the box for a few years now for Data Warehousing purposes, and its performance is comparable to the tuned-and-managed Oracle servers that I am pulling from. Reddit obviously is a different scenario, but most people don't work at Reddit.