4 ms·

RDS PostgreSQL is like the Hotel California, you can check in any time you want, but you can never leave. Maybe it is OK as a simple data store for a single app

RDS PostgreSQL is like the Hotel California, you can check in any time you want, but you can never leave. Maybe it is OK as a simple data store for a single app, but not for a real database. I gained a lot of my knowledge of PostgreSQL internals by helping my company get off of RDS and onto a dedicated EC2 instance solution. RDS imposes too many limitations. Also, your snapshot backup solution is trivial to implement on EC2 or anywhere else for that matter. But it is not easy to do it right in some scenarios. Read https://www.postgresql.org/docs/9.6/static/backup-file.html https://www.postgresql.org/docs/9.6/static/backup-file.html for details. LVM or ZFS are likely needed under the db layer.

RhodesianHunter 10y ago
"Maybe it is OK as a simple data store for a single app, but not for a real database." Currently working at company number 2 with large (many terabytes) databases on RDS and can safely say this is horse shit. The amount of time and energy it allows our engineers to spend on our actual products instead of database management is worth all of the extra cost and lock in and then some. Edit: I just realized that you were talking about Postgres on RDS in particular. I don't have experience with Postgres so you may well be right.
- _Codemonkeyism 10y ago
  I'm interested in a. How many hours would you guess you are saving a month? b. What takes by a factor of 10 less time on RDS than doing it by hand? What task sees the largest time saved? Because I always wonder when reading this, what am I missing? What haven't we done? Were we lucky? We were running MySQL and Postgres for multi hundred million EUR companies with millions of users and we did not spend a lot of effort into managing them.
  - RhodesianHunter 10y ago
    1. Zero effort backups. 2. Zero effort re-deployment from backups. 3. Almost zero effort encryption at rest. 4. Zero effort hot backups, automatic fail-overs, and multiple datacenter deployments 5. Low effort migrations of massive amounts of data between DBs when someone inevitably wants to refactor something 6. Zero effort logging and log aggregation 7. Almost zero effort alerting of issues via sms/email/other I could go on but I'm on my way to work... When you're paying engineers north of 150K all of this adds up, and I'd much rather throw the money at Amazon to handle this and pay the engineers to focus on our actual product.
- pgaddict 10y ago
  We're in the business of PostgreSQL support, and some of our customers use RDS for various reasons. Not having to care about the deployment is one of the usual goals of using a managed environment, but considering they subsequently go and buy support from a third party might be a sign of something. Of course, my view is biased because we only hear about the issues - there might be a 100x more people using RDS without any issues, and we never hear about them. In general, the pattern we see is that people start using RDS, and they're fairly happy because it allows them to build their product and RDS more or less works. Then they grow a bit over time, and something breaks. Which brings me to two main RDS issues that we run into - lack of insight, and difficulty when migrating from RDS with minimum downtime. Once in a while we run into an issue where we wish we could connect over SSH and get some data from the OS, attach gdb to the backend, or something like that. Or using auto_explain. Not even mentioning custom C extensions that we use from time to time ...
  - RhodesianHunter 10y ago
    "and difficulty when migrating from RDS with minimum downtime" They're simply uninformed then. AWS database migration service makes zero downtime migrations trivial between just about any major databases (mysql, oracle, postgres, aurora, sqlserver, etc.)
atmosx 10y ago
Could you describe the limitations you encounter in the RDS PSQL setup vs a self-managed?

RhodesianHunter 10y ago

"Maybe it is OK as a simple data store for a single app, but not for a real database." Currently working at company number 2 with large (many terabytes) databases on RDS and can safely say this is horse shit. The amount of time and energy it allows our engineers to spend on our actual products instead of database management is worth all of the extra cost and lock in and then some. Edit: I just realized that you were talking about Postgres on RDS in particular. I don't have experience with Postgres so you may well be right.

_Codemonkeyism 10y ago
I'm interested in a. How many hours would you guess you are saving a month? b. What takes by a factor of 10 less time on RDS than doing it by hand? What task sees the largest time saved? Because I always wonder when reading this, what am I missing? What haven't we done? Were we lucky? We were running MySQL and Postgres for multi hundred million EUR companies with millions of users and we did not spend a lot of effort into managing them.
- RhodesianHunter 10y ago
  1. Zero effort backups. 2. Zero effort re-deployment from backups. 3. Almost zero effort encryption at rest. 4. Zero effort hot backups, automatic fail-overs, and multiple datacenter deployments 5. Low effort migrations of massive amounts of data between DBs when someone inevitably wants to refactor something 6. Zero effort logging and log aggregation 7. Almost zero effort alerting of issues via sms/email/other I could go on but I'm on my way to work... When you're paying engineers north of 150K all of this adds up, and I'd much rather throw the money at Amazon to handle this and pay the engineers to focus on our actual product.
pgaddict 10y ago
We're in the business of PostgreSQL support, and some of our customers use RDS for various reasons. Not having to care about the deployment is one of the usual goals of using a managed environment, but considering they subsequently go and buy support from a third party might be a sign of something. Of course, my view is biased because we only hear about the issues - there might be a 100x more people using RDS without any issues, and we never hear about them. In general, the pattern we see is that people start using RDS, and they're fairly happy because it allows them to build their product and RDS more or less works. Then they grow a bit over time, and something breaks. Which brings me to two main RDS issues that we run into - lack of insight, and difficulty when migrating from RDS with minimum downtime. Once in a while we run into an issue where we wish we could connect over SSH and get some data from the OS, attach gdb to the backend, or something like that. Or using auto_explain. Not even mentioning custom C extensions that we use from time to time ...
- RhodesianHunter 10y ago
  "and difficulty when migrating from RDS with minimum downtime" They're simply uninformed then. AWS database migration service makes zero downtime migrations trivial between just about any major databases (mysql, oracle, postgres, aurora, sqlserver, etc.)

atmosx 10y ago

Could you describe the limitations you encounter in the RDS PSQL setup vs a self-managed?