4 ms·

> Sure, maybe if you are some sort of SaaS with a need for a small single DB, that also needs to be resilient, backed up, rock solid bulletproof.. it makes sens

> Sure, maybe if you are some sort of SaaS with a need for a small single DB, that also needs to be resilient, backed up, rock solid bulletproof.. it makes sense? But how many cases are there of this? Most software startups these days? The blog post is about work done at a startup after all. By the time your db is big enough to cost an unreasonable amount on RDS, you’re likely a big enough team to have options. If you’re a small startup, saving a couple hundred bucks a month by self managing your database is rarely a good choice. There’re more valuable things to work on.

deleted 3y ago
[deleted]
tw04 3y ago
>By the time your db is big enough to cost an unreasonable amount on RDS, you’re likely a big enough team to have options. By the time your db is big enough to cost an unreasonable amount on RDS, you've likely got so much momentum that getting off is nearly impossible as you bleed cash. You can buy a used server and find colocation space and still be pennies on the dollar for even the smallest database. If you're doing more than prototyping, you're probably wasting money.
- theptip 3y ago
  That’s just another way of saying the opportunity cost isn’t worth paying to do the migration. Optionality and flexibility are extremely valuable, and that is why cloud compute continues to be popular, especially for rapidly/burstily growing businesses like startups.
  - latch 3y ago
    I don't mean to pick on your specific comments, but I find these analysis almost always lack a crucial perspective: level of knowledge. This is the single biggest factor, and it's the hardest one to be honest about. No one wants to say "RDS is a good choice . . . because I don't know how nor have I ever self managed a database." If you want a different opportunity cost, get people with different experience. If RDS is objectively expensive, objectively slow, but subjectively easy, change the subject.
    - pcl 3y ago
      > No one wants to say "RDS is a good choice . . . because I don't know how nor have I ever self managed a database." I don't think that's accurate. I've self-managed databases, and I still think that RDS is compelling for small engineering teams. There's a lot to get right when managing a database, and it's easy to screw something up. Perhaps none of the individual parts are super-complicated, but the cost of failure is high. Outsourcing that cost to AWS is pretty compelling. At a certain team size, you'll end up with a section of the team that's dedicated to these sorts of careful processes. But the first place these issues come up is with the database, and if you can put off that bit of organizational scaling until later, then that's a great path to choose.
    - maccard 3y ago
      I disagree here. This falls apart when you zoom out one step. I'm perfectly capable of managing a database. I'm also capable of maintaining load balancers, redis, container orchestrators, Jenkins, perforce, grafana, Loki, Oncall, individually. But each of those has the high chance of being a distraction from what our software actually does. Its about tradeoffs, and some tradeoffs are often more applicable than others - getting a ping at 7am on a Sunday because your ec2 instance filled it's drive up with logs and your log rotation script failed because it didn't have a long enough retey is a problem I'm happy to outsource when I should be focusing on the actual app.
    - theptip 3y ago
      Lack of expertise in some particular technology is simply another opportunity cost. I can learn how to operate a production DB at scale (I have racked servers and run other production workloads) but as cofounder/CTO in a startup is that the best use of my time? If the cost of a hosted DB is going to sink the company, then of course, I will figure it out and run it myself. But it’s not, for most startups. And therefore that knowledge isn’t providing much leverage. Starting an AI company with deep expertise in training models - that is an example of knowledge providing huge leverage. DB tech is not in this bucket for most businesses.
  - graemep 3y ago
    People do not really understand the value of the former. Even dealing with financial options (buy/sell and underlying) which are a pure form of it, people either do not understand the value, or do so in a very abstract way they do not intuit.
    - matwood 3y ago
      Good point. And, since you brought up financials, you also see this when people use a majority of their savings to lump sum pay off a mortgage. They take an overweighted view of saving on interest and, IMO, underweight the flexibility of liquidity.
  - graemep 3y ago
    On the other hand cloud platforms can be hard to migrate off, which is very much taking away options.
- macNchz 3y ago
  In the small SaaS startup case, I’d say the production database is typically the most critical single piece of infra, so self hosting is just not a compelling proposition unless you have a strong technical reason where having super powerful database hardware is important, or a team with multiple people who have sysadmin or DBA experience. I think both of those cases are unusual. I’ve been the guy managing a critical self-hosted database in a small team, and it’s such a distraction from focusing on the actual core product. To me, the cost of RDS covers tons of risks and time sinks: having to document the db server setup so I’m not the only one on the team who actually knows how to operate it, setting up monitoring, foolproof backups so I don’t need to worry that they’re silently failing because a volume is full and I misconfigured the monitoring, PITR for when someone ships a bad migration, one click HA so the database itself is very unlikely to wake me at 3am, blue/green deploys to make major version upgrades totally painless, never having to think about hardware failures or borked dist-upgrades, and so on. Each of those is ultimately either undifferentiated work to develop in-house RDS features that could have been better spent on product, or a risk of significant data loss, downtime, or firefighting. RDS looks like a pretty good deal, up to a point.
  - networkchad 3y ago
    [dead]
  - remus 3y ago
    I like fiddling with databases, but I totally agree with this. Unless you really need a big database and are going to save 100k+ per year by going self managed then RDS or similar just saves you so much stress. We've been using it for the best part of 10 years and uptime and latency have consistently been excellent, and functionality is all rock solid. I never have to think about it, which is just what I want from something so core to the business.
  - matwood 3y ago
    I am good at databases (have been a DBA in the past), and 100% agree with this. RDS is easy to standup and get all the things you mentioned, and not have to think about again. If we grow to the point where the overhead is more than a FT DBA, awesome. It means we are successful, and are fortunate to have options.
- crazygringo 3y ago
  > you've likely got so much momentum that getting off is nearly impossible as you bleed cash. Databases are not particularly difficult to migrate between machines. Of all the cloud services to migrate, they might actually be the easiest, since the databases don't have different API's that need to be rewritten for, and database replication is a well-established thing. Getting off is quite the opposite of nearly impossible.

deleted 3y ago

[deleted]

tw04 3y ago

>By the time your db is big enough to cost an unreasonable amount on RDS, you’re likely a big enough team to have options. By the time your db is big enough to cost an unreasonable amount on RDS, you've likely got so much momentum that getting off is nearly impossible as you bleed cash. You can buy a used server and find colocation space and still be pennies on the dollar for even the smallest database. If you're doing more than prototyping, you're probably wasting money.

theptip 3y ago
That’s just another way of saying the opportunity cost isn’t worth paying to do the migration. Optionality and flexibility are extremely valuable, and that is why cloud compute continues to be popular, especially for rapidly/burstily growing businesses like startups.
- latch 3y ago
  I don't mean to pick on your specific comments, but I find these analysis almost always lack a crucial perspective: level of knowledge. This is the single biggest factor, and it's the hardest one to be honest about. No one wants to say "RDS is a good choice . . . because I don't know how nor have I ever self managed a database." If you want a different opportunity cost, get people with different experience. If RDS is objectively expensive, objectively slow, but subjectively easy, change the subject.
  - pcl 3y ago
    > No one wants to say "RDS is a good choice . . . because I don't know how nor have I ever self managed a database." I don't think that's accurate. I've self-managed databases, and I still think that RDS is compelling for small engineering teams. There's a lot to get right when managing a database, and it's easy to screw something up. Perhaps none of the individual parts are super-complicated, but the cost of failure is high. Outsourcing that cost to AWS is pretty compelling. At a certain team size, you'll end up with a section of the team that's dedicated to these sorts of careful processes. But the first place these issues come up is with the database, and if you can put off that bit of organizational scaling until later, then that's a great path to choose.
  - maccard 3y ago
    I disagree here. This falls apart when you zoom out one step. I'm perfectly capable of managing a database. I'm also capable of maintaining load balancers, redis, container orchestrators, Jenkins, perforce, grafana, Loki, Oncall, individually. But each of those has the high chance of being a distraction from what our software actually does. Its about tradeoffs, and some tradeoffs are often more applicable than others - getting a ping at 7am on a Sunday because your ec2 instance filled it's drive up with logs and your log rotation script failed because it didn't have a long enough retey is a problem I'm happy to outsource when I should be focusing on the actual app.
  - theptip 3y ago
    Lack of expertise in some particular technology is simply another opportunity cost. I can learn how to operate a production DB at scale (I have racked servers and run other production workloads) but as cofounder/CTO in a startup is that the best use of my time? If the cost of a hosted DB is going to sink the company, then of course, I will figure it out and run it myself. But it’s not, for most startups. And therefore that knowledge isn’t providing much leverage. Starting an AI company with deep expertise in training models - that is an example of knowledge providing huge leverage. DB tech is not in this bucket for most businesses.
- graemep 3y ago
  People do not really understand the value of the former. Even dealing with financial options (buy/sell and underlying) which are a pure form of it, people either do not understand the value, or do so in a very abstract way they do not intuit.
  - matwood 3y ago
    Good point. And, since you brought up financials, you also see this when people use a majority of their savings to lump sum pay off a mortgage. They take an overweighted view of saving on interest and, IMO, underweight the flexibility of liquidity.
- graemep 3y ago
  On the other hand cloud platforms can be hard to migrate off, which is very much taking away options.
macNchz 3y ago
In the small SaaS startup case, I’d say the production database is typically the most critical single piece of infra, so self hosting is just not a compelling proposition unless you have a strong technical reason where having super powerful database hardware is important, or a team with multiple people who have sysadmin or DBA experience. I think both of those cases are unusual. I’ve been the guy managing a critical self-hosted database in a small team, and it’s such a distraction from focusing on the actual core product. To me, the cost of RDS covers tons of risks and time sinks: having to document the db server setup so I’m not the only one on the team who actually knows how to operate it, setting up monitoring, foolproof backups so I don’t need to worry that they’re silently failing because a volume is full and I misconfigured the monitoring, PITR for when someone ships a bad migration, one click HA so the database itself is very unlikely to wake me at 3am, blue/green deploys to make major version upgrades totally painless, never having to think about hardware failures or borked dist-upgrades, and so on. Each of those is ultimately either undifferentiated work to develop in-house RDS features that could have been better spent on product, or a risk of significant data loss, downtime, or firefighting. RDS looks like a pretty good deal, up to a point.
- networkchad 3y ago
  [dead]
- remus 3y ago
  I like fiddling with databases, but I totally agree with this. Unless you really need a big database and are going to save 100k+ per year by going self managed then RDS or similar just saves you so much stress. We've been using it for the best part of 10 years and uptime and latency have consistently been excellent, and functionality is all rock solid. I never have to think about it, which is just what I want from something so core to the business.
- matwood 3y ago
  I am good at databases (have been a DBA in the past), and 100% agree with this. RDS is easy to standup and get all the things you mentioned, and not have to think about again. If we grow to the point where the overhead is more than a FT DBA, awesome. It means we are successful, and are fortunate to have options.
crazygringo 3y ago
> you've likely got so much momentum that getting off is nearly impossible as you bleed cash. Databases are not particularly difficult to migrate between machines. Of all the cloud services to migrate, they might actually be the easiest, since the databases don't have different API's that need to be rewritten for, and database replication is a well-established thing. Getting off is quite the opposite of nearly impossible.