4 ms·

Don't: - Manage your own DBs for production. I used to manage my own but now I use IaaS for it. It's worth the extra cost even if your budget is low since it l

by throwaway2016a 5y ago

Don't: - Manage your own DBs for production. I used to manage my own but now I use IaaS for it. It's worth the extra cost even if your budget is low since it lowers your risk and lets you move faster. AWS Aurora is awesome but expensive. There's a SaaS/IaaS out there for most DBs. And they handle the backups! - Use Kubernetes. k8s is awesome, I love it, but it is a beast and probably overkill for your solo ops team that is also multitasking. - Manually SSH / rsync / scp directly onto prod. Do: - Version control (Git) everything - Use containers where it makes sense (I use Docker when I'm not using Serverless). It keeps your dev and prod in sync and helps with immutable deployments (see later points) - Use Infrastructure as Code - Use Github actions to do your deploys - Use immutable deployments. More time to setup but makes rollbacks easier and if you never hot patch production you can't create a unique state that can't be restored. Finally, (and possibly controversially): use serverless where you can (I personally use AWS Lambda, S3, and Cloudfront). My personal flow: - All my infrastructure is in a parameterized CloudFormation template. - The template is checked into Git with the service code. - After CI runs the tests, CD builds my serverless functions and uploads the zip. - Then it runs the cloudformation template. The same flow also works for Docker except "uploads the zip" becomes "pushes the container" Using this setup I can tear down and build up my setup in minutes, making it easy (and cheaper) to separate dev and production.

dsgrillo 5y ago
> Don't: manage your own DBs for production. [...] Why, though? If you're really on a low budget, the extra $$ for a managed DB doesn't really pay for itself. I'd rather prefer spinning up a container running whatever DB I need and run daily backups to an external vendor. In my case, the website is free, living on ads revenue, so every penny counts.
- throwaway2016a 5y ago
  Good question and I have an answer for you. It depends on your risk tolerance and resiliency. If you compare it to a single machine with multiple tenants and no replication, it doesn't make sense. But that comes with risks and given the OPs question in the context of a k8s cluster (which for HA is a minimum of 5 servers), I figured they are prioritizing time savings over money. Losing customer data, downtime, and data breaches can kill some companies before they even take off. If your particular business is not sensitive to that than you can run your own database. Or better yet, if you're a content business you may not even need a database at all. But even so, if you run your own DB you need to worry about backups and such so it may even still be worth the money. And if you are going for High Availability (HA), running a properly replicated cluster can be time consuming. The cost of a t3.medium RDS instance on AWS for instance is $0.068 / hour compared to a t3.medium instance on EC2 is $0.0416 per hour. That is an extra $19 per month. For my use case, paying an extra $19 a month is well worth the savings in time and the piece of mind I have knowing my backups are running and if the server crashes I can recover quickly. But you must consider use cases and your's might be different.
  - dsgrillo 5y ago
    You bring up good points, fair enough. I might being naive here, still: - losing customer data: the backup must be done frequently enough. So I'm not risking losing data. - downtime: I'd expect that RDS could still go south as well, maybe less frequently, but still. In my case I start another VM, run again the container and apply the backup. - data breaches: Of course one can misconfigure something here, but not sure how it'd be different using a managed database. Regarding the cost, I know that there are companies spending hundreds of $$ on their infra, but it just baffles me for most use cases. I still feel that one can get really far with just a budget of 50$/month.
    - throwaway2016a 5y ago
      True. You have downtime risk with RDS and managed databases as well. I have taken down RDS instances with a bad query and indexes once or twice. $50 total would be tough with fully managed AND high availability. I agree. But to digress from the tech side a bit and talk my own experiences, the question I ask myself is: as a solo-prenuer can I use that time to be doing something that will make me more than $X per month? If the answer is "yes, immediately!" I go with the managed server. If the answer is, "it would take me years to break even"... I may roll my own. To address your points, thought: I think managed databases do help here. Data breaches and outages are usually configuration issues. Since they start with a good known configuration, managed services do help. Also, I see not being able to SSH in as a positive in this scenario. For example: Say you misconfigured a database an break backups (happens all the time even to good DBs). That is much harder to do on a managed database. If you're managing just one machine and being down for 10-30 minutes while your backup is being restored is fine, you probably don't gain much by using a managed service. In that case I'd take frequent backups, upload them immediately off the server (probably S3), and containerize the database. In my architectures I always run at least 3 servers. With most database engines, if you have three servers you need to lose 2 to cause an outage. But a 3 server cluster is much harder to manage manually, tilting the scales more towards a managed database. Also, to your point, would blow $50 out of the water. If I can, I use a managed serverless database like DynamoDB (there is also others) that are pay as you go. You can easily run a service on just DynamoDB free tier if your use case is well suited to that kind of key/value stored based DB. You can also, always start with your own and switch to managed later as you add machines to the cluster. Just, whichever you choose... but "especially" when rolling your own... make sure you run fire drills. I.e test your backup and restore process regularly.
sebiw 5y ago
> Using this setup I can tear down and build up my setup in minutes [...] Until you decide to move away from AWS. ;-) Nevermind, ignoring the potential lock-in there are many pros to this setup.
- throwaway2016a 5y ago
  I've been using 10+ years and have definitely considered that a lot over the years. It's a valid point. I used to be concerned with that, but now I think that is not a big issue for two reasons: 1. Vendor lockin when you are a startup should be a low priority in my opinion. Your priority should be moving fast and getting customers. 2. To expand on 1. It's not really vendor lockin, the only things vendor specific are your infrastructure templates and interface code. Rewriting the interface for another cloud provider takes a fraction of the time of rewriting your codebase (source: I've done it many times). To come full circle to the question, one thing you should probably not do as a solo-prenuer is concern yourself pre-maturely with vendor lockin at the expense of getting a working product. Edit/Example: The code for interfacing with AWS Aurora is the same as any other MySQL or PostgreSQL code except for authentication. Switching to a non-IaaS database WILL take time but no more time than rolling your own. So your choices are not "lock in" or "freedom".., it's, spend lots of time setting up databases now or later when you have proven your business model (and potentially never) Edit 2: This is an anecdote certainly but... of the dozens of startups I've worked on and with, some of which have grown pretty large. The only vendor lock-ins I've ever found to be issues have been with software frameworks (especially ERPs). Never infrastructure. I used to be worried about infrastructure lockin, now I prioritize product market fit.

dsgrillo 5y ago

> Don't: manage your own DBs for production. [...] Why, though? If you're really on a low budget, the extra $$ for a managed DB doesn't really pay for itself. I'd rather prefer spinning up a container running whatever DB I need and run daily backups to an external vendor. In my case, the website is free, living on ads revenue, so every penny counts.

throwaway2016a 5y ago
Good question and I have an answer for you. It depends on your risk tolerance and resiliency. If you compare it to a single machine with multiple tenants and no replication, it doesn't make sense. But that comes with risks and given the OPs question in the context of a k8s cluster (which for HA is a minimum of 5 servers), I figured they are prioritizing time savings over money. Losing customer data, downtime, and data breaches can kill some companies before they even take off. If your particular business is not sensitive to that than you can run your own database. Or better yet, if you're a content business you may not even need a database at all. But even so, if you run your own DB you need to worry about backups and such so it may even still be worth the money. And if you are going for High Availability (HA), running a properly replicated cluster can be time consuming. The cost of a t3.medium RDS instance on AWS for instance is $0.068 / hour compared to a t3.medium instance on EC2 is $0.0416 per hour. That is an extra $19 per month. For my use case, paying an extra $19 a month is well worth the savings in time and the piece of mind I have knowing my backups are running and if the server crashes I can recover quickly. But you must consider use cases and your's might be different.
- dsgrillo 5y ago
  You bring up good points, fair enough. I might being naive here, still: - losing customer data: the backup must be done frequently enough. So I'm not risking losing data. - downtime: I'd expect that RDS could still go south as well, maybe less frequently, but still. In my case I start another VM, run again the container and apply the backup. - data breaches: Of course one can misconfigure something here, but not sure how it'd be different using a managed database. Regarding the cost, I know that there are companies spending hundreds of $$ on their infra, but it just baffles me for most use cases. I still feel that one can get really far with just a budget of 50$/month.
  - throwaway2016a 5y ago
    True. You have downtime risk with RDS and managed databases as well. I have taken down RDS instances with a bad query and indexes once or twice. $50 total would be tough with fully managed AND high availability. I agree. But to digress from the tech side a bit and talk my own experiences, the question I ask myself is: as a solo-prenuer can I use that time to be doing something that will make me more than $X per month? If the answer is "yes, immediately!" I go with the managed server. If the answer is, "it would take me years to break even"... I may roll my own. To address your points, thought: I think managed databases do help here. Data breaches and outages are usually configuration issues. Since they start with a good known configuration, managed services do help. Also, I see not being able to SSH in as a positive in this scenario. For example: Say you misconfigured a database an break backups (happens all the time even to good DBs). That is much harder to do on a managed database. If you're managing just one machine and being down for 10-30 minutes while your backup is being restored is fine, you probably don't gain much by using a managed service. In that case I'd take frequent backups, upload them immediately off the server (probably S3), and containerize the database. In my architectures I always run at least 3 servers. With most database engines, if you have three servers you need to lose 2 to cause an outage. But a 3 server cluster is much harder to manage manually, tilting the scales more towards a managed database. Also, to your point, would blow $50 out of the water. If I can, I use a managed serverless database like DynamoDB (there is also others) that are pay as you go. You can easily run a service on just DynamoDB free tier if your use case is well suited to that kind of key/value stored based DB. You can also, always start with your own and switch to managed later as you add machines to the cluster. Just, whichever you choose... but "especially" when rolling your own... make sure you run fire drills. I.e test your backup and restore process regularly.

sebiw 5y ago

> Using this setup I can tear down and build up my setup in minutes [...] Until you decide to move away from AWS. ;-) Nevermind, ignoring the potential lock-in there are many pros to this setup.

throwaway2016a 5y ago
I've been using 10+ years and have definitely considered that a lot over the years. It's a valid point. I used to be concerned with that, but now I think that is not a big issue for two reasons: 1. Vendor lockin when you are a startup should be a low priority in my opinion. Your priority should be moving fast and getting customers. 2. To expand on 1. It's not really vendor lockin, the only things vendor specific are your infrastructure templates and interface code. Rewriting the interface for another cloud provider takes a fraction of the time of rewriting your codebase (source: I've done it many times). To come full circle to the question, one thing you should probably not do as a solo-prenuer is concern yourself pre-maturely with vendor lockin at the expense of getting a working product. Edit/Example: The code for interfacing with AWS Aurora is the same as any other MySQL or PostgreSQL code except for authentication. Switching to a non-IaaS database WILL take time but no more time than rolling your own. So your choices are not "lock in" or "freedom".., it's, spend lots of time setting up databases now or later when you have proven your business model (and potentially never) Edit 2: This is an anecdote certainly but... of the dozens of startups I've worked on and with, some of which have grown pretty large. The only vendor lock-ins I've ever found to be issues have been with software frameworks (especially ERPs). Never infrastructure. I used to be worried about infrastructure lockin, now I prioritize product market fit.