4 ms·

It's easy to install k8s curl a helm chart too, doesn't mean you should. What you're missing is complexity and second order effects. Making decisions about da

It's easy to install k8s curl a helm chart too, doesn't mean you should. What you're missing is complexity and second order effects. Making decisions about databases early results in needing to make a lot of secondary decisions about all sorts of things far earlier than you would if you just left it all in sqlite on one VM. It's not for everyone. People have varying levels of experience and needs. I'll almost always setup a MySQL db but I'd be lying if I said it never resulted in navel gazing and premature db/application design I didn't need.

hurril 4y ago
That's a cop-out answer. You're making the claim that simply using the filesystem compared to going with Postgres somehow has less complexity and fewer (or lesser) second order effects, but you don't even indicate how come. So here's my question: how come?
- geocar 4y ago
  > simply using the filesystem compared to going with Postgres somehow has less complexity Postgres is also using the file system, it’s just got some extra layers in-front like networking and authentication. People often refer to those “extra layers” as complexity. > how come? Why is it that more code is more stuff that can go wrong? That’s just how it is.
  - hurril 4y ago
    You are also missing the point and in the same way. You are at A, you want to go to C. We are debating which B to pick between: 1. a filesystem + implementing library over that to cover the interface between A and C. 2. Postgres (which is the same as above - just not invented here) Now, both of you are imagining a world where #2 somehow implies more complexity and also more second order effects than #1 would. Nobody uses all of Postgres, but the cost of not using more than 1% of what's available is hardly larger than implementing that 1% yourself for most use cases. And please don't make the misstake of thinking that I'm trying to sell Postgres over any other database product - this is all to do with the worse is better idea that so many of you cultivate.
    - geocar 4y ago
      > We are debating which B to pick between: No we are not. Postgres isn't a webserver that can handle 1m http GET requests per second; it doesn't distribute data from 30 event collectors to a reporting database without configuration. *Other stuff needs to be written to satisfy the business.* That's life. > both of you are imagining a world where #2 somehow implies more complexity and also more second order effects than #1 would. Most studies into product resolution agree that there's zero difference (from a pass/fail perspective) between writing it yourself or trying to reuse third-party applications. Here's one from some quick ddging: https://standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf https://standishgroup.com/sample_research_files/CHAOSReport2... and see the resolution of all software products surveyed (page 6). That means that if you can, it is still cheaper to build new software and test it than to try to use Postgres and scale it, because once software is written the cost is (effectively) zero, whereas the cost to support Postgres is a body -- that I might only be able to use 1% of, and that I might need to hire two so that one can go on holiday sometime, with a real salary that we need to pay. Now maybe you're working for a company whose software never gets finished. In that case, by all means use Postgres in your application, since you can share the churn on that part with every other person using Postgres in the world, but I'm going kayaking today.
senttoschool 4y ago
>Making decisions about databases early results in needing to make a lot of secondary decisions about all sorts of things far earlier than you would if you just left it all in sqlite on one VM. Like what? What could be easier than going to AWS, click a few things to spin up a Postgres instance, click a few more things to scale automatically without downtime, create read replicas, have automatic backups, one-click restore? I feel like it's the opposite. Trying to put SQLite on a VM via Kubernetes will likely have a lot of secondary decisions that will make "scaling to millions" far harder and more expensive.
- gjulianm 4y ago
  > Like what? What could be easier than going to AWS, click a few things to spin up a Postgres instance, click a few more things to scale automatically without downtime, create read replicas, have automatic backups, one-click restore? Not do that and not have to maintain that? Just a regular AWS instance with SQLite will be far easier to setup, with regular filesystem backup which is easier to manage and restore. > Trying to put SQLite on a VM via Kubernetes will likely have a lot of secondary decisions that will make "scaling to millions" far harder and more expensive. The issue is assuming that "scaling to millions" is a design goal from the start. For a lot of projects, the best-case-scenario does not require it. For another big set of them, they will never get to it. For the few that do, it's possible that the cost of maintaining the more complex solution while it isn't needed plus the cost of adapting over time (because, let's be honest, it's almost impossible to get the scaling solution right from the start) is more than the cost of just implementing the complex solution when it's actually needed.
  - senttoschool 4y ago
    >Not do that and not have to maintain that? Just a regular AWS instance with SQLite will be far easier to setup, with regular filesystem backup which is easier to manage and restore. You'd have less maintenance using RDS than you would with an SQLite hosted on a VM. >The issue is assuming that "scaling to millions" is a design goal from the start. I was responding to the OP who said you can scale to "millions of revenue" on SQLite. This part is true. You could. But it'd be easier to do it on a managed Postgres instance.
ngc248 4y ago
Choosing a DB is not premature optimization, but it eliminates a whole host of data related problems and smoothens development.
- geocar 4y ago
  > Choosing a DB is not premature optimization That's your opinion. > it eliminates a whole host of data related problems and smoothens development I don't find this to be true, but maybe you have different problems than I do.

hurril 4y ago

That's a cop-out answer. You're making the claim that simply using the filesystem compared to going with Postgres somehow has less complexity and fewer (or lesser) second order effects, but you don't even indicate how come. So here's my question: how come?

geocar 4y ago
> simply using the filesystem compared to going with Postgres somehow has less complexity Postgres is also using the file system, it’s just got some extra layers in-front like networking and authentication. People often refer to those “extra layers” as complexity. > how come? Why is it that more code is more stuff that can go wrong? That’s just how it is.
- hurril 4y ago
  You are also missing the point and in the same way. You are at A, you want to go to C. We are debating which B to pick between: 1. a filesystem + implementing library over that to cover the interface between A and C. 2. Postgres (which is the same as above - just not invented here) Now, both of you are imagining a world where #2 somehow implies more complexity and also more second order effects than #1 would. Nobody uses all of Postgres, but the cost of not using more than 1% of what's available is hardly larger than implementing that 1% yourself for most use cases. And please don't make the misstake of thinking that I'm trying to sell Postgres over any other database product - this is all to do with the worse is better idea that so many of you cultivate.
  - geocar 4y ago
    > We are debating which B to pick between: No we are not. Postgres isn't a webserver that can handle 1m http GET requests per second; it doesn't distribute data from 30 event collectors to a reporting database without configuration. *Other stuff needs to be written to satisfy the business.* That's life. > both of you are imagining a world where #2 somehow implies more complexity and also more second order effects than #1 would. Most studies into product resolution agree that there's zero difference (from a pass/fail perspective) between writing it yourself or trying to reuse third-party applications. Here's one from some quick ddging: https://standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf https://standishgroup.com/sample_research_files/CHAOSReport2... and see the resolution of all software products surveyed (page 6). That means that if you can, it is still cheaper to build new software and test it than to try to use Postgres and scale it, because once software is written the cost is (effectively) zero, whereas the cost to support Postgres is a body -- that I might only be able to use 1% of, and that I might need to hire two so that one can go on holiday sometime, with a real salary that we need to pay. Now maybe you're working for a company whose software never gets finished. In that case, by all means use Postgres in your application, since you can share the churn on that part with every other person using Postgres in the world, but I'm going kayaking today.

senttoschool 4y ago

>Making decisions about databases early results in needing to make a lot of secondary decisions about all sorts of things far earlier than you would if you just left it all in sqlite on one VM. Like what? What could be easier than going to AWS, click a few things to spin up a Postgres instance, click a few more things to scale automatically without downtime, create read replicas, have automatic backups, one-click restore? I feel like it's the opposite. Trying to put SQLite on a VM via Kubernetes will likely have a lot of secondary decisions that will make "scaling to millions" far harder and more expensive.

gjulianm 4y ago
> Like what? What could be easier than going to AWS, click a few things to spin up a Postgres instance, click a few more things to scale automatically without downtime, create read replicas, have automatic backups, one-click restore? Not do that and not have to maintain that? Just a regular AWS instance with SQLite will be far easier to setup, with regular filesystem backup which is easier to manage and restore. > Trying to put SQLite on a VM via Kubernetes will likely have a lot of secondary decisions that will make "scaling to millions" far harder and more expensive. The issue is assuming that "scaling to millions" is a design goal from the start. For a lot of projects, the best-case-scenario does not require it. For another big set of them, they will never get to it. For the few that do, it's possible that the cost of maintaining the more complex solution while it isn't needed plus the cost of adapting over time (because, let's be honest, it's almost impossible to get the scaling solution right from the start) is more than the cost of just implementing the complex solution when it's actually needed.
- senttoschool 4y ago
  >Not do that and not have to maintain that? Just a regular AWS instance with SQLite will be far easier to setup, with regular filesystem backup which is easier to manage and restore. You'd have less maintenance using RDS than you would with an SQLite hosted on a VM. >The issue is assuming that "scaling to millions" is a design goal from the start. I was responding to the OP who said you can scale to "millions of revenue" on SQLite. This part is true. You could. But it'd be easier to do it on a managed Postgres instance.

ngc248 4y ago

Choosing a DB is not premature optimization, but it eliminates a whole host of data related problems and smoothens development.

geocar 4y ago
> Choosing a DB is not premature optimization That's your opinion. > it eliminates a whole host of data related problems and smoothens development I don't find this to be true, but maybe you have different problems than I do.