3 ms·

And I don't understand the obsession with server-based databases for single apps. Especially in containerised setups, every "app" gets its own database anyways,

by franga2000 5mo ago

And I don't understand the obsession with server-based databases for single apps. Especially in containerised setups, every "app" gets its own database anyways, and if the app is further broken down into services, they usually communicate between each other and not with a shared database. So in those cases, what do you gain by pulling the database out of the "process" and onto the other end of a socket? In most cases, absolutely nothing. So why bother? Don't get me wrong, I've worked with plenty of server-based databases, including proper dedicated database servers. It's great tech and often the best tool for the job. But not always and I'd argue not in the majority of uses.

stingraycharles 5mo ago
“Especially in containerised setups, every "app" gets its own database anyways, and if the app is further broken down into services, they usually communicate between each other and not with a shared database. “ You seem to be talking about a vastly different use case. Containerized apps having their own database? What? Aren’t these types of containers stateless? I always very much try to keep state out of app containers. What kind of data storage are we talking about?
- franga2000 5mo ago
  If an app needs a database, it gets a database server container, instead of getting a user and database on a shared database server as things used to be done. Every little django app has its own postgres container. Every wordpress site gets its own mysql container. That is the modern way. Those database containers get a PVC/volume/mount for their data dirs. The only thing ever connecting to them is their "owner" application container. So at that point, why not drop the postgres container and PVC mount a sqlite directory in the app container? The result is the same.
  - mxey 5mo ago
    Yes if you run a database server like an embedded application database, then it won’t be very different from an embedded application database.
  - jappgar 5mo ago
    And when you need to scale to thousands of instances of your microservice?
    - stingraycharles 5mo ago
      Yeah this is the part I don’t get. It seems like people are talking about 1 distinct app = 1 container and this is the new normal? We’re back to managing cows instead of cattle again?
      - jappgar 5mo ago
        I just think a lot of people here haven't ever worked on large scale systems. They don't know what the don't know.
        andersmurphy 5mo ago
        I think a lot businesses build large distributed systems prematurely. They don't know what they don't know.
    - graerg 5mo ago
      That's the whole thesis; YAGNI.
    - andersmurphy 5mo ago
      What on earth needs thousands of instances? Are you building a CDN? Most apps can be a single server or sharded by business/region. Honestly, this whole leys run loads of nodes seems to have sprung up from languages that are slow oe don't have decent concurrency.
HDThoreaun 5mo ago
Every container gets its own database?
- franga2000 5mo ago
  Yes? Well, every "app", as I quite explicity wrote. Look up the docker compose file or helm chart for basically any app. I'm running dozens of apps, each with their own postgres, redis and nginx containers alongside the main application server. That's what the stack is designed for.
  - mxey 5mo ago
    The Compose file is written like that so you can quickly try the app without setting up extra dependencies. Usually not for production use. Especially since in production you might want to scale the parts separately. I like to have a Postgres cluster to connect where backup is already handled, and the app then doesn’t have any persistent data, doesn’t need any network volume mounts.
cortesoft 5mo ago
How do you do server maintenance or handle hardware failure if your database is SQLite? You are going to have to take downtime, even in the best scenario.
- franga2000 5mo ago
  1. Proxmox live migration or HA, Ceph storage 2. K8S DaemonSet, PVC backed by probably Ceph 3. Just..don't care? Do maintenance outside of working hours, fix issues quickly and explain things nicely to your customers. Not everything is google-scale. Most people can deal with some downtime. And it's not like you won't have downtime in let's say a postgres-backed app. But now you have two "servers" to deal with.
  - cortesoft 5mo ago
    Those first two options add more complexity than just using an external database. I guess I am just not used to downtime being acceptable. Spent most of my career working for a CDN, any sort of downtime was simply unacceptable. I can't stop myself from that sort of thinking now.
    - franga2000 5mo ago
      It's complexity you already have. You need some sort of HA for your app server and some sort of resilient storage for your database server. Using sqlite just means the storage is used by the app server directly, nothing more.

stingraycharles 5mo ago

“Especially in containerised setups, every "app" gets its own database anyways, and if the app is further broken down into services, they usually communicate between each other and not with a shared database. “ You seem to be talking about a vastly different use case. Containerized apps having their own database? What? Aren’t these types of containers stateless? I always very much try to keep state out of app containers. What kind of data storage are we talking about?

franga2000 5mo ago
If an app needs a database, it gets a database server container, instead of getting a user and database on a shared database server as things used to be done. Every little django app has its own postgres container. Every wordpress site gets its own mysql container. That is the modern way. Those database containers get a PVC/volume/mount for their data dirs. The only thing ever connecting to them is their "owner" application container. So at that point, why not drop the postgres container and PVC mount a sqlite directory in the app container? The result is the same.
- mxey 5mo ago
  Yes if you run a database server like an embedded application database, then it won’t be very different from an embedded application database.
- jappgar 5mo ago
  And when you need to scale to thousands of instances of your microservice?
  - stingraycharles 5mo ago
    Yeah this is the part I don’t get. It seems like people are talking about 1 distinct app = 1 container and this is the new normal? We’re back to managing cows instead of cattle again?
    - jappgar 5mo ago
      I just think a lot of people here haven't ever worked on large scale systems. They don't know what the don't know.
      - andersmurphy 5mo ago
        I think a lot businesses build large distributed systems prematurely. They don't know what they don't know.
  - graerg 5mo ago
    That's the whole thesis; YAGNI.
  - andersmurphy 5mo ago
    What on earth needs thousands of instances? Are you building a CDN? Most apps can be a single server or sharded by business/region. Honestly, this whole leys run loads of nodes seems to have sprung up from languages that are slow oe don't have decent concurrency.

HDThoreaun 5mo ago

Every container gets its own database?

franga2000 5mo ago
Yes? Well, every "app", as I quite explicity wrote. Look up the docker compose file or helm chart for basically any app. I'm running dozens of apps, each with their own postgres, redis and nginx containers alongside the main application server. That's what the stack is designed for.
- mxey 5mo ago
  The Compose file is written like that so you can quickly try the app without setting up extra dependencies. Usually not for production use. Especially since in production you might want to scale the parts separately. I like to have a Postgres cluster to connect where backup is already handled, and the app then doesn’t have any persistent data, doesn’t need any network volume mounts.

cortesoft 5mo ago

How do you do server maintenance or handle hardware failure if your database is SQLite? You are going to have to take downtime, even in the best scenario.

franga2000 5mo ago
1. Proxmox live migration or HA, Ceph storage 2. K8S DaemonSet, PVC backed by probably Ceph 3. Just..don't care? Do maintenance outside of working hours, fix issues quickly and explain things nicely to your customers. Not everything is google-scale. Most people can deal with some downtime. And it's not like you won't have downtime in let's say a postgres-backed app. But now you have two "servers" to deal with.
- cortesoft 5mo ago
  Those first two options add more complexity than just using an external database. I guess I am just not used to downtime being acceptable. Spent most of my career working for a CDN, any sort of downtime was simply unacceptable. I can't stop myself from that sort of thinking now.
  - franga2000 5mo ago
    It's complexity you already have. You need some sort of HA for your app server and some sort of resilient storage for your database server. Using sqlite just means the storage is used by the app server directly, nothing more.