4 ms·

I'm calling out the difference between the storage mechanism (SSDs, Block Storage, etc...) - where transactions and journal-logs are dumped, so as to allow reco

I'm calling out the difference between the storage mechanism (SSDs, Block Storage, etc...) - where transactions and journal-logs are dumped, so as to allow recovery if the database server goes down. Once you design your database server/application/transactions such they don't care whatsoever if the database server goes offline - you not only get the ability to run fine in containers, you also make your application stack just significantly more robust for all sorts of reasons. I have colleagues who bemoan a major application server going down - which is a no-op in my world where the ephemeral nodes rarely last more than a day or two in GKE - so we're completely used to them going down. Building transactions, idempotency, tasks in rabbitmq/celery rather than memory - not only do they make deploying in containers straightforward, they also give an overall robustness win for all applications. I mean, physical servers go down as well.

kjeetgill 6y ago
Sorry, I'm not quite piecing together what scheme you're proposing. Like, where are you storing things like your user/password tables that aren't persistent databases? I'm familiar with the topics around HA and how you could ignore individual hosts going down, but the whole database?
- singingboyo 6y ago
  Not who you replied to, but: you're probably thinking Frontend->Backend->DB or similar, right? Frontend runs in browser and is either served by its own container or the backend, backend is in a containerized server, DB in something more persistent. Except, the reality is it's Frontend->Backend->DB->Storage. To some extent, the DB is just a standardized (and very quick) caching/sorting mechanism in front of an SSD, HDD, network drive, whatever. Storage can be separate and persistent, even if the DB is in a container and gets killed/dies. If that's the case, then even if the DB is in an ephemeral container the backing storage should always survive. With it, you can bring up a new DB container with all the same data being available. So yes, your passwords etc are in the "DB", but really they're in storage on disk. Yeah, your app won't work well if the DB disappears, but that's a given no matter whether your DB is in a container or somewhere else. Once it's back up, everything should work fine.
- ghshephard 6y ago
  When the database is down, no transactions occur. But as long as the storage medium (SSD, Spinning disks, PVC, whatever) - doesn't go down - then when the database comes back up - anything that needs to be rolled back is rolled back, and things just keep running along without any problem. So - the user will see a "Pause" when using the application if the database is down - but that's it. The great thing about architecting your application stack in such a way that every one of the servers/applications/databases can disappear in a moments notice, is that your application becomes super robust to hardware failure - because having a sever go "down" is Business As Usual in Kubernetes.

kjeetgill 6y ago

Sorry, I'm not quite piecing together what scheme you're proposing. Like, where are you storing things like your user/password tables that aren't persistent databases? I'm familiar with the topics around HA and how you could ignore individual hosts going down, but the whole database?

singingboyo 6y ago
Not who you replied to, but: you're probably thinking Frontend->Backend->DB or similar, right? Frontend runs in browser and is either served by its own container or the backend, backend is in a containerized server, DB in something more persistent. Except, the reality is it's Frontend->Backend->DB->Storage. To some extent, the DB is just a standardized (and very quick) caching/sorting mechanism in front of an SSD, HDD, network drive, whatever. Storage can be separate and persistent, even if the DB is in a container and gets killed/dies. If that's the case, then even if the DB is in an ephemeral container the backing storage should always survive. With it, you can bring up a new DB container with all the same data being available. So yes, your passwords etc are in the "DB", but really they're in storage on disk. Yeah, your app won't work well if the DB disappears, but that's a given no matter whether your DB is in a container or somewhere else. Once it's back up, everything should work fine.
ghshephard 6y ago
When the database is down, no transactions occur. But as long as the storage medium (SSD, Spinning disks, PVC, whatever) - doesn't go down - then when the database comes back up - anything that needs to be rolled back is rolled back, and things just keep running along without any problem. So - the user will see a "Pause" when using the application if the database is down - but that's it. The great thing about architecting your application stack in such a way that every one of the servers/applications/databases can disappear in a moments notice, is that your application becomes super robust to hardware failure - because having a sever go "down" is Business As Usual in Kubernetes.