3 ms·
> is that the unbelievable bloat in the complexity of our systems is going to bite us in the ass. My favorite example of this right now is Vitess. Sure, it's a
by falcolas 6y ago
> is that the unbelievable bloat in the complexity of our systems is going to bite us in the ass.
My favorite example of this right now is Vitess. Sure, it's a beautiful piece of technology. But, for a usecase my company is looking at, we'll be replacing one (exceptionally large) DB with in excess of 80 mysql pods, managed by another opaque-through-complexity system running on the top of kubernetes (which already bites us regularly even though it's "managed").
The complexity and failure scenarios makes my head ache, even though I should never have to interact with it myself.
Oh, and my current favorite PITA - having to change the API version of deployment objects from 'v1beta1' to 'v1' in over 160 microservice charts as part of a kubernetes version upgrade. Helm 2 doesn't recognize the deployments as being identical, so we're also have to do a helm3 upgrade as well, just to avoid taking down our entire ecosystem to do the API version upgrade. Wheeee!
- roman_sf 6y agogood thing kubernetes can restart pods fast and have persistent volumes!
- falcolas 6y agoPersistant volumes rely on NFS (or a flavor thereof), which is not great for database performance. But that's a moot point anyways, since Vitess doesn't use persistent volumes - it reloads the individual DBs from backups and binlogs when a pod is moved or restarted.
- roman_sf 6y agocouldn't they shovel it into dynamodb?
- snuxoll 6y ago> Persistant volumes rely on NFS (or a flavor thereof), which is not great for database performance. NFS is an option, but it’s not the only option. If you need locally attached storage you can use local PV’s which went GA in Kubernetes 1.14, or any of the plethora of volume plugins that exist for various network storage solutions.
- falcolas 6y agoNetwork storage, NFS or other, is never optimal for DB storage. It can and is used, but it’s never going to be as good as local storage. I had forgotten about local storage; it’s not something we can use in our environment. It’s a moot point, in either case. Vitess doesn’t rely on persistent storage, it relies on replicas and backups.
- dilyevsky 6y agoAre you saying that iscsi is a “flavor of nfs”? What makes it not suitable for good db performance?
- falcolas 6y agoI sincerely beg your forgiveness for forgetting about a feature that’s only about a year old, and a feature we can’t take advantage of in our own kubernetes environment, or with the Vitess cluster. Forest, trees.
- dilyevsky 6y agoYou should leave snark for topics you have some knowledge of. PVs based on network attached drives have been around from the very start. Their performance is quite good. Vitess can use them (https://github.com/vitessio/vitess/tree/master/helm/vitess#custom-pvc-for-mysql-data https://github.com/vitessio/vitess/tree/master/helm/vitess#c...)
- harpratap 6y ago> Oh, and my current favorite PITA - having to change the API version of deployment objects from 'v1beta1' to 'v1' in over 160 microservice charts as part of a kubernetes version upgrade. How is this a problem unique to Kubernetes? Don't you have to make similar changes when upgrading a library or dependency that was in beta?