4 ms·

Several reasons. We did some POC testing before we deployed this gear and knew that we would achieve high density. Honestly, I didn't expect this much. Hard

Several reasons. We did some POC testing before we deployed this gear and knew that we would achieve high density. Honestly, I didn't expect this much. Hard to believe that we've basically deployed every non-database app we have on this cluster and we're only 10% utilized. Every service gets run in two environments: test and prod. Both are co-located on the same Kube cluster in different namespaces. We also don't put any datastores in Kube. That stuff still lives in OpenStack for now. Ceph can make it possible but for fast disk I/O, it's tough to beat the local SAS bus.

snuxoll 10y ago
> Ceph can make it possible but for fast disk I/O, it's tough to beat the local SAS bus. Personal anecdote - we have everything on-prem using a pretty standard vSphere setup, I've got a couple PostgreSQL databases that aren't even that heavily used (I mean, they top out around 20-40 tx/sec) backed by a hybrid Tegile array. Randomly throughout the day my IOWAIT starts spiking because even over 8Gb fiber channel our storage latency starts spiking because everything else is on the same storage (well over 200 full VM's), or sometimes I need to run a table scan over a 40GB table for a one-off query and the storage bottlenecks the crap out of us because only our small active set is cached on the SSD's. You can run databases on remote storage, people obviously do it, but there's a reason why even on AWS the best practice is to use your instance disks instead of EBS volumes if you care about performance. Noisy neighbors suck.
- deleted 10y ago
  [deleted]
- shermanyo 10y ago
  I didn't consider latency spikes on slower resources like that, thanks for the examples.