Our DBs are tiny for most users. We run s3 (minio rook ceph on k8s) locally so the network latency is a cluster latency. I figure you could just throw hardware

Our DBs are tiny for most users. We run s3 (minio rook ceph on k8s) locally so the network latency is a cluster latency. I figure you could just throw hardware at it like you mentioned. Move them to nvme backed S3 if needed. And our use case is only ever load, do a read or write, and then save. So they DBs aren’t open for very long. And with S3 compression you could save on download time but pay a decompress cost. This approach has its downsides don’t get me wrong but it scales nicely but forget running aggregates across the databases at least not for a real-time result.