4 ms·

Hopefully this comment doesn't sound too stupid but is it at all possible (or likely to be implemented) to get a worse network on GCE and pay less for it? I as

by Veratyr 11y ago

Hopefully this comment doesn't sound too stupid but is it at all possible (or likely to be implemented) to get a worse network on GCE and pay less for it? I ask because I don't need a CDN level network like GCE provides but I do need a ton of compute and a ton of bandwidth for batch data processing on data hosted at another provider (and not a candidate for GCS). A pre-emptible network offering would open up the way for me to use compute resources, which is surely a good thing.

boulos 11y ago
Not stupid at all! A cheaper, lesser networking offering is a common request, so you're not alone. Can you describe more about the "data hosted at another provider"? Would one of our interconnect options (https://cloud.google.com/interconnect/ https://cloud.google.com/interconnect/) help? If it's at AWS would a pair of interconnect options help (DirectConnect to Equinix in VA, then Carrier Interconnect to us).
- Veratyr 11y ago
  Just to say it upfront, I'm nothing like big enough for a lot of the options you have available. > Can you describe more about the "data hosted at another provider"? It's all at OVH and Hetzner. They offer very cheap storage and bandwidth. > interconnect I'm not big enough for those options and regardless, all of your peering/interconnect options are CDN priced for inter-region and only slightly better for intra-region. They just don't make sense. Say I want to store 100TB of data and pump it through some kind of processing pipeline, outputting the same amount of data. At OVH I pay ~$0.0075/GB on storage coming to ~$1.5k/month (since the processing results in double storage). Say my processing is light on memory and can run at 1MB/s/CPU and I want to be finished in a day. I need 1150 CPU cores for 24h and at OVH that would cost me ~$1300 and bandwidth is free and unlimited. At GCE I can run on pre-emptible instances and pay ~$284 for the compute but I have to pay for 100TB of premium egress bandwidth I don't need, which adds 100 * 1000 * 0.04 (generously assuming I can get peering/interconnect) which is $4000. GCE is absolutely unusable for jobs like this in its current state. Sure I could move data to GCS but in that case my $1.5k bill for storage turns into a $5.2k bill per month.

by Veratyr 11y ago

boulos 11y ago

Not stupid at all! A cheaper, lesser networking offering is a common request, so you're not alone. Can you describe more about the "data hosted at another provider"? Would one of our interconnect options (https://cloud.google.com/interconnect/ https://cloud.google.com/interconnect/) help? If it's at AWS would a pair of interconnect options help (DirectConnect to Equinix in VA, then Carrier Interconnect to us).

Veratyr 11y ago
Just to say it upfront, I'm nothing like big enough for a lot of the options you have available. > Can you describe more about the "data hosted at another provider"? It's all at OVH and Hetzner. They offer very cheap storage and bandwidth. > interconnect I'm not big enough for those options and regardless, all of your peering/interconnect options are CDN priced for inter-region and only slightly better for intra-region. They just don't make sense. Say I want to store 100TB of data and pump it through some kind of processing pipeline, outputting the same amount of data. At OVH I pay ~$0.0075/GB on storage coming to ~$1.5k/month (since the processing results in double storage). Say my processing is light on memory and can run at 1MB/s/CPU and I want to be finished in a day. I need 1150 CPU cores for 24h and at OVH that would cost me ~$1300 and bandwidth is free and unlimited. At GCE I can run on pre-emptible instances and pay ~$284 for the compute but I have to pay for 100TB of premium egress bandwidth I don't need, which adds 100 * 1000 * 0.04 (generously assuming I can get peering/interconnect) which is $4000. GCE is absolutely unusable for jobs like this in its current state. Sure I could move data to GCS but in that case my $1.5k bill for storage turns into a $5.2k bill per month.