3 ms·

The unexpected thing was that Google colab and GCS are separate such that transfer costs between them often end up as international external egress fees

55555 7y ago
Sorry, can you please try to explain that in another way? I don't understand what you mean.
reitzensteinm 7y ago
I don't know anything about your architecture, but you can quickly drop in something like BunnyCDN as a caching proxy for about 4% of price of Google egress. Even if it's a Google->Google transfer, since ingress is free, Google->Caching Proxy->Google should be much better. You could also consider a webhost like Hetzner, on which you can get a bunch of 1 gbps machines very cheap, though you have to manage them yourself. You could try Cloudflare, but they may well consider it abuse and cut you off (though they like to pretend on HN that they don't do this). Finally, I know that Google is handing out substantial credits for game developers and this could very well be up their alley. If you can optimize the download size a bit and swap to a cheaper delivery method, it should become feasible to run on a hobbyist budget.
ljm 7y ago
This is just GCS. For example, if you use their managed Kubernetes service, you will get a fresh load balancer for every service you expose to the internet. Not a shared load-balancer, a new one. Unless you set up an alternative you'll get absolutely rinsed through the cost of the instance and then the egress charges on top.
- aianus 7y ago
  All of the load balancers on GCP are shared. Maybe you meant to say you get a new, fresh IP address which is true but also not very expensive. "Cloud Load Balancing is a fully distributed, software-defined, managed service for all your traffic. It is not an instance- or device-based solution, so you won’t be locked into physical load balancing infrastructure or face the HA, scale, and management challenges inherent in instance-based LBs." https://cloud.google.com/load-balancing/ https://cloud.google.com/load-balancing/
  - ljm 7y ago
    That must have changed. I was billed for individual LBs a couple of years back.
  - pm90 7y ago
    Nothing in the paragraph indicates it’s shared. Also: it might be shared in the implementation but you will still be billed for every single https LB that you use (or an NLB if you’re doing tcp load balancing). Every unique kubernetes ingress resource WILL spin up a NEW, uniquely billed Https LB. Every unique kubernetes service with specific annotations will spin up a NEW, unique LB (internal or external). The author is correct.
    - aianus 7y ago
      You are not billed by the "load balancer", you are billed by the "forwarding rule" which makes it very obvious that the infrastructure is shared and that you will have additional costs with every K8S ingress.

55555 7y ago

Sorry, can you please try to explain that in another way? I don't understand what you mean.

reitzensteinm 7y ago

I don't know anything about your architecture, but you can quickly drop in something like BunnyCDN as a caching proxy for about 4% of price of Google egress. Even if it's a Google->Google transfer, since ingress is free, Google->Caching Proxy->Google should be much better. You could also consider a webhost like Hetzner, on which you can get a bunch of 1 gbps machines very cheap, though you have to manage them yourself. You could try Cloudflare, but they may well consider it abuse and cut you off (though they like to pretend on HN that they don't do this). Finally, I know that Google is handing out substantial credits for game developers and this could very well be up their alley. If you can optimize the download size a bit and swap to a cheaper delivery method, it should become feasible to run on a hobbyist budget.

ljm 7y ago

This is just GCS. For example, if you use their managed Kubernetes service, you will get a fresh load balancer for every service you expose to the internet. Not a shared load-balancer, a new one. Unless you set up an alternative you'll get absolutely rinsed through the cost of the instance and then the egress charges on top.

aianus 7y ago
All of the load balancers on GCP are shared. Maybe you meant to say you get a new, fresh IP address which is true but also not very expensive. "Cloud Load Balancing is a fully distributed, software-defined, managed service for all your traffic. It is not an instance- or device-based solution, so you won’t be locked into physical load balancing infrastructure or face the HA, scale, and management challenges inherent in instance-based LBs." https://cloud.google.com/load-balancing/ https://cloud.google.com/load-balancing/
- ljm 7y ago
  That must have changed. I was billed for individual LBs a couple of years back.
- pm90 7y ago
  Nothing in the paragraph indicates it’s shared. Also: it might be shared in the implementation but you will still be billed for every single https LB that you use (or an NLB if you’re doing tcp load balancing). Every unique kubernetes ingress resource WILL spin up a NEW, uniquely billed Https LB. Every unique kubernetes service with specific annotations will spin up a NEW, unique LB (internal or external). The author is correct.
  - aianus 7y ago
    You are not billed by the "load balancer", you are billed by the "forwarding rule" which makes it very obvious that the infrastructure is shared and that you will have additional costs with every K8S ingress.