4 ms·

I like Cloud Run a lot as an evolution of Cloud Functions. It has the ability to scale to 0 docker containers. My OpenResty API gateway takes 400ms to spin up a

I like Cloud Run a lot as an evolution of Cloud Functions. It has the ability to scale to 0 docker containers. My OpenResty API gateway takes 400ms to spin up and can serve a ton of traffic concurrently (https://futurice.com/blog/openresty-a-swiss-army-proxy-for-serverless https://futurice.com/blog/openresty-a-swiss-army-proxy-for-s...). This is much better and cheaper than minimum scale of 1 Fargate containers IMHO. There are a lot of legacy things you can put in Cloud Run (e.g. JVM application servers) that would not be possible in Cloud Functions. Cloud Functions are good for small custom node.js express apps. Cloud Run is awesome for heavier IO bound services as you can make use of the runtime resources when waiting for other services to complete (by serving concurrent requests).

yalooze 6y ago
I’m using Cloud Run too and love it in principle and, mostly, in practice. I’m running a Rails app which takes about 30 seconds to start on Run (much quicker on a standard instance or local and I’m not sure why). My other gripe is that it will just randomly restart (not after being idle for some time but literally as I’m clicking around the website). There are no server errors and nothing I can see in the logs. Coupled with the fact that restarts take around 30 seconds it makes for quite a poor initial experience for my web users.
- Just1689 6y ago
  This advice might not be at all helpful, but anyway. Consider checking the following: - Number of CPUs. The number of CPUs is not CPU cores but Kubernetes millicores. 1 vCPU on CR = 400m (4/10 of a CPU core). - Background processes - not recommended [https://stackoverflow.com/questions/61154349/google-cloud-run-is-very-slow-vs-local-machine https://stackoverflow.com/questions/61154349/google-cloud-ru...] - Container size on disk. Google Cloud is fast. However they need to cap the moving container images around as they can't use all the network capacity for moving a 400MB Docker image. If you're not already, use multi-stage builds with tiny final images.
  - kircr 6y ago
    Where do you get the info that 1 vCPU on CR = 400m? I would have expected 1 vCPU on CR = 1000m
  - nikon 6y ago
    This is not correct. > Limits and requests for CPU resources are measured in cpu units. One cpu, in Kubernetes, is equivalent to 1 vCPU/Core for cloud providers and 1 hyperthread on bare-metal Intel processors.
- tlarkworthy 6y ago
  Yeah I have a jvm jboss type thing that take 5 mins to startup, which is why I have that pubsub to self in the proxy so I can absorb delays in a managed queue. This of course relies on the upstream not being time sensative or needing a syncronous reply. However, nginx does start in 400ms on cloud run so it's clearly it's the cruft in the docker that's doing it and not the hosting (code is here BTW https://github.com/futurice/terraform-examples/tree/master/google_cloud/openresty-beyondcorp https://github.com/futurice/terraform-examples/tree/master/g...)
- spyspy 6y ago
  For comparison I’ve been using Go and a distroless container image and never had any problems. However you can set the minimum instance count now. That feature was opened up for Cloud Run managed a few weeks ago.
- kelseyhightower 6y ago
  Cold starts can be one of the biggest trade offs when adopting a Serverless platform like Cloud Run, especially with 30 second start up times. We introduced minimum instances[1], currently in beta, which will allow you to specify a minimum number of container instances to be kept warm and ready to serve requests. There is a cost for doing so as instances kept running in this way do incur billing costs [1]. I've done some work with Ruby in the past, and based on my experience, you might not be able to take advantage of Cloud Run's best feature, concurrency[2], which is another way to reduce cold starts by routing concurrent requests to the same container instance. Ruby maybe blocking in flight requests and forcing us to fire up another instance to handle it. Once the first request has completed there is a chance the container instance processing that request will be spun down, this is something minimum instances can help with. You might need more than one minimum instance to compensate for the lack of concurrency if you really want to see improvement in overall latency. [1] https://cloud.google.com/run/docs/configuring/min-instances https://cloud.google.com/run/docs/configuring/min-instances [2] https://cloud.google.com/run/docs/about-concurrency https://cloud.google.com/run/docs/about-concurrency
spyspy 6y ago
I’ve been using GCP full time since 2015 and I expect Google to deprecate AppEngine and Functions in favor of Cloud Run almost any day now.
- kelseyhightower 6y ago
  Thanks for your comment. This is something the team keeps top of mind and influences our decisions and road maps. Please continue to keep us honest here. I'll get others to weight in on this, but here's my personal opinion on the matter. AppEngine has been around for a long time, 12 years to be exact, and in my mind it's a completely different product, and still one of our most successful. While Cloud Run competes with AppEngine on some fronts, customers really appreciate AppEngine's deep integration with other GCP managed services. It's a full blown PaaS. I expect Cloud Run to improve over time and gain more features that will no doubt match the requirements of many AppEngine customers, some may switch to Cloud Run, but we are not forcing them to. Cloud Functions already shares a lot of underlying infrastructure with Cloud Run, which shares a lot of underlying infrastructure with AppEngine, and that's by design. These days I like to think of Cloud Functions as a simplified developer experience on top of Cloud Run focused on task orientated workloads backed by events and triggers. Can we continue to support 3 somewhat overlapping Serverless platforms going forward? I believe the answer is yes, thanks to the shared infrastructure, and the fact we truly believe all three platforms offer a unique set of developer experiences worth preserving. Maybe there is future state where one product satisfies all use cases, but that's not today, so we plan to keep listening to customers, and invest across the board. One recent example of investing across the board: you can now leverage global load balancing[1] across all three Serverless platforms. [1] https://cloud.google.com/load-balancing/docs/negs/serverless-neg-concepts https://cloud.google.com/load-balancing/docs/negs/serverless...
  - spyspy 6y ago
    Hi Kelsey! Appreciate your take here. You obviously have deeper insight than I do. Honestly I'm in favor of the path I described. Engineers I work with spend too much time already arguing the best platform for any given service, especially when the pros and cons are very, very similar. That being said, with the recent features released for Run (load-balancing, min instances, streaming gRPC) we're basically 100% bought in to that direction. I can't think of any feature AppEngine or Functions have (esp AppEngine) that would pull me back. Though I will say I dearly miss the days of first generation GAE standard, when things like memcache, taskqeue, datastore, auth and logging were all baked in as first-class citizens and Just Worked™.
victor106 6y ago
> There are a lot of legacy things you can put in Cloud Run (e.g. JVM application servers) I wouldn’t call JVM application servers legacy. If you are referring to WebSphere and Weblogic, then maybe. But the open source JVM application servers like tomcat,jetty, netty etc., are widely used in modern applications

yalooze 6y ago

I’m using Cloud Run too and love it in principle and, mostly, in practice. I’m running a Rails app which takes about 30 seconds to start on Run (much quicker on a standard instance or local and I’m not sure why). My other gripe is that it will just randomly restart (not after being idle for some time but literally as I’m clicking around the website). There are no server errors and nothing I can see in the logs. Coupled with the fact that restarts take around 30 seconds it makes for quite a poor initial experience for my web users.

Just1689 6y ago
This advice might not be at all helpful, but anyway. Consider checking the following: - Number of CPUs. The number of CPUs is not CPU cores but Kubernetes millicores. 1 vCPU on CR = 400m (4/10 of a CPU core). - Background processes - not recommended [https://stackoverflow.com/questions/61154349/google-cloud-run-is-very-slow-vs-local-machine https://stackoverflow.com/questions/61154349/google-cloud-ru...] - Container size on disk. Google Cloud is fast. However they need to cap the moving container images around as they can't use all the network capacity for moving a 400MB Docker image. If you're not already, use multi-stage builds with tiny final images.
- kircr 6y ago
  Where do you get the info that 1 vCPU on CR = 400m? I would have expected 1 vCPU on CR = 1000m
- nikon 6y ago
  This is not correct. > Limits and requests for CPU resources are measured in cpu units. One cpu, in Kubernetes, is equivalent to 1 vCPU/Core for cloud providers and 1 hyperthread on bare-metal Intel processors.
tlarkworthy 6y ago
Yeah I have a jvm jboss type thing that take 5 mins to startup, which is why I have that pubsub to self in the proxy so I can absorb delays in a managed queue. This of course relies on the upstream not being time sensative or needing a syncronous reply. However, nginx does start in 400ms on cloud run so it's clearly it's the cruft in the docker that's doing it and not the hosting (code is here BTW https://github.com/futurice/terraform-examples/tree/master/google_cloud/openresty-beyondcorp https://github.com/futurice/terraform-examples/tree/master/g...)
spyspy 6y ago
For comparison I’ve been using Go and a distroless container image and never had any problems. However you can set the minimum instance count now. That feature was opened up for Cloud Run managed a few weeks ago.
kelseyhightower 6y ago
Cold starts can be one of the biggest trade offs when adopting a Serverless platform like Cloud Run, especially with 30 second start up times. We introduced minimum instances[1], currently in beta, which will allow you to specify a minimum number of container instances to be kept warm and ready to serve requests. There is a cost for doing so as instances kept running in this way do incur billing costs [1]. I've done some work with Ruby in the past, and based on my experience, you might not be able to take advantage of Cloud Run's best feature, concurrency[2], which is another way to reduce cold starts by routing concurrent requests to the same container instance. Ruby maybe blocking in flight requests and forcing us to fire up another instance to handle it. Once the first request has completed there is a chance the container instance processing that request will be spun down, this is something minimum instances can help with. You might need more than one minimum instance to compensate for the lack of concurrency if you really want to see improvement in overall latency. [1] https://cloud.google.com/run/docs/configuring/min-instances https://cloud.google.com/run/docs/configuring/min-instances [2] https://cloud.google.com/run/docs/about-concurrency https://cloud.google.com/run/docs/about-concurrency

spyspy 6y ago

I’ve been using GCP full time since 2015 and I expect Google to deprecate AppEngine and Functions in favor of Cloud Run almost any day now.

kelseyhightower 6y ago
Thanks for your comment. This is something the team keeps top of mind and influences our decisions and road maps. Please continue to keep us honest here. I'll get others to weight in on this, but here's my personal opinion on the matter. AppEngine has been around for a long time, 12 years to be exact, and in my mind it's a completely different product, and still one of our most successful. While Cloud Run competes with AppEngine on some fronts, customers really appreciate AppEngine's deep integration with other GCP managed services. It's a full blown PaaS. I expect Cloud Run to improve over time and gain more features that will no doubt match the requirements of many AppEngine customers, some may switch to Cloud Run, but we are not forcing them to. Cloud Functions already shares a lot of underlying infrastructure with Cloud Run, which shares a lot of underlying infrastructure with AppEngine, and that's by design. These days I like to think of Cloud Functions as a simplified developer experience on top of Cloud Run focused on task orientated workloads backed by events and triggers. Can we continue to support 3 somewhat overlapping Serverless platforms going forward? I believe the answer is yes, thanks to the shared infrastructure, and the fact we truly believe all three platforms offer a unique set of developer experiences worth preserving. Maybe there is future state where one product satisfies all use cases, but that's not today, so we plan to keep listening to customers, and invest across the board. One recent example of investing across the board: you can now leverage global load balancing[1] across all three Serverless platforms. [1] https://cloud.google.com/load-balancing/docs/negs/serverless-neg-concepts https://cloud.google.com/load-balancing/docs/negs/serverless...
- spyspy 6y ago
  Hi Kelsey! Appreciate your take here. You obviously have deeper insight than I do. Honestly I'm in favor of the path I described. Engineers I work with spend too much time already arguing the best platform for any given service, especially when the pros and cons are very, very similar. That being said, with the recent features released for Run (load-balancing, min instances, streaming gRPC) we're basically 100% bought in to that direction. I can't think of any feature AppEngine or Functions have (esp AppEngine) that would pull me back. Though I will say I dearly miss the days of first generation GAE standard, when things like memcache, taskqeue, datastore, auth and logging were all baked in as first-class citizens and Just Worked™.

victor106 6y ago

> There are a lot of legacy things you can put in Cloud Run (e.g. JVM application servers) I wouldn’t call JVM application servers legacy. If you are referring to WebSphere and Weblogic, then maybe. But the open source JVM application servers like tomcat,jetty, netty etc., are widely used in modern applications