3 ms·
As someone who works in Data Engineering, I'm a big fan of cloud-based docker containers with services like AWS Fargate/Google Run. It allows us to be able to p
by aelzeiny 7y ago
As someone who works in Data Engineering, I'm a big fan of cloud-based docker containers with services like AWS Fargate/Google Run. It allows us to be able to pay for resource use rather than provisioning. For the longest time I had to justify the additional cost and constant maintenance of an n-th server just to accommodate for capacity at peak hours. Anecdotally, most of our batch jobs don't need high availability, so the startup time of a server-less docker is an acceptable trade-off to parallelism.
- wtmt 7y ago> It allows us to be able to pay for resource use rather than provisioning. Not arguing against the choice, but at some level, you are paying for the provisioning costs of the cloud provider. The higher initial costs of provisioning is what you may be escaping, and instead paying on an ongoing basis (as if you’re leasing equipment and services). Once your needs scale significantly, cloud providers would end up being quite expensive if your load is not highly variable.
- chrisbroadfoot 7y agoIt's compatible with Knative, which is easy to set up in GKE, so you can migrate loads over to that if you want more control over your compute costs. And of course, you can run Knative outside of GKE, too. Disclaimer: I work on GCP but have only touched Cloud Run/Knative a little bit.
- cwyers 7y ago> if your load is not highly variable Most real-world loads are in fact pretty variable.
- kortilla 7y agoYou know you can also boot instances that bill at hour and less granularity right? Billing is probably the least compelling argument to a service like this.