I think this problem is almost entirely self-inflicted (by either a person or an organization, sorry to people in organizations that do this and who recognize t

I think this problem is almost entirely self-inflicted (by either a person or an organization, sorry to people in organizations that do this and who recognize the insanity and are powerless to stop it). BigQuery and Colab are fantastic tools largely because they allow less technical folks to try things out with very little friction and BigQuery scales to ridiculous amounts of data making formerly impossible queries possible. The example fits neither of these as the author is quite apparently very technical and the data is minuscule. The author could have applied some economics thinking to this and correctly concluded that they could have run this exploration on a laptop using a regular old file and Jupyter (or just normal Python if they want to avoid the IPython infrastructure). Post-exploration they can run their application on a properly specced VM. IDK if BERT requires a GPU these days or if it can run well enough on a CPU, but either way, these resources are easily available on GCP without all the vendor-ops the author talks about. To cap things off, the author says at the end that "in no previous universe would I be able to try out Stable Diffusion". Stable Diffusion requires 6 GB of GPU RAM and ~10 GB of storage. That was modest 5 years ago for commodity hardware. I am not a cloud-decrier by any means, I love the cloud, I use it every day, but most of what I use is basic storage (buckets) and VMs. With just a little bit of abstraction it's easy to make something that runs in the cloud run locally just as easily. That also makes it easier to switch clouds if you find a better deal somewhere else. TL;DR: use the right tools for the job