Recently I've been using GCP to train a model, some notes: * Like @leetharris, credits were pulled/not distributed, what was promised had to be cajoled out wit

Recently I've been using GCP to train a model, some notes: * Like @leetharris, credits were pulled/not distributed, what was promised had to be cajoled out with most of it being sent to some weird SaaS product that we'll never use * The GCP rep literally ghosted us halfway through the month where it when we had some expiring credits and were in the middle of training * Not that the credits mattered, our quota requests for lifting GPU or TPU was rejected twice. It was impossible to get any GPUs that were within our credits, even writing a script to try to look for machines for weeks didn't work. * Right after the credits expired, suddenly our last quota request, which was hanging around for weeks was approved. I assume they have an internal system setup to do that, but like we literally couldn't pay for GCP if we wanted to. * Also, GCP rates are like 2-4X the market rate. Like you can get an H100-80 from Runpod (and actually get one) for what GCP charges for an A100-40. Basically, the lesson learned was that no one should ever depend on GCP unless your time is worthless and you're not serious about getting any work done. They can go suck eggs.