5 ms·

Ask HN: Best way to spend $25k in GCP

21 points by soheil 5y ago

I have $25k remaining credit from Google Cloud Platforms, which expires in about a month. I got the credit over 2 years ago and had hundreds of severs for various websites, database storage, elasticsearch instances, dedicated servers for random cron tasks, etc. But I still haven't been able to exhaust the credit. I even used it to train several models using multiple Tesla GPUs. But beyond a point I felt like I didn't know what I was doing with the models and stopped wasting GPU cycles on them. I feel like if I don't use it for something useful it'll be wasted. Any ideas?

Cilvic 5y ago
How about seeing how big of an index of the web one can build for 25k? I am inspired by this: https://news.ycombinator.com/item?id=28550764 https://news.ycombinator.com/item?id=28550764 I was thinking about building a domain name index, this would give the project a nice deadline and budget :-) If you want to collaborate, my email is in my profile.
captn3m0 5y ago
You are allowed to run BOINC[0] and folding@home[1] [0]: https://old.reddit.com/r/gridcoin/comments/7vhku5/fyi_we_are_allowed_to_run_boinc_on_google_cloud/ https://old.reddit.com/r/gridcoin/comments/7vhku5/fyi_we_are... [1]: https://github.com/GoogleCloudPlatform/terraform-folding-at-home https://github.com/GoogleCloudPlatform/terraform-folding-at-...
- junon 5y ago
  This is probably the best answer, aside from "don't contribute to the environmental waste".
  - pokepim 5y ago
    I think it’s even better than not contributing to the environmental waste. I believe helping to find cure for many of the deadly diseases will save more lives than cause harm to environment by running some gpus/cpus. Like way more.
    - hoseja 5y ago
      I suppose it's the distinction between finding the Haber-Bosch process good or not.
- chrisandchris 5y ago
  That is the only „valid“ answer IMHO. Spend all on BOINC and do something for the world.
- netizen-936824 5y ago
  10000% this We need more computing resources dedicated to scientific work rather than the (in my biased opinion) bullshit consumer apps that don't actually help anyone with anything besides making their owners assloads of money
aborsy 5y ago
Don’t use it if not needed. On the contrary, it contributes to the environmental waste.
AtlasBarfed 5y ago
Can't you just buy some really long term upfront storage contract? You'll always need storage.
- captn3m0 5y ago
  No, credits can't be used for purchases beyond the credit expiry. This is how AWS[0] limits it: > Promotional Credit will be applied only to offset eligible fees and charges incurred during or following the billing cycle in which you apply the applicable Promotional Credit code to your AWS account. Promotional Credit will be applied only to the specific Services designated by your AWS contracting entity (collectively, “Eligible Services”). Promotional Credit will not be applied to any fees or charges for Amazon Mechanical Turk, AWS Managed Services, Ineligible AWS Support, AWS Marketplace, AWS Professional Services, AWS Training, AWS Certification, Amazon Route 53 domain name registration or transfer, any Services for mining for cryptocurrency, any other Services as may be designated by your AWS contracting entity, or any upfront fee for any Services such as Savings Plans and Reserved Instances (collectively, “Ineligible Services”). For purposes of this Section 1, “Ineligible AWS Support” means AWS Support (as described on the AWS Site) that is at the Enterprise Support level. Can't find the relevant GCP doc, but I'm sure that is similarly limited. [0]: https://aws.amazon.com/awscredits/ https://aws.amazon.com/awscredits/
unixhero 5y ago
Maybe you can mine some Ethereum. Not sure if that breaks their TOS, you'd have to check to see that this is allowed.
- junon 5y ago
  It certainly breaks their TOS.
  - interator7 5y ago
    Why?
option_greek 5y ago
It really shouldn't, but stories like this irks me a lot because I have been trying to get credits for my bootstrapped startup for sometime and there is just no way to get them unless you are part of a accelerator or took funding from well known investor. Of course, Gcp, Aws, DO don't owe me anything and these programs must be successful based on their data but still shows the irony of the world where resources aren't matched well against needs.
- nowherebeen 5y ago
  You can join startup school and get them.
sillysaurusx 5y ago
Dear soheil, I happen to need around few thousand in GCP credits in order to egress around 30TB of data from our cloud buckets. It's training data, training logs, model snapshots, and pretty much all of our ML history (http://wiki.tensorfork.com/ http://wiki.tensorfork.com/). The real tally is actually 83TB, but around 30TB of that is incremental model snapshots which we don't need to download (turns out, you can generate a lot of data when you're using TPUs for training), and around ~30TB is whatever-data that no one cares about / duplicated data across regions. (To use TPUs in EU and US, you need to place the training data in both EU and US.) Normally I wouldn't ask, but we've been playing "hot potato" with this data for several months now, since our last credits ran out. We've basically been making new GCP accounts (where you get $300 credit) and then letting it sit there for a month (because they give you a month before deleting it, even if the credits run out). If you'd be willing to help, it would be quite simple: I would transfer the buckets to your account, and then download the data to a Hetzner server. I'd then make the data publicly available, such as it is. You can DM me on twitter (https://twitter.com/theshawwn https://twitter.com/theshawwn) or email me (see profile). Thanks for considering! EDIT: For an example of how we've amassed terabytes of training logs, you can see some of our BigGAN outputs here: https://twitter.com/search?q=from%3Atheshawwn%20biggan&src=typed_query https://twitter.com/search?q=from%3Atheshawwn%20biggan&src=t...
- google234123 5y ago
  Do these snapshots or models even matter anymore? Are snapshots from training some image generating model really useful to anyone and arent there already newer and better models out?
cprecioso 5y ago
Setup a cloud gaming rig and max it out :D
aynyc 5y ago
How did you manage to get that much? I’ve been trying with no avail to get some decent credit to run my side project. I’m playing the whack-a-mole game to signing up new account every time my $300 runs out.
tester34 5y ago
How about running heavy computations for somebody else for some $$? "ROI"
Jugurtha 5y ago
You mention training models... Do you want to donate them to ML researchers or students? Maybe people can use these on our platform (https://iko.ai https://iko.ai) which is on GCP and GKE. The platform provides real-time collaborative notebooks so people can train, track, package, deploy, and monitor machine learning models on the users' Kubernetes clusters (AWS, GCP, Azure, Digital Ocean, etc. So if you have credits elsewhere, this may be doable). You could transfer your credits to us, we'll generate a special link, then you donate that link to people or entities who, when signing up with the link, could use your resources.

21 points by soheil 5y ago

Cilvic 5y ago

How about seeing how big of an index of the web one can build for 25k? I am inspired by this: https://news.ycombinator.com/item?id=28550764 https://news.ycombinator.com/item?id=28550764 I was thinking about building a domain name index, this would give the project a nice deadline and budget :-) If you want to collaborate, my email is in my profile.

captn3m0 5y ago

You are allowed to run BOINC[0] and folding@home[1] [0]: https://old.reddit.com/r/gridcoin/comments/7vhku5/fyi_we_are_allowed_to_run_boinc_on_google_cloud/ https://old.reddit.com/r/gridcoin/comments/7vhku5/fyi_we_are... [1]: https://github.com/GoogleCloudPlatform/terraform-folding-at-home https://github.com/GoogleCloudPlatform/terraform-folding-at-...

junon 5y ago
This is probably the best answer, aside from "don't contribute to the environmental waste".
- pokepim 5y ago
  I think it’s even better than not contributing to the environmental waste. I believe helping to find cure for many of the deadly diseases will save more lives than cause harm to environment by running some gpus/cpus. Like way more.
  - hoseja 5y ago
    I suppose it's the distinction between finding the Haber-Bosch process good or not.
chrisandchris 5y ago
That is the only „valid“ answer IMHO. Spend all on BOINC and do something for the world.
netizen-936824 5y ago
10000% this We need more computing resources dedicated to scientific work rather than the (in my biased opinion) bullshit consumer apps that don't actually help anyone with anything besides making their owners assloads of money

aborsy 5y ago

Don’t use it if not needed. On the contrary, it contributes to the environmental waste.

AtlasBarfed 5y ago

Can't you just buy some really long term upfront storage contract? You'll always need storage.

captn3m0 5y ago
No, credits can't be used for purchases beyond the credit expiry. This is how AWS[0] limits it: > Promotional Credit will be applied only to offset eligible fees and charges incurred during or following the billing cycle in which you apply the applicable Promotional Credit code to your AWS account. Promotional Credit will be applied only to the specific Services designated by your AWS contracting entity (collectively, “Eligible Services”). Promotional Credit will not be applied to any fees or charges for Amazon Mechanical Turk, AWS Managed Services, Ineligible AWS Support, AWS Marketplace, AWS Professional Services, AWS Training, AWS Certification, Amazon Route 53 domain name registration or transfer, any Services for mining for cryptocurrency, any other Services as may be designated by your AWS contracting entity, or any upfront fee for any Services such as Savings Plans and Reserved Instances (collectively, “Ineligible Services”). For purposes of this Section 1, “Ineligible AWS Support” means AWS Support (as described on the AWS Site) that is at the Enterprise Support level. Can't find the relevant GCP doc, but I'm sure that is similarly limited. [0]: https://aws.amazon.com/awscredits/ https://aws.amazon.com/awscredits/

unixhero 5y ago

Maybe you can mine some Ethereum. Not sure if that breaks their TOS, you'd have to check to see that this is allowed.

junon 5y ago
It certainly breaks their TOS.
- interator7 5y ago
  Why?

option_greek 5y ago

It really shouldn't, but stories like this irks me a lot because I have been trying to get credits for my bootstrapped startup for sometime and there is just no way to get them unless you are part of a accelerator or took funding from well known investor. Of course, Gcp, Aws, DO don't owe me anything and these programs must be successful based on their data but still shows the irony of the world where resources aren't matched well against needs.

nowherebeen 5y ago
You can join startup school and get them.

sillysaurusx 5y ago

Dear soheil, I happen to need around few thousand in GCP credits in order to egress around 30TB of data from our cloud buckets. It's training data, training logs, model snapshots, and pretty much all of our ML history (http://wiki.tensorfork.com/ http://wiki.tensorfork.com/). The real tally is actually 83TB, but around 30TB of that is incremental model snapshots which we don't need to download (turns out, you can generate a lot of data when you're using TPUs for training), and around ~30TB is whatever-data that no one cares about / duplicated data across regions. (To use TPUs in EU and US, you need to place the training data in both EU and US.) Normally I wouldn't ask, but we've been playing "hot potato" with this data for several months now, since our last credits ran out. We've basically been making new GCP accounts (where you get $300 credit) and then letting it sit there for a month (because they give you a month before deleting it, even if the credits run out). If you'd be willing to help, it would be quite simple: I would transfer the buckets to your account, and then download the data to a Hetzner server. I'd then make the data publicly available, such as it is. You can DM me on twitter (https://twitter.com/theshawwn https://twitter.com/theshawwn) or email me (see profile). Thanks for considering! EDIT: For an example of how we've amassed terabytes of training logs, you can see some of our BigGAN outputs here: https://twitter.com/search?q=from%3Atheshawwn%20biggan&src=typed_query https://twitter.com/search?q=from%3Atheshawwn%20biggan&src=t...

google234123 5y ago
Do these snapshots or models even matter anymore? Are snapshots from training some image generating model really useful to anyone and arent there already newer and better models out?

cprecioso 5y ago

Setup a cloud gaming rig and max it out :D

aynyc 5y ago

How did you manage to get that much? I’ve been trying with no avail to get some decent credit to run my side project. I’m playing the whack-a-mole game to signing up new account every time my $300 runs out.

tester34 5y ago

How about running heavy computations for somebody else for some $$? "ROI"

Jugurtha 5y ago

You mention training models... Do you want to donate them to ML researchers or students? Maybe people can use these on our platform (https://iko.ai https://iko.ai) which is on GCP and GKE. The platform provides real-time collaborative notebooks so people can train, track, package, deploy, and monitor machine learning models on the users' Kubernetes clusters (AWS, GCP, Azure, Digital Ocean, etc. So if you have credits elsewhere, this may be doable). You could transfer your credits to us, we'll generate a special link, then you donate that link to people or entities who, when signing up with the link, could use your resources.