4 ms·

Or use something like google vertex to run a docker job on gpus

tysam_and 4y ago
Personally I would recommend Colab or another notebook environment like a Lambda machine. Much cheaper, and simpler than a bare metal machine, data ingress/egress is hard though for Colab you can just mount a Gdrive. Unfortunately the API for training on M* chips (via MPS) is apparently still extremely buggy, so we have a ways to go before that is fully mainstream. And yes, I know that PyTorch just mainlined their mps support last week too...but from what I've heard the low level interface itself still needs some work. D:
- nwoli 4y ago
  Just fyi colab is way way more expensive now after the “credits” update than it was a year ago. Lamdalabs is about as cheap at this point
  - capableweb 4y ago
    If you're fine with using other individuals machines (meaning, you don't care about data privacy for the training set), using vast.ai is probably the cheapest way to do it today. But, quality of machines/network speed vary greatly as the machines are hosted by individuals around the world.
  - tysam_and 4y ago
    I've used it for a long while after the credits update and it's been cheaper for me for a few reasons than Lambdalabs, one reason being the spinup/spindown times and being able to switch to a very cheap instance for prototyping. Lambdalabs has a crazy spinup time (comparatively), something like 3-5 minutes or so D: I find them good for the more mega runs where I don't have to touch my instance for several hours and it's worth the notebook porting time/cost. :D One of those usecases where raw cost doesn't always translate into money saved, at least in my personal experience. :D It's also nice that you can more easily edit in browser in colab before launching an instance, so you can do development->debugging->training via a single webpage without it breaking the bank or having huge privacy concerns. I think a close second would be Jupyter on vscode w/ a lambda backend but latency + fragmented architecture can add an extra step of complexity (which does matter! D:)
- lxe 4y ago
  So hard I’ve been using lambda labs, vast.ai, and runpod to rent machines. A 3090 is about 30 cents an hour.
- te_chris 4y ago
  Vertex has a training service, rather than spinning up a notebook. If you can dockerize the training job you just need to upload the container and data to gcs then it’s point and click to run once - assume it’s some sort of kubeflow or whatever in the background.
  - tysam_and 4y ago
    That sounds nice for the more involved big runs. I'll try to take a look at it sometime. I have some training runs that loop in just under 7 seconds at the shortest for the actual training portion of the run (maybe 9-10 including notebook spinup). After doing this for so many years, I've found that keeping it as short as humanly possible is usually the way to go. :D That said, I can definitely take a look at it for scale once I need to go over something like 1-2 hours maybe! :D
    - te_chris 4y ago
      Yep, definitely best to do somewhere else if you can and just need to iterate, but it’s a good option when you have to do a big one and eat some real GPU time.

Personally I would recommend Colab or another notebook environment like a Lambda machine. Much cheaper, and simpler than a bare metal machine, data ingress/egress is hard though for Colab you can just mount a Gdrive. Unfortunately the API for training on M* chips (via MPS) is apparently still extremely buggy, so we have a ways to go before that is fully mainstream. And yes, I know that PyTorch just mainlined their mps support last week too...but from what I've heard the low level interface itself still needs some work. D:

nwoli 4y ago
Just fyi colab is way way more expensive now after the “credits” update than it was a year ago. Lamdalabs is about as cheap at this point
- capableweb 4y ago
  If you're fine with using other individuals machines (meaning, you don't care about data privacy for the training set), using vast.ai is probably the cheapest way to do it today. But, quality of machines/network speed vary greatly as the machines are hosted by individuals around the world.
- tysam_and 4y ago
  I've used it for a long while after the credits update and it's been cheaper for me for a few reasons than Lambdalabs, one reason being the spinup/spindown times and being able to switch to a very cheap instance for prototyping. Lambdalabs has a crazy spinup time (comparatively), something like 3-5 minutes or so D: I find them good for the more mega runs where I don't have to touch my instance for several hours and it's worth the notebook porting time/cost. :D One of those usecases where raw cost doesn't always translate into money saved, at least in my personal experience. :D It's also nice that you can more easily edit in browser in colab before launching an instance, so you can do development->debugging->training via a single webpage without it breaking the bank or having huge privacy concerns. I think a close second would be Jupyter on vscode w/ a lambda backend but latency + fragmented architecture can add an extra step of complexity (which does matter! D:)
lxe 4y ago
So hard I’ve been using lambda labs, vast.ai, and runpod to rent machines. A 3090 is about 30 cents an hour.
te_chris 4y ago
Vertex has a training service, rather than spinning up a notebook. If you can dockerize the training job you just need to upload the container and data to gcs then it’s point and click to run once - assume it’s some sort of kubeflow or whatever in the background.
- tysam_and 4y ago
  That sounds nice for the more involved big runs. I'll try to take a look at it sometime. I have some training runs that loop in just under 7 seconds at the shortest for the actual training portion of the run (maybe 9-10 including notebook spinup). After doing this for so many years, I've found that keeping it as short as humanly possible is usually the way to go. :D That said, I can definitely take a look at it for scale once I need to go over something like 1-2 hours maybe! :D
  - te_chris 4y ago
    Yep, definitely best to do somewhere else if you can and just need to iterate, but it’s a good option when you have to do a big one and eat some real GPU time.