4 ms·

Is there any library that allows you to train with a Mac M1/M2? I know it will be slower but I rather spend money on a Mac Studio rather multiple graphic cards

Is there any library that allows you to train with a Mac M1/M2? I know it will be slower but I rather spend money on a Mac Studio rather multiple graphic cards to get around the VRAM limitation.

speedgoose 4y ago
For training you could rent a GPU server for a short period.
- te_chris 4y ago
  Or use something like google vertex to run a docker job on gpus
  - tysam_and 4y ago
    Personally I would recommend Colab or another notebook environment like a Lambda machine. Much cheaper, and simpler than a bare metal machine, data ingress/egress is hard though for Colab you can just mount a Gdrive. Unfortunately the API for training on M* chips (via MPS) is apparently still extremely buggy, so we have a ways to go before that is fully mainstream. And yes, I know that PyTorch just mainlined their mps support last week too...but from what I've heard the low level interface itself still needs some work. D:
    - nwoli 4y ago
      Just fyi colab is way way more expensive now after the “credits” update than it was a year ago. Lamdalabs is about as cheap at this point
      - capableweb 4y ago
        If you're fine with using other individuals machines (meaning, you don't care about data privacy for the training set), using vast.ai is probably the cheapest way to do it today. But, quality of machines/network speed vary greatly as the machines are hosted by individuals around the world.
      - tysam_and 4y ago
        I've used it for a long while after the credits update and it's been cheaper for me for a few reasons than Lambdalabs, one reason being the spinup/spindown times and being able to switch to a very cheap instance for prototyping. Lambdalabs has a crazy spinup time (comparatively), something like 3-5 minutes or so D: I find them good for the more mega runs where I don't have to touch my instance for several hours and it's worth the notebook porting time/cost. :D One of those usecases where raw cost doesn't always translate into money saved, at least in my personal experience. :D It's also nice that you can more easily edit in browser in colab before launching an instance, so you can do development->debugging->training via a single webpage without it breaking the bank or having huge privacy concerns. I think a close second would be Jupyter on vscode w/ a lambda backend but latency + fragmented architecture can add an extra step of complexity (which does matter! D:)
    - lxe 4y ago
      So hard I’ve been using lambda labs, vast.ai, and runpod to rent machines. A 3090 is about 30 cents an hour.
    - te_chris 4y ago
      Vertex has a training service, rather than spinning up a notebook. If you can dockerize the training job you just need to upload the container and data to gcs then it’s point and click to run once - assume it’s some sort of kubeflow or whatever in the background.
      - tysam_and 4y ago
        That sounds nice for the more involved big runs. I'll try to take a look at it sometime. I have some training runs that loop in just under 7 seconds at the shortest for the actual training portion of the run (maybe 9-10 including notebook spinup). After doing this for so many years, I've found that keeping it as short as humanly possible is usually the way to go. :D That said, I can definitely take a look at it for scale once I need to go over something like 1-2 hours maybe! :D
        te_chris 4y ago
        Yep, definitely best to do somewhere else if you can and just need to iterate, but it’s a good option when you have to do a big one and eat some real GPU time.
- tempaccount420 4y ago
  Renting is always less economical than owning. You can always sell your Apple Studio and still pay less.
  - qeternity 4y ago
    Perhaps on a per unit basis. But there's a reason I buy steaks and not cows.
- flemhans 4y ago
  Data will transmit to a third party, which may be a problem in some usecases

Is there any library that allows you to train with a Mac M1/M2? I know it will be slower but I rather spend money on a Mac Studio rather multiple graphic cards to get around the VRAM limitation.

speedgoose 4y ago

For training you could rent a GPU server for a short period.

te_chris 4y ago
Or use something like google vertex to run a docker job on gpus
- tysam_and 4y ago
  Personally I would recommend Colab or another notebook environment like a Lambda machine. Much cheaper, and simpler than a bare metal machine, data ingress/egress is hard though for Colab you can just mount a Gdrive. Unfortunately the API for training on M* chips (via MPS) is apparently still extremely buggy, so we have a ways to go before that is fully mainstream. And yes, I know that PyTorch just mainlined their mps support last week too...but from what I've heard the low level interface itself still needs some work. D:
  - nwoli 4y ago
    Just fyi colab is way way more expensive now after the “credits” update than it was a year ago. Lamdalabs is about as cheap at this point
    - capableweb 4y ago
      If you're fine with using other individuals machines (meaning, you don't care about data privacy for the training set), using vast.ai is probably the cheapest way to do it today. But, quality of machines/network speed vary greatly as the machines are hosted by individuals around the world.
    - tysam_and 4y ago
      I've used it for a long while after the credits update and it's been cheaper for me for a few reasons than Lambdalabs, one reason being the spinup/spindown times and being able to switch to a very cheap instance for prototyping. Lambdalabs has a crazy spinup time (comparatively), something like 3-5 minutes or so D: I find them good for the more mega runs where I don't have to touch my instance for several hours and it's worth the notebook porting time/cost. :D One of those usecases where raw cost doesn't always translate into money saved, at least in my personal experience. :D It's also nice that you can more easily edit in browser in colab before launching an instance, so you can do development->debugging->training via a single webpage without it breaking the bank or having huge privacy concerns. I think a close second would be Jupyter on vscode w/ a lambda backend but latency + fragmented architecture can add an extra step of complexity (which does matter! D:)
  - lxe 4y ago
    So hard I’ve been using lambda labs, vast.ai, and runpod to rent machines. A 3090 is about 30 cents an hour.
  - te_chris 4y ago
    Vertex has a training service, rather than spinning up a notebook. If you can dockerize the training job you just need to upload the container and data to gcs then it’s point and click to run once - assume it’s some sort of kubeflow or whatever in the background.
    - tysam_and 4y ago
      That sounds nice for the more involved big runs. I'll try to take a look at it sometime. I have some training runs that loop in just under 7 seconds at the shortest for the actual training portion of the run (maybe 9-10 including notebook spinup). After doing this for so many years, I've found that keeping it as short as humanly possible is usually the way to go. :D That said, I can definitely take a look at it for scale once I need to go over something like 1-2 hours maybe! :D
      - te_chris 4y ago
        Yep, definitely best to do somewhere else if you can and just need to iterate, but it’s a good option when you have to do a big one and eat some real GPU time.
tempaccount420 4y ago
Renting is always less economical than owning. You can always sell your Apple Studio and still pay less.
- qeternity 4y ago
  Perhaps on a per unit basis. But there's a reason I buy steaks and not cows.
flemhans 4y ago
Data will transmit to a third party, which may be a problem in some usecases