4 ms·

I don’t understand this logic. If someone has $1000 for an entry level m1 machine then they also have enough money for a separate rig with a GPU that’s probably

I don’t understand this logic. If someone has $1000 for an entry level m1 machine then they also have enough money for a separate rig with a GPU that’s probably another $600-1000 for something decent? Cloud GPUs are also pretty expensive. I don’t think anyone is seriously training their ML models on their ultra thin laptop but I think the ability to do so would make it easier for lots of people to get started with what they have. It might be that they don’t have lots of extra cash to throw around or it’s hard to justify on something they are just starting. Cloud training is also just less convenient and an extra hassle compared to doing things locally, even if it’s something like google collab which is probably the best option right now for training on thin laptops

ipsum2 6y ago
If someone is shelling out for a brand new, early adopter product, then they probably have a decent amount of money. Even when TensorFlow and PyTorch implement training support on the M1, it will be useless for practically anything except training 2-3 layer models on MNIST. So why should valuable engineering time be spent on this?
- tpetry 6y ago
  Because with support for M1 you can prototype your network on your local machine with „good“ performance. There are many cloud solutions etc. but for convenience nothing beats your local machine. You can use an IDE you like etc.
  - rfoo 6y ago
    Because contrary to what you believe, M1 simply is not performant enough to be used to "prototype" your network. NNs can't be simply scaled up and down. It is *NOT* like those web apps which you can run on potatoes just fine as long as nobody are hitting them heavily.
    - rsfern 6y ago
      Sure you probably don’t want to do full training runs locally, but There’s a lot you can do locally that has a lot of added friction on a gpu cluster or other remote compute resource I like to start a new project by prototyping and debugging my training and cunning config code, setting up the data loading and evaluation pipeline, hacking around with some baseline models and making sure they can overfit some small subset of my data After all that’s done it’s finally time to scale out to the gpu cluster. But I still do a lot of debugging locally Maybe this kind of workflow isn’t as necessary if you have a task that’s pretty plug and play like image classification, but for nonstandard tasks I think there’s lots of prototyping work that doesn’t require hardware acceleration
      - jefft255 6y ago
        Coding somewhat locally is a must for me too because the cluster I have access to has pretty serious wait times (up to a couple hours on busy days). Imagine only being able to run the code you’re writing a few times a day at most! Iterative development and doing a lot of mistakes is how I code; I don’t want to go back to punch card days where you waited and waited before you ended up with a silly error.
    - tpetry 6y ago
      You can just run your training with a lot less of data.
    - jefft255 6y ago
      Not so sure about that. Here’s two things you can do (assuming you’re not training huge transformers or something). 1. Test your code with super low batch size. Bad for convergence, good for sanity check before submitting your job to a super computer. 2. Post-training evaluation. I’m pretty sure the M1 has enough power to do inference for not-so-big models. These two reasons are why I’m sometimes running stuff on my own GTX 1060, even though it’s pretty anemic and I wouldn’t actually do a training run there. There quite a bit of friction to training in the cloud, especially if it’s on shared cluster (which is what I have access to). You have a quota, and wait time when the supercomputer is under load. Sometimes you just need to quickly fire up something!
- sjwright 6y ago
  I think you underestimate just how many MacBooks Apple sells. There’s only so many affluent early adopters out there. And the M1 MacBooks are especially ideal for mainstream customers.
- sumnuyungi 6y ago
  This is just patently false. Most of the folks I know that have an M1 are students that were saving up to upgrade from a much older computer and got the M1 Air. I can assure you that they don't have a decent amount of money. Tensorflow has a branch that's optimized for metal with impressive performance. [1] It's fast enough to do transfer learning quickly on a large resnet, which is a common use-case for photo/video editing apps that have ML-powered workflows. It's best for everyone to do this locally: maintains privacy for the user and eliminates cloud costs for the developer. Also, not everyone has an imagenet sized dataset. A lot of applied ML uses small networks where prototyping is doable on a local machine. [1] https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...

ipsum2 6y ago

If someone is shelling out for a brand new, early adopter product, then they probably have a decent amount of money. Even when TensorFlow and PyTorch implement training support on the M1, it will be useless for practically anything except training 2-3 layer models on MNIST. So why should valuable engineering time be spent on this?

tpetry 6y ago
Because with support for M1 you can prototype your network on your local machine with „good“ performance. There are many cloud solutions etc. but for convenience nothing beats your local machine. You can use an IDE you like etc.
- rfoo 6y ago
  Because contrary to what you believe, M1 simply is not performant enough to be used to "prototype" your network. NNs can't be simply scaled up and down. It is *NOT* like those web apps which you can run on potatoes just fine as long as nobody are hitting them heavily.
  - rsfern 6y ago
    Sure you probably don’t want to do full training runs locally, but There’s a lot you can do locally that has a lot of added friction on a gpu cluster or other remote compute resource I like to start a new project by prototyping and debugging my training and cunning config code, setting up the data loading and evaluation pipeline, hacking around with some baseline models and making sure they can overfit some small subset of my data After all that’s done it’s finally time to scale out to the gpu cluster. But I still do a lot of debugging locally Maybe this kind of workflow isn’t as necessary if you have a task that’s pretty plug and play like image classification, but for nonstandard tasks I think there’s lots of prototyping work that doesn’t require hardware acceleration
    - jefft255 6y ago
      Coding somewhat locally is a must for me too because the cluster I have access to has pretty serious wait times (up to a couple hours on busy days). Imagine only being able to run the code you’re writing a few times a day at most! Iterative development and doing a lot of mistakes is how I code; I don’t want to go back to punch card days where you waited and waited before you ended up with a silly error.
  - tpetry 6y ago
    You can just run your training with a lot less of data.
  - jefft255 6y ago
    Not so sure about that. Here’s two things you can do (assuming you’re not training huge transformers or something). 1. Test your code with super low batch size. Bad for convergence, good for sanity check before submitting your job to a super computer. 2. Post-training evaluation. I’m pretty sure the M1 has enough power to do inference for not-so-big models. These two reasons are why I’m sometimes running stuff on my own GTX 1060, even though it’s pretty anemic and I wouldn’t actually do a training run there. There quite a bit of friction to training in the cloud, especially if it’s on shared cluster (which is what I have access to). You have a quota, and wait time when the supercomputer is under load. Sometimes you just need to quickly fire up something!
sjwright 6y ago
I think you underestimate just how many MacBooks Apple sells. There’s only so many affluent early adopters out there. And the M1 MacBooks are especially ideal for mainstream customers.
sumnuyungi 6y ago
This is just patently false. Most of the folks I know that have an M1 are students that were saving up to upgrade from a much older computer and got the M1 Air. I can assure you that they don't have a decent amount of money. Tensorflow has a branch that's optimized for metal with impressive performance. [1] It's fast enough to do transfer learning quickly on a large resnet, which is a common use-case for photo/video editing apps that have ML-powered workflows. It's best for everyone to do this locally: maintains privacy for the user and eliminates cloud costs for the developer. Also, not everyone has an imagenet sized dataset. A lot of applied ML uses small networks where prototyping is doable on a local machine. [1] https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...