4 ms·
What ML are people actually wanting to do on M1 laptops? I was under the impression that outside toy problems, you can't really get decent results unless you h
by londons_explore 5y ago
What ML are people actually wanting to do on M1 laptops?
I was under the impression that outside toy problems, you can't really get decent results unless you have a TPU or fleet of GPU's for training. If you just train on a laptop you're shooting yourself in the foot compared to someone who gets the same job done 10x faster in the cloud (usually with free credits).
Obviously inference is still possible on a laptop for many models... but not many 'hacker' tasks involve only inference.
- shimmeringleaf 5y agoTotally, performance for actual useful training etc. is at least a couple of hardware generations away. This is super useful however for people like me who like to use an m1 Mac as their daily driver and don’t always want to have to fire up a remote rig just to sketch a little. Also being able to do some local inference can be great for when building a web GUI, etc. So absolutely not mature, but having PyTorch working better also on macs is only positive imho.
- andybak 5y agoThe whole Diffusion text to image thing which is hugely popular right now doesn't involve any training. Same for text generation where you download pre-trained models. I'm using a laptop with a 3080 and getting much faster results than I typically get from a paid Colab account (not Pro mind you).
- natly 5y agoWhich repo/tool are you using for local text to image diffusion if you don't mind sharing? :)
- andybak 5y agoNot at all. I discovered that Visions of Chaos (originally developed for fractals etc) has extensive support for installing and running ML models https://softology.pro/tutorials/tensorflow/tensorflow.htm https://softology.pro/tutorials/tensorflow/tensorflow.htm It handles (most) of the installation issues and gives you a GUI. You can however bypass the GUI. Every time you run a command the console shows you what the command line looks like so copy/paste into a terminal and away you go.
- Tenoke 5y ago>I was under the impression that outside toy problems, you can't really get decent results unless you have a TPU or fleet of GPU's for training. This is most certainly not the case. It's impossible to achieve SOTA on most benchmarks but there's a lot more ML and data analysis done on smaller datasets where you don't particularly need a fleet of GPUs.
- nl 5y agoI've fine tuned transformers on CPUs before for weird reasons. Sometimes the computer you have is what you make do with.
- visionscaper 5y agoI’m not using an M1 machine yet, but I think is extremely useful to be able to do some small scale experiments for testing and debugging purposes locally before starting a full-scale training job in the cloud. Further, in general, I’m really charmed by the potential of having unified memory; the idea you can test some batch training iterations with a batch size that fills almost all 128MB is a unique capability! Last, a large amount of unified memory allows to do inference and “prompt engineering” with very large models, locally. E.g. Using GPT-J like models (6B parameters).
- microtonal 5y agoWhat ML are people actually wanting to do on M1 laptops? (1) development - quickly train smaller networks locally to validate the code. Then offload to a big GPU machine. (2) Smaller convolutional networks and RNNs may be below SOTA, but can still provide a good performance-accuracy trade-off. You could already train these networks pretty fast on an M1 thanks to AMX, but his makes training such networks even faster.
- singularity2001 5y agoIdeally fine-tuning foundation models. Training them costs several millions, so out of scope no matter which GPU you have.
- arecurrence 5y agoThere's plenty of training related stuff that 10 TFlops is sufficient for. Fine tuning models for example is a great use case for a high ram yet relatively low TFlops environment. I don't know how much you've done with running ml jobs on cloud boxes but it can be very annoying to work with them when you are actively iterating. Especially if you are doing a lot of work outside the cloud box environment. Even turn key consumer products like Colab can fall over if you accidentally press "Back" in your browser... I once lost many hours of work because I inadvertently swiped left :) Apple will continue improving their performance in this space and once jobs that run for hours can be run overnight... I suspect people will often choose to run overnight simply for the ergonomics benefit.