5 ms·
Does anyone doing serious commercial grade ML actually train models on their macbook?
by deeeeplearning 6y ago
Does anyone doing serious commercial grade ML actually train models on their macbook?
- fxtentacle 6y agoSurely not. I'm working on a PC with 64GB of main RAM and 12GB of dedicated GPU RAM and I already have to do tricks to be able to squeeze state of the art models into my limited memory. For comparison, NVIDIA just upgraded from 40GB to 80GB.
- vimy 6y agoUnified memory on Apple silicon means the gpu/neural engine has as much ram as the mac. A future mbp with 64 gb ram will be able to fit a lot of models. A whole ML laptop for the price of one gpu.
- fxtentacle 6y agoAgree, if Apple ever goes back to offering 64+ GB RAM Notebooks, they will become a viable choice again.
- simonh 6y agoContrary to the way you and a lot of people have been spinning things recently, Apple has not discontinued the 16" Macbook Pro, and you can buy an Apple laptop today in all their form factors with just as much memory as you could a few weeks ago. The M1 machines are only replacing models that only went up to 16 GB already.
- tantalor 6y agoI thought ML models were tiny, small enough to run on mobile devices.
- nl 6y agoMost ML models can be shrunk at the cost of some accuracy, but some models are extremely large.
- m463 6y agoI believe you have confused training and inference. Training usually uses large amounts of data to get your system to recognize a pattern. It generally uses huge memory and compute and generates a model. Inference will use that generated model to recognize the pattern in new data. It uses significantly fewer resources and can run either very fast or on a smaller system. I believe the speed of inference may be affected by the resources available during training, where more speed/memory for training can produce better models.
- tantalor 6y agoI think it's parent commenter who is confused, complains how hard it is to "squeeze state of the art models into my limited memory", which sounds like inference.
- deleted 6y ago[deleted]
- therealmarv 6y agoYeah when you've a trained model you can copy it even to mobile devices.
- site-packages1 6y agoThis would depend on the model
- deeviant 6y agoVery, very much depends on the model.
- grandmczeb 6y agoProbably very few if any. I think the main use cases would be convenience for playing around with things locally and students.
- nojito 6y agoOn device ML is better for privacy.
- grecy 6y agoNot yet, but the fact this post exists, that Apple have worked to accelerate TensorFlow, that the Apple Silicon specifically has hardware for these workloads, and they're specifically highlighting the performance on the Mac Pro leads me to think people might be soon. Let's wait and see what Apple Silicon has in store for the iMac Pro and Mac Pro.
- megablast 6y agoI use it to make simple models for apps, because that is what I have.
- roseway4 6y agoWhen prototyping or developing new model code, yes. If you’re building production ML systems, there’s plenty code to write and test before deploying to prod infrastructure. To be clear, you would work with a very small data sample or synthetic data as your objective isn’t to train a model for production use. Edit: clarity and grammar
- skwb 6y agoYeah, exactly this. I mostly am using cloud VMs for training, but sometimes just need to be able to mess around on my local machine for my deployment pipeline. I don't really care if it takes ~10 seconds to do inference on a handful of images, I really just need to tinker with the interfaces to ensure everything is hooked up correctly.
- raverbashing 6y agoIf you think "commercial grade ML" means only big models, that's the wrong way to think already. You don't have to throw the biggest model you can at any problem you have.
- deeeeplearning 6y agoSince this is referencing Tensorflow performance that implies Deep Learning models which are anything but small so I'm not sure I see your point.