4 ms·
Surely not. I'm working on a PC with 64GB of main RAM and 12GB of dedicated GPU RAM and I already have to do tricks to be able to squeeze state of the art model
by fxtentacle 6y ago
Surely not. I'm working on a PC with 64GB of main RAM and 12GB of dedicated GPU RAM and I already have to do tricks to be able to squeeze state of the art models into my limited memory.
For comparison, NVIDIA just upgraded from 40GB to 80GB.
- vimy 6y agoUnified memory on Apple silicon means the gpu/neural engine has as much ram as the mac. A future mbp with 64 gb ram will be able to fit a lot of models. A whole ML laptop for the price of one gpu.
- fxtentacle 6y agoAgree, if Apple ever goes back to offering 64+ GB RAM Notebooks, they will become a viable choice again.
- simonh 6y agoContrary to the way you and a lot of people have been spinning things recently, Apple has not discontinued the 16" Macbook Pro, and you can buy an Apple laptop today in all their form factors with just as much memory as you could a few weeks ago. The M1 machines are only replacing models that only went up to 16 GB already.
- tantalor 6y agoI thought ML models were tiny, small enough to run on mobile devices.
- nl 6y agoMost ML models can be shrunk at the cost of some accuracy, but some models are extremely large.
- m463 6y agoI believe you have confused training and inference. Training usually uses large amounts of data to get your system to recognize a pattern. It generally uses huge memory and compute and generates a model. Inference will use that generated model to recognize the pattern in new data. It uses significantly fewer resources and can run either very fast or on a smaller system. I believe the speed of inference may be affected by the resources available during training, where more speed/memory for training can produce better models.
- tantalor 6y agoI think it's parent commenter who is confused, complains how hard it is to "squeeze state of the art models into my limited memory", which sounds like inference.
- deleted 6y ago[deleted]
- therealmarv 6y agoYeah when you've a trained model you can copy it even to mobile devices.
- site-packages1 6y agoThis would depend on the model
- deeviant 6y agoVery, very much depends on the model.