3 ms·
Isn't it better to rent a cloud with as many GPUs as necessary for a time needed to train the model? I don't know state of things in ML.
by vbezhenar 6y ago
Isn't it better to rent a cloud with as many GPUs as necessary for a time needed to train the model? I don't know state of things in ML.
- nicdc 6y agoCurious to hear responses to this too..
- nelsondev 6y agoFor productionizing/training massive models, yes. But in the development phase, when you are testing on a smaller corpus of data, to make sure your code works, the on-laptop dedicated chip could expedite the development process.
- PeterisP 6y agoI agree with the parent poster that it's probably more about inference, not training. If ML developers can assume that consumer machines (at least "proper consumer machines, like those made by Apple") will have support to do small-scale ML calculations efficiently, then that enables including various ML-based thingies in random consumer apps.
- FridgeSeal 6y agoNot necessarily. It can be surprisingly cost-effective to invest a few $k in a hefty machine(s) with some high-end GPU's to train with due to the exceedingly hefty price of cloud GPU compute. The money invested up-front in the machine(s) pays itself off in (approximately) a couple of months. The "neural" chips in these machines are for accelerating inference. I.e. you already have a trained model, you quantise and shrink it, export it to ONNX or whatever Apple's CoreML requires, ship it to the client, and then it runs extra-fast, with relatively small power draw on the client machine due to the dedicated/specialised hardware.
- imtringued 6y agoCloud GPU instances are very expensive. If you get consumer GPUs not only do you save money, you can sell them afterwards for 50% of the purchasing price.