5 ms·

How feasible is CPU only deep learning? I keep hearing about outrageous training times for GPUs on the order of weeks with something like 4-8 GPUs, is anyone ac

How feasible is CPU only deep learning? I keep hearing about outrageous training times for GPUs on the order of weeks with something like 4-8 GPUs, is anyone actually using CPUs instead?

tux3 9y ago
My (limited) understanding is that 4-8 GPUs will be around 2 orders of magnitude faster than a CPU at training, and that it's not very realistic to use a CPU for training anything but the smallest models.
nl 9y ago
It's slower but can be cheaper for cloud training. See http://minimaxir.com/2017/07/cpu-or-gpu/ http://minimaxir.com/2017/07/cpu-or-gpu/ Those long training times are for training from scratch. Most image tasks don't need that, although text tasks often do.
deleted 9y ago
[deleted]
Mr_P 9y ago
> I keep hearing about outrageous training times for GPUs on the order of weeks with something like 4-8 GPUs This is for training a competitive model from scratch on a fundamental problem like image recognition. If you don't care about the last 1-2%, it's possible to train a useful model in a few hours (but still on a GPU). > is anyone actually using CPUs instead There are useful things you can do without a GPU. For example, "Transfer learning", which can be as simple as chopping off the last layer of someone else's GPU-trained model and substituting your own, can be done on a CPU in reasonable time. This is because you typically need less data and because far fewer parameters need to be fit. For example, HBO's "Hotdog/Not-hotdog" app was done this way. See their description of "V1" here: https://medium.com/@timanglade/how-hbos-silicon-valley-built-not- https://medium.com/@timanglade/how-hbos-silicon-valley-built...
mljoe 9y ago
I wouldn't recommend for training. CPU's work just fine for low intensity inference. Especially online/interactive inference, these are situations where you pass in one example at a time. GPU excel when you pass data in continuous predictable batches.
deleted 9y ago
[deleted]
alfalfasprout 9y ago
Well... I feel like people are only focusing on training. CPUs for inference are sometimes helpful in online inference settings where you don't ever really "batch" requests. In this case, they can be cheap.

How feasible is CPU only deep learning? I keep hearing about outrageous training times for GPUs on the order of weeks with something like 4-8 GPUs, is anyone actually using CPUs instead?

tux3 9y ago

My (limited) understanding is that 4-8 GPUs will be around 2 orders of magnitude faster than a CPU at training, and that it's not very realistic to use a CPU for training anything but the smallest models.

nl 9y ago

It's slower but can be cheaper for cloud training. See http://minimaxir.com/2017/07/cpu-or-gpu/ http://minimaxir.com/2017/07/cpu-or-gpu/ Those long training times are for training from scratch. Most image tasks don't need that, although text tasks often do.

deleted 9y ago

[deleted]

Mr_P 9y ago

> I keep hearing about outrageous training times for GPUs on the order of weeks with something like 4-8 GPUs This is for training a competitive model from scratch on a fundamental problem like image recognition. If you don't care about the last 1-2%, it's possible to train a useful model in a few hours (but still on a GPU). > is anyone actually using CPUs instead There are useful things you can do without a GPU. For example, "Transfer learning", which can be as simple as chopping off the last layer of someone else's GPU-trained model and substituting your own, can be done on a CPU in reasonable time. This is because you typically need less data and because far fewer parameters need to be fit. For example, HBO's "Hotdog/Not-hotdog" app was done this way. See their description of "V1" here: https://medium.com/@timanglade/how-hbos-silicon-valley-built-not- https://medium.com/@timanglade/how-hbos-silicon-valley-built...

mljoe 9y ago

I wouldn't recommend for training. CPU's work just fine for low intensity inference. Especially online/interactive inference, these are situations where you pass in one example at a time. GPU excel when you pass data in continuous predictable batches.

deleted 9y ago

[deleted]

alfalfasprout 9y ago

Well... I feel like people are only focusing on training. CPUs for inference are sometimes helpful in online inference settings where you don't ever really "batch" requests. In this case, they can be cheap.