4 ms·

Question: if I'm learning about neural networks and want to e.g. train a network to recognize MNIST digits, do I need a discrete graphics card (probably attache

Question: if I'm learning about neural networks and want to e.g. train a network to recognize MNIST digits, do I need a discrete graphics card (probably attached to a VPS that I would rent)? Or can I use the i5 Kaby Lake (which has an integrated GPU) in my laptop to train my network?

ktta 9y ago
You actually don't even need the iGPU for simple MNIST training. The CPU is enough. Check out the sample MNIST training example with Tensorflow. If you are asking about using the iGPU for a bit more power, most frameworks don't support them. The only frameworks with which you can use with iGPUs are which have OpenCL support, like caffe[1] (even for those you need to have a recent enough CPU) Tensorflow either uses CPU or CUDA (Not even AMD, but the support is coming for openCL) [1]:https://github.com/BVLC/caffe/tree/opencl https://github.com/BVLC/caffe/tree/opencl
daveguy 9y ago
To train an NN to recognize MNIST digits you need any cpu made in the last 5 years. Last 10-20 years if you want to wait a little longer. MNIST recognition was first done by convolutional neural network in the late 90s. With less than 1% error. Here's a link to the paper: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf
p1esk 9y ago
MNIST is a toy problem. It shouldn't take you more than a few days or even hours to completely understand it. Do yourself a favor and get a GPU because soon you will want to try bigger networks on real world images. 1060, 1070, or 1080Ti are all good choices, depending on your budget and ambitions.
solomatov 9y ago
Simple network form tensroflow's MNIST tutorial trains in about 30 minutes on my macbook pro.
marvy 9y ago
As other people have said, for MNIST, CPU is fine. One core. You can get over 90% accuracy with a network with just 1 hidden layer with only a few seconds of training. Or something like that. If you've never played with MNIST before, it's kind of amazing how easy it is. For instance, the following idea "works", in that you get results that are pretty bad, but much better than chance. (Even more than 50% right, I think.) Suppose you have 10 arrays: zeros: each element of this array is a picture of a "0" ones: each element is a picture of a "1" ... nines: each element is a "9" Compute the average of each array: the "average" 0, the average 1, and so on. Then classify new digits based on which average element they are closest to, using Euclidean distance. This sounds way too dumb to do any good, but the MNIST digits are normalized so well that this actually does something. Even better: you can have a neural "network" that has zero hidden layers. This actually achieves almost respectable performance, believe it or not.
- huac 9y ago
  A 0 hidden layer NN is just a linear (or logistic) regression. On MNIST I think you get something like 60% accuracy.
  - marvy 9y ago
    Yes, it is just logistic regression. I just dug up my old code from a few weeks (months?) ago, and it seems that somewhere around 90% is not too hard. 60% is way too low, unless I'm measuring wrong somehow.
sabalaba 9y ago
You can buy a 1 GPU machine with an i5 for $2,450 from Lambda Labs.

ktta 9y ago

You actually don't even need the iGPU for simple MNIST training. The CPU is enough. Check out the sample MNIST training example with Tensorflow. If you are asking about using the iGPU for a bit more power, most frameworks don't support them. The only frameworks with which you can use with iGPUs are which have OpenCL support, like caffe[1] (even for those you need to have a recent enough CPU) Tensorflow either uses CPU or CUDA (Not even AMD, but the support is coming for openCL) [1]:https://github.com/BVLC/caffe/tree/opencl https://github.com/BVLC/caffe/tree/opencl

daveguy 9y ago

To train an NN to recognize MNIST digits you need any cpu made in the last 5 years. Last 10-20 years if you want to wait a little longer. MNIST recognition was first done by convolutional neural network in the late 90s. With less than 1% error. Here's a link to the paper: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf

p1esk 9y ago

MNIST is a toy problem. It shouldn't take you more than a few days or even hours to completely understand it. Do yourself a favor and get a GPU because soon you will want to try bigger networks on real world images. 1060, 1070, or 1080Ti are all good choices, depending on your budget and ambitions.

solomatov 9y ago

Simple network form tensroflow's MNIST tutorial trains in about 30 minutes on my macbook pro.

marvy 9y ago

As other people have said, for MNIST, CPU is fine. One core. You can get over 90% accuracy with a network with just 1 hidden layer with only a few seconds of training. Or something like that. If you've never played with MNIST before, it's kind of amazing how easy it is. For instance, the following idea "works", in that you get results that are pretty bad, but much better than chance. (Even more than 50% right, I think.) Suppose you have 10 arrays: zeros: each element of this array is a picture of a "0" ones: each element is a picture of a "1" ... nines: each element is a "9" Compute the average of each array: the "average" 0, the average 1, and so on. Then classify new digits based on which average element they are closest to, using Euclidean distance. This sounds way too dumb to do any good, but the MNIST digits are normalized so well that this actually does something. Even better: you can have a neural "network" that has zero hidden layers. This actually achieves almost respectable performance, believe it or not.

huac 9y ago
A 0 hidden layer NN is just a linear (or logistic) regression. On MNIST I think you get something like 60% accuracy.
- marvy 9y ago
  Yes, it is just logistic regression. I just dug up my old code from a few weeks (months?) ago, and it seems that somewhere around 90% is not too hard. 60% is way too low, unless I'm measuring wrong somehow.

sabalaba 9y ago

You can buy a 1 GPU machine with an i5 for $2,450 from Lambda Labs.