3 ms·

Can someone say a bit about minimum number of layers and neurons needed for structure to be called artificial neural network? I have seen papers with something

Can someone say a bit about minimum number of layers and neurons needed for structure to be called artificial neural network? I have seen papers with something like 10 neurons (which is very very tiny but authors claim it does the job), while on the other hand there are Google sized ANNs. Thanks :)

florianletsch 10y ago
You can call it an artificial neural network (ANN) as soon as you have two or more artifical neurons connected to each other. As simple as that. A small amount of neurons might already solve some problem you're having. The XOR problem can be learned by 4 neurons connected to each other. When you want raw images or similar as an input and have it be classified into 100 classes (e.g. look up CIFAR-10 or CIFAR-100), you will need an architecture with many more neurons. After all, ANN are simply a tool. Depending on the task, that tool might need to more elaborate. And when you have all those different possible architectures, you want a common way of naming them. Labels such as Deep Learning are simply nomenclature of talking about certain groups of artificial neural networks.
PeterisP 10y ago
ANN is a description about structure, not size. XOR (an input layer of 2 neurons, 1 hidden layer of usually 2 neurons, and an output layer of 1 neuron) is a classic toy example of an ANN. The minimal would probably be something like a NOT gate - input layer with 1 neuron connected directly to the output layer with 1 neuron. In practice you either use whatever size your problem needs (for small problems) or match the network size to match the RAM amount in the video cards you're using.
rasbt 10y ago
Technically, you can say that even a single perceptron unit is a artificial neural network -- a single-layer ANN. Multi-layer neural networks (such as a multi-layer perceptron) have at least one input layer, one hidden layer, and one output layer.
MichailP 10y ago
Thanks for all replies. For some reason I felt that ANN has to be large to perform something useful, and small ~10 neurons ANNs seemed like a toy compared to ~1e6 nets. But, as you pointed out, if it does the job then it is good enough, no matter the count of neurons.

florianletsch 10y ago

You can call it an artificial neural network (ANN) as soon as you have two or more artifical neurons connected to each other. As simple as that. A small amount of neurons might already solve some problem you're having. The XOR problem can be learned by 4 neurons connected to each other. When you want raw images or similar as an input and have it be classified into 100 classes (e.g. look up CIFAR-10 or CIFAR-100), you will need an architecture with many more neurons. After all, ANN are simply a tool. Depending on the task, that tool might need to more elaborate. And when you have all those different possible architectures, you want a common way of naming them. Labels such as Deep Learning are simply nomenclature of talking about certain groups of artificial neural networks.

PeterisP 10y ago

ANN is a description about structure, not size. XOR (an input layer of 2 neurons, 1 hidden layer of usually 2 neurons, and an output layer of 1 neuron) is a classic toy example of an ANN. The minimal would probably be something like a NOT gate - input layer with 1 neuron connected directly to the output layer with 1 neuron. In practice you either use whatever size your problem needs (for small problems) or match the network size to match the RAM amount in the video cards you're using.

rasbt 10y ago

Technically, you can say that even a single perceptron unit is a artificial neural network -- a single-layer ANN. Multi-layer neural networks (such as a multi-layer perceptron) have at least one input layer, one hidden layer, and one output layer.

MichailP 10y ago

Thanks for all replies. For some reason I felt that ANN has to be large to perform something useful, and small ~10 neurons ANNs seemed like a toy compared to ~1e6 nets. But, as you pointed out, if it does the job then it is good enough, no matter the count of neurons.