9 ms·

Genetic Algorithms and Neural Networks are clearly inspired by the scientific fields of Genetics and Neuroscience.

by lev99 9y ago

kxyvr 9y ago
I'll disagree with this. Cybenko's approximation for a multilayer perceptron is simply a concrete expression for Kolmogorov's representation of a continuous function, which was created in order to solve Hilbert's Thirteenth problem. This has absolutely nothing to do with the brain. Yes, Cybenko uses the word "neural network", but just look at expression (1) in his paper "Approximation by Superpositions of a Sigmoidal Function" and compare that to expression (1) in Kolmogorov's paper "On the Representation of Continuous Functions of Many Variables by Superposition of Continuous Functions of One Variable and Addition". The brain diagram used by most neural network books is really just a graph diagram of a matrix-vector multiplication used in the superposition. While I can't claim to know everyone's inspiration for their research, there really is a mundane explanation and history of these algorithms that has nothing to do with the brain.
- bertil 9y ago
  As a data scientist dating a neuro-scientist, I can confirm that the two fields are quite apart in practice (except from the occasional image processing of fMRIs) — however, Cybenko’s work started in the late 70s; the inspiration for using a formalism of unit perceptrons organised in layers (as well as the convolution structure) comes from a paper on cat’s vision system published in 1962 [1]. As far as I’m aware, all key papers (Hinton, Le Cunn) cite Hubel & Wiesel as their explicit inspiration. The current practice has moved far from neuro-science, but I don’t think it’s pedantic when discussing machine learning to talk about “artificial neural networks” and acknowledge that neuroscience is still working on understanding the structure of brains. Some people, focused on their research, didn’t have that discretion. I wouldn’t imitate them. [1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/

Genetic Algorithms and Neural Networks are clearly inspired by the scientific fields of Genetics and Neuroscience.

by lev99 9y ago

Genetic Algorithms and Neural Networks are clearly inspired by the scientific fields of Genetics and Neuroscience.

I'll disagree with this. Cybenko's approximation for a multilayer perceptron is simply a concrete expression for Kolmogorov's representation of a continuous function, which was created in order to solve Hilbert's Thirteenth problem. This has absolutely nothing to do with the brain. Yes, Cybenko uses the word "neural network", but just look at expression (1) in his paper "Approximation by Superpositions of a Sigmoidal Function" and compare that to expression (1) in Kolmogorov's paper "On the Representation of Continuous Functions of Many Variables by Superposition of Continuous Functions of One Variable and Addition". The brain diagram used by most neural network books is really just a graph diagram of a matrix-vector multiplication used in the superposition. While I can't claim to know everyone's inspiration for their research, there really is a mundane explanation and history of these algorithms that has nothing to do with the brain.

bertil 9y ago
As a data scientist dating a neuro-scientist, I can confirm that the two fields are quite apart in practice (except from the occasional image processing of fMRIs) — however, Cybenko’s work started in the late 70s; the inspiration for using a formalism of unit perceptrons organised in layers (as well as the convolution structure) comes from a paper on cat’s vision system published in 1962 [1]. As far as I’m aware, all key papers (Hinton, Le Cunn) cite Hubel & Wiesel as their explicit inspiration. The current practice has moved far from neuro-science, but I don’t think it’s pedantic when discussing machine learning to talk about “artificial neural networks” and acknowledge that neuroscience is still working on understanding the structure of brains. Some people, focused on their research, didn’t have that discretion. I wouldn’t imitate them. [1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/