4 ms·

AI even has this problem among it's researchers. What the fuck is a neural network?

by ouid 9y ago

wuliwong 9y ago
I was also curious about the use of 'neural networks' in deep learning or whatever. My PhD research was working on problems in computational neuroscience. I think the use of "neural networks" is because the topology of these deep learning networks is similar to what is observed in biological brains.
- mayank 9y ago
  Brain networks and neural networks are not similar in structure or function, other than the superficial abstraction of “nodes and connections” (graphs). They’re about as related as genetic algorithms are to actual genetics.
  - aje403 9y ago
    Which is what he's saying, the use of the term is because of the extremely-vague-homeomorphism implied by the structure Unfortunately, the word choice can lead to "they've gots em a neural networks and an AI singularities and them robots coming to take over"
    - Maybestring 9y ago
      >extremely-vague-homeomorphism Analogy is the concept in its full generality.
      - aje403 9y ago
        If only. Analogy is more like the bastardization of an Ideal Form
        Maybestring 9y ago
        Thats a very foreign view of the concept for me. I think of analogy as a non-rigorous functor. A mapping between concepts and relationships among concepts in one context to concepts and relationships among concepts in another context.
  - lev99 9y ago
    Genetic Algorithms and Neural Networks are clearly inspired by the scientific fields of Genetics and Neuroscience.
    - kxyvr 9y ago
      I'll disagree with this. Cybenko's approximation for a multilayer perceptron is simply a concrete expression for Kolmogorov's representation of a continuous function, which was created in order to solve Hilbert's Thirteenth problem. This has absolutely nothing to do with the brain. Yes, Cybenko uses the word "neural network", but just look at expression (1) in his paper "Approximation by Superpositions of a Sigmoidal Function" and compare that to expression (1) in Kolmogorov's paper "On the Representation of Continuous Functions of Many Variables by Superposition of Continuous Functions of One Variable and Addition". The brain diagram used by most neural network books is really just a graph diagram of a matrix-vector multiplication used in the superposition. While I can't claim to know everyone's inspiration for their research, there really is a mundane explanation and history of these algorithms that has nothing to do with the brain.
      - bertil 9y ago
        As a data scientist dating a neuro-scientist, I can confirm that the two fields are quite apart in practice (except from the occasional image processing of fMRIs) — however, Cybenko’s work started in the late 70s; the inspiration for using a formalism of unit perceptrons organised in layers (as well as the convolution structure) comes from a paper on cat’s vision system published in 1962 [1]. As far as I’m aware, all key papers (Hinton, Le Cunn) cite Hubel & Wiesel as their explicit inspiration. The current practice has moved far from neuro-science, but I don’t think it’s pedantic when discussing machine learning to talk about “artificial neural networks” and acknowledge that neuroscience is still working on understanding the structure of brains. Some people, focused on their research, didn’t have that discretion. I wouldn’t imitate them. [1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/
lev99 9y ago
Neural Network describes a class of algorithms. It's often misused in marketing (buzzword), but there is an engineering definition used by people that practice machine learning. A perceptron, the basic unit in a neural network, has one or more inputs, an algorithm for processing the input, and one or more outputs. Layers of perceptron are often used so that the original input gets transformed over each layer. The most common output for a neural network is a classification. The major variants of neural networks are ways a perceptron's algorithms change, the types of algorithms inside a perceptron, the number of layers and the number of inputs/outputs. Neural Networks are computationally heavy to train, but not computationally heavy to use once created. They are specifically good for working with image data. http://natureofcode.com/book/chapter-10-neural-networks/ http://natureofcode.com/book/chapter-10-neural-networks/
- ouid 9y ago
  Sorry, I'm not actually confused as to how the word neural network is used. I'm saying that it's stupid nomenclature. But thanks for clarifying anyway :).
bertil 9y ago
Are you referring to the distinction between the software frameworks known as an artificial neural networks and biology, specifically neuroscience?

AI even has this problem among it's researchers. What the fuck is a neural network?

by ouid 9y ago

AI even has this problem among it's researchers. What the fuck is a neural network?

I was also curious about the use of 'neural networks' in deep learning or whatever. My PhD research was working on problems in computational neuroscience. I think the use of "neural networks" is because the topology of these deep learning networks is similar to what is observed in biological brains.

mayank 9y ago
Brain networks and neural networks are not similar in structure or function, other than the superficial abstraction of “nodes and connections” (graphs). They’re about as related as genetic algorithms are to actual genetics.
- aje403 9y ago
  Which is what he's saying, the use of the term is because of the extremely-vague-homeomorphism implied by the structure Unfortunately, the word choice can lead to "they've gots em a neural networks and an AI singularities and them robots coming to take over"
  - Maybestring 9y ago
    >extremely-vague-homeomorphism Analogy is the concept in its full generality.
    - aje403 9y ago
      If only. Analogy is more like the bastardization of an Ideal Form
      - Maybestring 9y ago
        Thats a very foreign view of the concept for me. I think of analogy as a non-rigorous functor. A mapping between concepts and relationships among concepts in one context to concepts and relationships among concepts in another context.
- lev99 9y ago
  Genetic Algorithms and Neural Networks are clearly inspired by the scientific fields of Genetics and Neuroscience.
  - kxyvr 9y ago
    I'll disagree with this. Cybenko's approximation for a multilayer perceptron is simply a concrete expression for Kolmogorov's representation of a continuous function, which was created in order to solve Hilbert's Thirteenth problem. This has absolutely nothing to do with the brain. Yes, Cybenko uses the word "neural network", but just look at expression (1) in his paper "Approximation by Superpositions of a Sigmoidal Function" and compare that to expression (1) in Kolmogorov's paper "On the Representation of Continuous Functions of Many Variables by Superposition of Continuous Functions of One Variable and Addition". The brain diagram used by most neural network books is really just a graph diagram of a matrix-vector multiplication used in the superposition. While I can't claim to know everyone's inspiration for their research, there really is a mundane explanation and history of these algorithms that has nothing to do with the brain.
    - bertil 9y ago
      As a data scientist dating a neuro-scientist, I can confirm that the two fields are quite apart in practice (except from the occasional image processing of fMRIs) — however, Cybenko’s work started in the late 70s; the inspiration for using a formalism of unit perceptrons organised in layers (as well as the convolution structure) comes from a paper on cat’s vision system published in 1962 [1]. As far as I’m aware, all key papers (Hinton, Le Cunn) cite Hubel & Wiesel as their explicit inspiration. The current practice has moved far from neuro-science, but I don’t think it’s pedantic when discussing machine learning to talk about “artificial neural networks” and acknowledge that neuroscience is still working on understanding the structure of brains. Some people, focused on their research, didn’t have that discretion. I wouldn’t imitate them. [1] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523/

lev99 9y ago

Neural Network describes a class of algorithms. It's often misused in marketing (buzzword), but there is an engineering definition used by people that practice machine learning. A perceptron, the basic unit in a neural network, has one or more inputs, an algorithm for processing the input, and one or more outputs. Layers of perceptron are often used so that the original input gets transformed over each layer. The most common output for a neural network is a classification. The major variants of neural networks are ways a perceptron's algorithms change, the types of algorithms inside a perceptron, the number of layers and the number of inputs/outputs. Neural Networks are computationally heavy to train, but not computationally heavy to use once created. They are specifically good for working with image data. http://natureofcode.com/book/chapter-10-neural-networks/ http://natureofcode.com/book/chapter-10-neural-networks/

ouid 9y ago
Sorry, I'm not actually confused as to how the word neural network is used. I'm saying that it's stupid nomenclature. But thanks for clarifying anyway :).

bertil 9y ago

Are you referring to the distinction between the software frameworks known as an artificial neural networks and biology, specifically neuroscience?