4 ms·

After a cursory glance, my feeling is that "The Mathematics of Neuronal Networks" would be a better title? In recent years, the term "Artificial Intelligence"

by mg 5y ago

After a cursory glance, my feeling is that "The Mathematics of Neuronal Networks" would be a better title? In recent years, the term "Artificial Intelligence" is often used instead of "Neuronal Networks". I wouldn't be surprised if this will change again. If there is evidence that it will not and Neuronal Networks are for some reason the optimal medium for intelligence, I would love to read about it.

elcomet 5y ago
We usually say "neural networks" and not "neuronal networks"
- mdp2021 5y ago
  I think in Greek 'neura' and 'neuron' are equivalent. The perception that '-on' indicate the single or ideal entity ("ion, electron, muon...") comes from Whewell and Faraday ('-on' ex "ienai": the "goer" particles) and fully postdates the use of 'neuron'. Edit: also see the comment from member martopix, nearby ( https://news.ycombinator.com/item?id=30988150 https://news.ycombinator.com/item?id=30988150 )
  - tomrod 5y ago
    You have the correct etymology, but the convention is for "Neural Network" in ML and "Neuronal Network" in Neuroscience for we laymen.
    - mdp2021 5y ago
      As noted in the reference to the post from martopix :)
martopix 5y ago
Because AI basically "stole" the term neural networks from neuroscience, neuroscientists now often speak of "neuronal networks" while referring to biological ones, to avoid confusion. So I would recommend to not use "neuronal networks" in this context.
DataDaoDe 5y ago
Came here to say the same thing. This is not a paper on mathematical artificial intelligence, at last as I understand it, not in the way Goertzel or Hutter write about it. I was actually thinking this would be something like Hutter's universal artificial intelligence.
paskozdilar 5y ago
This is a very good point. To anyone interested in what Artificial Intelligence looked like before neural networks, the "Artificial Intelligence - A Modern Approach" by Stuart Russell and Peter Norvig is a good book on the subject.
- kriro 5y ago
  Agreed. I was teaching from it (AI 101, search/agents, no ML) and it's possibly the best textbook I have encountered. It's structured very well. As an aside, I will forever remember the rough geography of Romania. I still plan to do an A* route visit of the country some day :D
plafl 5y ago
I like to explain that AI > ML > NN. Now, NNs are the ones getting results at computer vision and natural language, and more. I think most people would say that other ML approaches are computational statistics. The goalpost for AI keeps moving. If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gentle intro would be Learning From Data [2]. If someone recommends a book/paper it would be awesome, I'm always on the look. [1] https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearning/ https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearni... [2] https://work.caltech.edu/telecourse https://work.caltech.edu/telecourse
- MrBusch 5y ago
  Seconding this. "Learning From Data" is one of the best intros to machine learning theory I've seen through the years!
- akomtu 5y ago
  NN > ML. The proof is that nematode or fly, I dont remember, with its simple NN fully mapped, and still remaining a mystery how it works. ML, which is just a matrix multiplication at its core, is a laughably simplistic model of NN.
  - soVeryTired 5y ago
    GP means artificial neural network, not an actual nervous system. Machine learning contains ANNs as a sub-discipline. Other non-ANN topics in ML include ensembled trees, Gaussian processes, and sampling theory.
  - jjtheblunt 5y ago
    linear-algebraic matrix multiplication complicated by non-linear threshold discontinuities evaluators, no?
  - kvathupo 5y ago
    I think one can't compare deep learning to machine learning since they have different purposes. While there have been great strides in the interpretability of NNs, the analytic models of classical machine learning are favored from a computational and interpretability perspective. Conversely, if the relationships in our data are constantly evolving (e.g. the change in the stochastic process followed by a time-series from one interval to another), then NNs are more appropriate.
mdp2021 5y ago
> Neuronal Networks are for some reason the optimal medium for intelligence No, that's not the same 'intelligence' ("General Intelligence") as the "I" side in Artificial Intelligence. The term 'intelligence' applied to Artificial Neural Networks makes sense, as such: to reach a procedural solution it takes an engineer; the engineer is said to have reached the solution because "intelligent"; ANNs are (semi-)automated builders of function approximators; ANNs are said "intelligent" because they "reach solutions" (like the engineer would have done - "approximator":"engineer"="Artificial":"Natural" Intelligence). And of course they are not "intelligent", while they are in some sense. It's just an expression, it's rhetoric (it requires considerate interpretation). That some sort of "intelligence" is achievable in other ways, or that ANNs spawn as an idea from anatomical considerations of naturally intelligent entities, or that ANNs could help in modelling general intelligence (etc.), is tangential.
- akomtu 5y ago
  ANNs dont have the key component of intelligence - ability to learn. If our brains worked like ANNs, they'd be wired once at the brains factory, they'd produce the same output given the same input, and learning anything new would need a complete rewiring at the factory, possibly erasing other learnings (often unpredictably). Even if ANNs could update themselves on the fly, in a reasonably incremental manner, they'd just imitate insect-level reactions. Intelligence would need those ANNs to have a virtual reality, run thought experiments there and learn from that.
  - mdp2021 5y ago
    Well this does not mean that some mechanisms similar to those we are studying cannot be part of the original or that they cannot be part of a General Intelligence system. Even if that were simply "adjusting a structure to function and information retention". The very "compositional" nature of the perceived that makes Convolutional ANNs work was noted after biomimicry. This, of course, as an aside.
  - sdenton4 5y ago
    You're asking for online learning. This is certainly possible with modern techniques - define an unsupervised loss and update the weights with gradients as new data is encountered. Federated learning also works along these lines. 'Raw' online learning is unpopular because you have no garauntee that the network won't do something funky in the field. That said, I think there are production systems in the world which learn on a day by day basis. Eg, take all of the logs from the last day and use them to update the production model for tomorrow. Then there's enough data that you don't risk a bad step. Think of it as learning while dreaming...

by mg 5y ago

elcomet 5y ago

We usually say "neural networks" and not "neuronal networks"

mdp2021 5y ago
I think in Greek 'neura' and 'neuron' are equivalent. The perception that '-on' indicate the single or ideal entity ("ion, electron, muon...") comes from Whewell and Faraday ('-on' ex "ienai": the "goer" particles) and fully postdates the use of 'neuron'. Edit: also see the comment from member martopix, nearby ( https://news.ycombinator.com/item?id=30988150 https://news.ycombinator.com/item?id=30988150 )
- tomrod 5y ago
  You have the correct etymology, but the convention is for "Neural Network" in ML and "Neuronal Network" in Neuroscience for we laymen.
  - mdp2021 5y ago
    As noted in the reference to the post from martopix :)

martopix 5y ago

Because AI basically "stole" the term neural networks from neuroscience, neuroscientists now often speak of "neuronal networks" while referring to biological ones, to avoid confusion. So I would recommend to not use "neuronal networks" in this context.

DataDaoDe 5y ago

Came here to say the same thing. This is not a paper on mathematical artificial intelligence, at last as I understand it, not in the way Goertzel or Hutter write about it. I was actually thinking this would be something like Hutter's universal artificial intelligence.

paskozdilar 5y ago

This is a very good point. To anyone interested in what Artificial Intelligence looked like before neural networks, the "Artificial Intelligence - A Modern Approach" by Stuart Russell and Peter Norvig is a good book on the subject.

kriro 5y ago
Agreed. I was teaching from it (AI 101, search/agents, no ML) and it's possibly the best textbook I have encountered. It's structured very well. As an aside, I will forever remember the rough geography of Romania. I still plan to do an A* route visit of the country some day :D

plafl 5y ago

I like to explain that AI > ML > NN. Now, NNs are the ones getting results at computer vision and natural language, and more. I think most people would say that other ML approaches are computational statistics. The goalpost for AI keeps moving. If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gentle intro would be Learning From Data [2]. If someone recommends a book/paper it would be awesome, I'm always on the look. [1] https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearning/ https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearni... [2] https://work.caltech.edu/telecourse https://work.caltech.edu/telecourse

MrBusch 5y ago
Seconding this. "Learning From Data" is one of the best intros to machine learning theory I've seen through the years!
akomtu 5y ago
NN > ML. The proof is that nematode or fly, I dont remember, with its simple NN fully mapped, and still remaining a mystery how it works. ML, which is just a matrix multiplication at its core, is a laughably simplistic model of NN.
- soVeryTired 5y ago
  GP means artificial neural network, not an actual nervous system. Machine learning contains ANNs as a sub-discipline. Other non-ANN topics in ML include ensembled trees, Gaussian processes, and sampling theory.
- jjtheblunt 5y ago
  linear-algebraic matrix multiplication complicated by non-linear threshold discontinuities evaluators, no?
- kvathupo 5y ago
  I think one can't compare deep learning to machine learning since they have different purposes. While there have been great strides in the interpretability of NNs, the analytic models of classical machine learning are favored from a computational and interpretability perspective. Conversely, if the relationships in our data are constantly evolving (e.g. the change in the stochastic process followed by a time-series from one interval to another), then NNs are more appropriate.

mdp2021 5y ago

> Neuronal Networks are for some reason the optimal medium for intelligence No, that's not the same 'intelligence' ("General Intelligence") as the "I" side in Artificial Intelligence. The term 'intelligence' applied to Artificial Neural Networks makes sense, as such: to reach a procedural solution it takes an engineer; the engineer is said to have reached the solution because "intelligent"; ANNs are (semi-)automated builders of function approximators; ANNs are said "intelligent" because they "reach solutions" (like the engineer would have done - "approximator":"engineer"="Artificial":"Natural" Intelligence). And of course they are not "intelligent", while they are in some sense. It's just an expression, it's rhetoric (it requires considerate interpretation). That some sort of "intelligence" is achievable in other ways, or that ANNs spawn as an idea from anatomical considerations of naturally intelligent entities, or that ANNs could help in modelling general intelligence (etc.), is tangential.

akomtu 5y ago
ANNs dont have the key component of intelligence - ability to learn. If our brains worked like ANNs, they'd be wired once at the brains factory, they'd produce the same output given the same input, and learning anything new would need a complete rewiring at the factory, possibly erasing other learnings (often unpredictably). Even if ANNs could update themselves on the fly, in a reasonably incremental manner, they'd just imitate insect-level reactions. Intelligence would need those ANNs to have a virtual reality, run thought experiments there and learn from that.
- mdp2021 5y ago
  Well this does not mean that some mechanisms similar to those we are studying cannot be part of the original or that they cannot be part of a General Intelligence system. Even if that were simply "adjusting a structure to function and information retention". The very "compositional" nature of the perceived that makes Convolutional ANNs work was noted after biomimicry. This, of course, as an aside.
- sdenton4 5y ago
  You're asking for online learning. This is certainly possible with modern techniques - define an unsupervised loss and update the weights with gradients as new data is encountered. Federated learning also works along these lines. 'Raw' online learning is unpopular because you have no garauntee that the network won't do something funky in the field. That said, I think there are production systems in the world which learn on a day by day basis. Eg, take all of the logs from the last day and use them to update the production model for tomorrow. Then there's enough data that you don't risk a bad step. Think of it as learning while dreaming...