3 ms·

> neural networks are not smart That's like saying computers are not smart. It's meaningless. Computers are as smart as the algorithms they run and neural netw

by azag0 11y ago

> neural networks are not smart That's like saying computers are not smart. It's meaningless. Computers are as smart as the algorithms they run and neural networks are as smart as the neural connections within them. Neural networks are models of computation as much as current computer architectures are. As far as we know, neural networks indeed are at the core of human intelligence. Human brain is an extremely complicated neural network with plenty of side mechanisms, many of which we possibly do not know about yet. Nonetheless it's unlikely that the underlying building blocks are not neural networks. What is probably several orders of magnitude more complex than uncovering the static architecture of the human brain, is understanding the way it is trained. Note that this is the case with artificial neural networks as well. I doubt that we will ever understand human intelligence "from above". It doesn't seem implausible to me that there is some general complexity theory law which prohibits intelligence "understanding" itself.

dangirsh 11y ago
> It doesn't seem implausible to me that there is some general complexity theory law which prohibits intelligence "understanding" itself. What about collective intelligence? What could prevent many intelligent agents from collectively reverse-engineering the algorithms they are individually running? Let say they then build a simulator and can use it to predict how any possible instantiation of their intelligence would behave. Can they (collectively) "understand" it any more than that? What question could you pose to test if they really understood it "from above"?
YeGoblynQueenne 11y ago
>> As far as we know, neural networks indeed are at the core of human intelligence. Right. Quick intro to neural models, for the only very slightly mathematically inclined. Neural networks solve this equation: Y = W ⋅ X Where X a matrix of inputs to a function ƒ, W a matrix of its coefficients and Y a matrix of its outputs. Once that's done once for all N instances of X in our dataset, a time period which we term an epoch, Y is used to calculate a measure of error, typically mean squared error: MSE = 1/N ∑ (Y' - Y)² Where Y' the true value of Y that we're trying to approximate. This is repeated until the cows come home, or until the network has converged, whichever comes home first (er, hint). And that's neural networks in a nutshell. There's nothing much more complicated than that (er, in theory), there's no magic and certainly nothing "like the brain" in them at all. They're just optimisation over systems of functions. We learned how to solve systems of functions with matrix multiplication when we were at high school. I'm pretty sure we did - I've personally repressed the memory but I'm pretty sure I did once have it. Why do people insist on calling them "neural"? Who the fuck knows. Way back in the 50s, the perceptron was based on a model of neural activation, where a neuron "fires" when its electrical charge crosses over a threshold. This was originally represented with a binary step function (the very one used in Perceptron classification). That was, at the time, deemed to be "a good model" of neural activation. A good model, my foot. Soon people moved on to sigmoid functions which are "a better model" of neural activation. Whatever. I've also heard it that, say, conv nets (convolutional neural networks) are "like the brain", presumably because if you squint at the layer connection diagrams long enough, conv nets start to look a bit like idealised models of the parts of the brain that handle vision. But if you squint long enough, everything starts to look like everything else. Hell, you might suggest that neural networks are based on the effin Tree of Life from Cabala, and Geoff Hinton is Alistair Crowley reincarnated. The whole "brain" thing is just an abstraction. You should not read too much into it. And don't listen to the people who try to convince you neural networks are magic- they're not. It's high school maths and a lot of very expensive processing power. P.S. If my notation doesn't make sense to you, you may need to use a different font in your browser; sorry about that.
- p1esk 11y ago
  Brain has a large number of connected neurons. Which, by definition, is a "neural network". The artificial neural network model you described works more like a brain than, say, a traditional von Neumann computer. It works less like a brain than, say, HTM, which is another type of an artificial neural network. There are even more biologically plausible NN models (e.g. those used in HBP). The bottom line is, a large number of connected neurons is at the core of human intelligence. Everything else is just details.
  - YeGoblynQueenne 11y ago
    >> The bottom line is, a large number of connected neurons is at the core of human intelligence. Everything else is just details. That's a great example of the magical thinking that prevails in discussions of neural networks today. You only need to put your ducks in a row to figure out why it makes no sense. "The brain" you say "has lots of connections". "Therefore, if something has many connections, it's like a brain". I hope that raises a big red flag with bells on for you, 'cause it sure does for me. Spaghetti carbonara looks a lot like a brain (it's made of fibers and it's an off-white, sticky mess). I've never known it to be intelligent, though maybe it's just too smart for me, eh?

by azag0 11y ago

dangirsh 11y ago

> It doesn't seem implausible to me that there is some general complexity theory law which prohibits intelligence "understanding" itself. What about collective intelligence? What could prevent many intelligent agents from collectively reverse-engineering the algorithms they are individually running? Let say they then build a simulator and can use it to predict how any possible instantiation of their intelligence would behave. Can they (collectively) "understand" it any more than that? What question could you pose to test if they really understood it "from above"?

YeGoblynQueenne 11y ago

>> As far as we know, neural networks indeed are at the core of human intelligence. Right. Quick intro to neural models, for the only very slightly mathematically inclined. Neural networks solve this equation: Y = W ⋅ X Where X a matrix of inputs to a function ƒ, W a matrix of its coefficients and Y a matrix of its outputs. Once that's done once for all N instances of X in our dataset, a time period which we term an epoch, Y is used to calculate a measure of error, typically mean squared error: MSE = 1/N ∑ (Y' - Y)² Where Y' the true value of Y that we're trying to approximate. This is repeated until the cows come home, or until the network has converged, whichever comes home first (er, hint). And that's neural networks in a nutshell. There's nothing much more complicated than that (er, in theory), there's no magic and certainly nothing "like the brain" in them at all. They're just optimisation over systems of functions. We learned how to solve systems of functions with matrix multiplication when we were at high school. I'm pretty sure we did - I've personally repressed the memory but I'm pretty sure I did once have it. Why do people insist on calling them "neural"? Who the fuck knows. Way back in the 50s, the perceptron was based on a model of neural activation, where a neuron "fires" when its electrical charge crosses over a threshold. This was originally represented with a binary step function (the very one used in Perceptron classification). That was, at the time, deemed to be "a good model" of neural activation. A good model, my foot. Soon people moved on to sigmoid functions which are "a better model" of neural activation. Whatever. I've also heard it that, say, conv nets (convolutional neural networks) are "like the brain", presumably because if you squint at the layer connection diagrams long enough, conv nets start to look a bit like idealised models of the parts of the brain that handle vision. But if you squint long enough, everything starts to look like everything else. Hell, you might suggest that neural networks are based on the effin Tree of Life from Cabala, and Geoff Hinton is Alistair Crowley reincarnated. The whole "brain" thing is just an abstraction. You should not read too much into it. And don't listen to the people who try to convince you neural networks are magic- they're not. It's high school maths and a lot of very expensive processing power. P.S. If my notation doesn't make sense to you, you may need to use a different font in your browser; sorry about that.

p1esk 11y ago
Brain has a large number of connected neurons. Which, by definition, is a "neural network". The artificial neural network model you described works more like a brain than, say, a traditional von Neumann computer. It works less like a brain than, say, HTM, which is another type of an artificial neural network. There are even more biologically plausible NN models (e.g. those used in HBP). The bottom line is, a large number of connected neurons is at the core of human intelligence. Everything else is just details.
- YeGoblynQueenne 11y ago
  >> The bottom line is, a large number of connected neurons is at the core of human intelligence. Everything else is just details. That's a great example of the magical thinking that prevails in discussions of neural networks today. You only need to put your ducks in a row to figure out why it makes no sense. "The brain" you say "has lots of connections". "Therefore, if something has many connections, it's like a brain". I hope that raises a big red flag with bells on for you, 'cause it sure does for me. Spaghetti carbonara looks a lot like a brain (it's made of fibers and it's an off-white, sticky mess). I've never known it to be intelligent, though maybe it's just too smart for me, eh?