3 ms·
it's not really mimicking how our brains work, although that was the initial inspiration, decades ago. There are still people trying to model the actual brain b
by currymj 8y ago
it's not really mimicking how our brains work, although that was the initial inspiration, decades ago. There are still people trying to model the actual brain but they don't overlap much with the machine learning community anymore.
the idea originally was that a neuron can be roughly modeled as something that sums up inputs from other neurons and then outputs some kind of threshold/activation of those inputs. this is probably sort of true of biological neurons although as living cells they are much more complicated than that.
but lots of recent and highly successful techniques in deep learning have given up on even that very weak amount of biological imitation. One simple example: lots of techniques involve multiplying together the outputs of "neurons". There is no sensible biological interpretation of this operation.
Neural networks are Turing-complete (assuming all the usual pedantic assumptions you need to make that statement meaningful) so in that sense they are as powerful as our brains.
- danmaz74 8y agoBeing Turing complete makes them as powerful as any other digital computer, but it has little (personally I would say nothing) to do with our brain.
- currymj 8y agoWhy not? A human given enough time and an infinite tape can easily simulate a Turing machine. The other way is more controversial, but I don't think it's outrageous to speculate that a brain could be accurately simulated by a Turing machine.
- danmaz74 8y agoMy point is that we know for a fact that our brain doesn't work anything like a Turing machine. Can a Turing machine simulate a brain - as in, create intelligence like the one our brains expose? That's very much an open question, but you wrote as if it was an obvious conclusion.
- currymj 8y agoIt's a pretty common assumption that any physical process could be simulated by a Turing machine (given infinite time and space etc.). We could probably never build such a Turing machine. I didn't mean to make any stronger claims than that. As I mentioned before, this is a pretty pedantic way of looking at the world.
- danmaz74 8y ago> We could probably never build such a Turing machine. I didn't mean to make any stronger claims than that Fair enough. I got a different impression from reading your original comment :)
- edanm 8y ago> My point is that we know for a fact that our brain doesn't work anything like a Turing machine. That's of course true, but then most computers aren't that similar to a classic Turing machine either. The idea of Turing machines is that they can simulate anything else that we mean when we say "computation". > Can a Turing machine simulate a brain - as in, create intelligence like the one our brains expose? Yes, it can. I mean, if it can't, that means there's some notion of "computability" that a Turing machine doesn't capture - this would violate the "Church-Turing" thesis. Such a thing is possible, of course, since we have no proof that a Turing machine is the limit of what we mean by "computable" - but it would be a completely revolutionary fact in Computer Science, and one that is very unlikely, considering how many years people have been trying to come up with alternative models of "computation" that all turn out to be equivalent to Turing machines. - The human brain isn't magic - it works according to the laws of physics like anything else. Either it is simulatable by a Turing machine, which would make the most sense, or it isn't, in which case we're missing something about how you can compute things, which would be surprising and amazing.