2 ms·
My neural networks teacher in university joked that "if you come here to study brains you are in the wrong place because this is a linear algebra course". I thi
by jillesvangurp 6y ago
My neural networks teacher in university joked that "if you come here to study brains you are in the wrong place because this is a linear algebra course". I think that's a fair characterization and I wasn't very good at it/interested in it (the math part) and subsequently dropped out. However, it's a field that clearly takes its inspiration from the way biological brains work and then reduces that to a mathematical problem that we know how to optimize. It's a bit like not seeing the trees for the forest. There's a lot of matrix computation going on but in the end it's maybe not about that.
My observation of the field 25 years later is that it's still dominated by a lot of people good at math that are at this point generating results that are definitely very interesting yet not fully understood. We've gotten very good at wiring together matrix computations in interesting ways and training them to do useful stuff. But just like with real brains, we struggle to understand how the resulting networks actually work or how to design them properly.
I've seen high school kids work with tensor flow and they can definitely pull some magic tricks with that but they'd probably struggle to explain how the magic works as it's basically a black box component for them. And I suspect that is the case for large parts of the machine learning community as well. Math makes all of the lower level stuff work obviously but it's not a great tool for the more complex behavior that is basically emergent complexity of these networks. We can put them together and tinker with them but we seem to not have a deep understanding of how they work or even why they work. Very much like real brains.