4 ms·

If your goal is to build something that works and is of interest to industry ($$$), I couldn't agree more :-) But I don't think that's the goal here. I view th

by cs702 7y ago

If your goal is to build something that works and is of interest to industry ($$$), I couldn't agree more :-) But I don't think that's the goal here. I view this type of research as part of early efforts in a long-term quest to discover if it's possible to build "neuromorphic" devices capable of universal computation that are much more efficient than current methods. These efforts today seem to be roughly at the same stage that "deep learning" was in the late 1970's to early 1980's, or even at an earlier stage. For what it's worth, I also think it's worthwhile to develop hardware/software that accelerates linear/multilinear algebra operations and/or makes it practical for us to use more general/flexible abstractions in our code.[a] -- [a] For example, I would love to be able to model each artificial neuron as a separate object in code, making it trivial to group/ungroup neurons into irregularly shaped and/or overlapping groups and easily shrink/expand/alter such groups on the fly without incurring a performance penalty. Right now, that's not possible: If we want decent performance, we have little choice but to pigeonhole all our designs into functions that operate on grid-like shapes (equal-sized vectors, matrices, higher-order tensors) and resort to thinks like masking so we can have fast multiply-sum operations.

p1esk 7y ago
What is "universal computation"? Turing complete? Then all we need is a good NAND gate. Or whatever computation is happening inside our brains? Then maybe we should first figure out what is actually happening there. The field today seems to be roughly at the same stage that "deep learning" was in the late 1970 Have all the research directions that seemed promising in 70s become as successful as DL? My point is - if you're interested in how brains work, focus on that. If you're interested in building better hardware, focus on that instead and try to build something that can be useful. This article got published because of "neuromorphic" (such a bullshit term!), and because there are MNIST results (which are embarassing, but the peer reviewers didn't catch that because they are probably device physicists, not ML people) EDIT (responding to your edit): I would love to be able to model each artificial neuron as a separate object in code So you want to run some experiments, without any idea whether it would lead to any good insights or benefits. Seems like building a specialized hardware just to accelerate this particular experiment is premature, to say the least. Just rent a bigger GPU cluster - you'll get your answer much faster and much cheaper.
- cs702 7y ago
  Similar arguments could have been made against deep (multilayer) neural networks (DNNs) for a couple of decades. No one knew for certain in the 70's and 80's if DNNs would pay off, yet a handful of then-obscure researchers persisted. And for quite a while DNNs were kind of a research backwater -- some researchers moved to Canada because they couldn't get funding anywhere else. As a close approximation, "almost no one" was interested in DNNs until Alexnet won ImageNet in 2012. Just because we don't know which research directions may or may not pay off decades from now does not mean such directions aren't worthwhile and should be abandoned.

by cs702 7y ago

p1esk 7y ago

What is "universal computation"? Turing complete? Then all we need is a good NAND gate. Or whatever computation is happening inside our brains? Then maybe we should first figure out what is actually happening there. The field today seems to be roughly at the same stage that "deep learning" was in the late 1970 Have all the research directions that seemed promising in 70s become as successful as DL? My point is - if you're interested in how brains work, focus on that. If you're interested in building better hardware, focus on that instead and try to build something that can be useful. This article got published because of "neuromorphic" (such a bullshit term!), and because there are MNIST results (which are embarassing, but the peer reviewers didn't catch that because they are probably device physicists, not ML people) EDIT (responding to your edit): I would love to be able to model each artificial neuron as a separate object in code So you want to run some experiments, without any idea whether it would lead to any good insights or benefits. Seems like building a specialized hardware just to accelerate this particular experiment is premature, to say the least. Just rent a bigger GPU cluster - you'll get your answer much faster and much cheaper.

cs702 7y ago
Similar arguments could have been made against deep (multilayer) neural networks (DNNs) for a couple of decades. No one knew for certain in the 70's and 80's if DNNs would pay off, yet a handful of then-obscure researchers persisted. And for quite a while DNNs were kind of a research backwater -- some researchers moved to Canada because they couldn't get funding anywhere else. As a close approximation, "almost no one" was interested in DNNs until Alexnet won ImageNet in 2012. Just because we don't know which research directions may or may not pay off decades from now does not mean such directions aren't worthwhile and should be abandoned.