5 ms·
Mark my words, in the next couple of years we'll see custom silicon that massively improves performance per watt for DNNs by using fixed point, quantization, an
by melted 11y ago
Mark my words, in the next couple of years we'll see custom silicon that massively improves performance per watt for DNNs by using fixed point, quantization, and saturation arithmetic. The gain in performance per watt will be at least an order of magnitude. This will make DNNs worthwhile in a lot more classification problems where currently they are simply too slow.
- nshm 11y agoMark my words, DNNs are not really the most efficient structure for predicting model due to the "distributed" representation that makes them good predictors, but also makes them hard to train and resource-consuming to apply. In few years DNNs will be replaced by more efficient models.
- melted 11y agoYou do realize you came up with this text using a deep biological neural network, right?
- nshm 11y agoSure, but they aren't necessary the most efficient model possible
- melted 11y agoSeems pretty efficient to me, at least per watt.
- emcq 11y agoPlease don't compare the complexity of the brain to artificial neural networks. The complexity of a single neuron is far more sophisticated than we use for machine learning. Many people are starting to call the machine learning "neurons" as "units" to prevent this comparison.
- dman 11y agoAny pointers to more approaches for more efficient alternatives?
- nshm 11y agoSorry, nothing definite yet. Any sort of tree-based computation which makes early pruning decision is more efficient than full matrix multiplication, even if matrix is sparse. Then, high-order computation like power is again more efficient than simple linear model. Those two directions are quite probable.