3 ms·
The post states: >"Perceptrons are the simplest form of machine learning and lend themselves to somewhat easier hardware implementations compared to some of th
by bogomipz 7y ago
The post states:
>"Perceptrons are the simplest form of machine learning and lend themselves to somewhat easier hardware implementations compared to some of the other machine learning algorithms."
Can someone explain what is it about perceptrons that make them easier to implement in hardware?
- StrangeDoctor 7y agoAll you need for a perceptron is to add up all of the inputs after they’ve been multiplied by their weight. This would probably be done in parallel with a fused multiply add circuit which is fairly simple and some place to store the weights. When they change the perceptron all they need to do is record the current weights and load in the new values.
- bogomipz 7y agoMakes sense, thanks for the explanation.
- matt_d 7y agoThere's a very approachable explanation in the following (around the 1:01:40 mark): https://youtu.be/8I_1TSs695I?t=1h1m40s https://youtu.be/8I_1TSs695I?t=1h1m40s -- part of Design of Digital Circuits - Lecture 18: Branch Prediction II (ETH Zürich, Spring 2019). Related readings: https://safari.ethz.ch/digitaltechnik/spring2019/doku.php?id=readings#lecture_18_0205_thu https://safari.ethz.ch/digitaltechnik/spring2019/doku.php?id.... These lectures are pretty great, by the way, highly recommended to anyone interested in computer architecture: http://people.inf.ethz.ch/omutlu/lecture-videos.html http://people.inf.ethz.ch/omutlu/lecture-videos.html Incidentally, this year's High-Performance Computer Architecture Test of Time Award has been given to "Dynamic Branch Prediction with Perceptrons" referenced in the lecture (from 2001, https://www.cs.utexas.edu/~lin/papers/hpca01.pdf https://www.cs.utexas.edu/~lin/papers/hpca01.pdf): https://engineering.tamu.edu/news/2019/02/jimenez-receives-high-performance-computer-architecture-test-of-time-award.html https://engineering.tamu.edu/news/2019/02/jimenez-receives-h....
- bogomipz 7y agoThese are fantastic links, thanks! I've watched many of Onur Mutlu's CMU lectures he's a wonderful teacher. Cheers.
- Der_Einzige 7y agoWait, how is K-NN harder to implement than a perception... Even in hardware? What about linear regression? What about decision trees? What about naive Bayes?
- jefft255 7y agoPerforming nearest neighbors search is much more expensive than a simple multiply-add. I don’t know about the other, my idea is that a perceptron is much easier to implement in hardware/low-level operations.
- kkmx 7y agoSame with Naive Bayes. That's very expensive compute-wise.
- ivalm 7y agoA single layer perceptron IS linear regression + some non-linearity of the output. I think in reality they could have used the word "logistic regression" but chose perceptron because it sounds better.