6 ms·
New chip reduces neural networks’ power consumption
- tziki 9y agoCompared to what? Seems like the fairest comparison point would be Google's TPUs.
- synthos 9y agoIt's not really comparable to TPU. The MIT chip uses analog tricks to perform binary neural network operations. It's probably directly comparable to FPGA DNN, I think.
- jaredhansen 9y agoDo read the whole thing, but here's the quick summary of most important parts: [neural net] “nodes” are just weights stored in a computer’s memory. Calculating a dot product usually involves fetching a weight from memory, fetching the associated data item, multiplying the two, storing the result somewhere, and then repeating the operation for every input to a node... In the [new] chip, a node’s input values are converted into electrical voltages and then multiplied by the appropriate weights. Summing the products is simply a matter of combining the voltages. Only the combined voltages are converted back into a digital representation and stored for further processing. The chip can thus calculate dot products for multiple nodes — 16 at a time, in the prototype — in a single step, instead of shuttling between a processor and memory for every computation... One of the keys to the system is that all the weights are either 1 or -1... Recent theoretical work suggests that neural nets trained with only two weights should lose little accuracy — somewhere between 1 and 2 percent. Biswas and Chandrakasan’s research bears that prediction out. ... Their chip’s results were generally within 2 to 3 percent of the conventional network’s.
- newen 9y agoIf the weights are only 1 or -1, and each neuron can only get 16 inputs, then this is more similar to neuromorphic systems like IBM's True North than regular neural networks. And True North consumes on the order of 70-100 milliwatts.
- deleted 9y ago[deleted]
- Invictus0 9y agoCorrect me if I'm misunderstanding something, but won't this chip need a ton of ADCs to process these computations?
- arcticbull 9y agoDACs on one end and ADCs on the other, yeah. I guess though, it may be possible to hook analog inputs directly to it in some way, such as a camera or mic.
- deepnotderp 9y agoShhhh. When it comes to analog computing, discussing reality is not appreciated.
- twtw 9y ago1. Single bit ADC/DACs are trivial. 2. CDAC + switched capacitor multiplier + SAR ADC can be competitive with digital multiplier.
- deepnotderp 9y ago1. Yes, and the advantage of analog is significantly reduced as well, as a binary multiply and add then reduce to single gate operations. 2. Switched capacitor multipliers (actually almost anything analog) have difficulty scaling with feature shrinks, and that's before considering noise. E.g. see all the difficulties Xilinx went through to get 16nm FinFET ADCs efficiently working.
- darknoon 9y agoUses less power to do less useful work (binary weights)…
- phkahler 9y agoRight, they need to compare it to the same algorithm running on a conventional computer (or GPU or whatever they compared to). Math like that can run faster and more efficiently on a regular CPU too.
- dietr1ch 9y agoWell, using 16bit floats works fine and allows to use more layers, but might be too little. I'd be interested on a more conservative change using a single byte or maybe 4 bits.
- mtgx 9y agoIt seems to target mobile phones. Does it come anywhere near close to the ML processor IP Arm has just announced? (4.6 TOPs at 2W).
- nairboon 9y agoWhere is the paper? How can you write about research news without at least linking to the actual research?