3 ms·

IBM's NorthPole is faster and uses less energy than Nvidia's best AI GPU

30 points by danboarder 3y ago

deleted 3y ago
[deleted]
eggy 3y ago
IBM was always a powerhouse of R&D, so glad to see this to keep the field competitive between the few players in the game. I love APL and that grew out of Ken Iverson's work at Harvard, but then completed at IBM with double the salary [1]. [1] https://en.wikipedia.org/wiki/Kenneth_E._Iverson
Ukv 3y ago
To my understanding this is a small low-energy chip designed for applications like basic object detection in a doorbell camera, or speech recognition for a home assistant. I think it's a bit misleading to say "faster [...] than Nvidia's best AI GPU" and "outperofrms[sic] all major architectures" - it specifically means latency and energy efficiency. Its throughput/memory are so tiny that it's not at all a replacement for Nvidia's best AI GPUs (and not intended to be): > NorthPole's core array includes 192 MB of flexible memory (768KB of unified memory per core). Assigning 2/3rd of this memory to parameters, such as weights and biases, provides 128MB for network storage. https://www.science.org/action/downloadSupplement?doi=10.1126%2Fscience.adh1174&file=science.adh1174_sm.pdf https://www.science.org/action/downloadSupplement?doi=10.112...
sillywalk 3y ago
"It's worth clarifying a few things early here. First, NorthPole does nothing to help the energy demand in training a neural network; it's purely designed for execution. Second, it is not a general AI processor; it's specifically designed for inference-focused neural networks. As noted above, inferences include things like figuring out the contents of an image or audio clip so they have a large range of uses, but this chip may do you any good if your needs include running a large language model because they're too large to fit in the hardware." [0] [0] https://arstechnica.com/science/2023/10/ibm-has-made-a-new-highly-efficient-ai-processor/ https://arstechnica.com/science/2023/10/ibm-has-made-a-new-h...

30 points by danboarder 3y ago

deleted 3y ago

[deleted]

eggy 3y ago

IBM was always a powerhouse of R&D, so glad to see this to keep the field competitive between the few players in the game. I love APL and that grew out of Ken Iverson's work at Harvard, but then completed at IBM with double the salary [1]. [1] https://en.wikipedia.org/wiki/Kenneth_E._Iverson

Ukv 3y ago

To my understanding this is a small low-energy chip designed for applications like basic object detection in a doorbell camera, or speech recognition for a home assistant. I think it's a bit misleading to say "faster [...] than Nvidia's best AI GPU" and "outperofrms[sic] all major architectures" - it specifically means latency and energy efficiency. Its throughput/memory are so tiny that it's not at all a replacement for Nvidia's best AI GPUs (and not intended to be): > NorthPole's core array includes 192 MB of flexible memory (768KB of unified memory per core). Assigning 2/3rd of this memory to parameters, such as weights and biases, provides 128MB for network storage. https://www.science.org/action/downloadSupplement?doi=10.1126%2Fscience.adh1174&file=science.adh1174_sm.pdf https://www.science.org/action/downloadSupplement?doi=10.112...

sillywalk 3y ago

"It's worth clarifying a few things early here. First, NorthPole does nothing to help the energy demand in training a neural network; it's purely designed for execution. Second, it is not a general AI processor; it's specifically designed for inference-focused neural networks. As noted above, inferences include things like figuring out the contents of an image or audio clip so they have a large range of uses, but this chip may do you any good if your needs include running a large language model because they're too large to fit in the hardware." [0] [0] https://arstechnica.com/science/2023/10/ibm-has-made-a-new-highly-efficient-ai-processor/ https://arstechnica.com/science/2023/10/ibm-has-made-a-new-h...