3 ms·

Interesting that they went with a 32bit ISA, given the growth and demand for ML/AI, which seems to lean upon lower precision (8bit), then I wonder if they could

by Zenst 7y ago

Interesting that they went with a 32bit ISA, given the growth and demand for ML/AI, which seems to lean upon lower precision (8bit), then I wonder if they could of gone for that instead of something more a jack of all trades design. Which makes me wonder, when will somebody do an ISA dedicated for ML/AI and will we ever see old 8bit CPU's reborn as on memory CPU's. But we have seen this approach for processing upon memory before and whilst that has not tractioned into a product, this might. Though with Chip design going towards chiplets, ram can and may well become another chiplet in the evolution of that design process.

radarsat1 7y ago
In what domains do people do 8 bit machine learning? Most example code I've seen e.g. for TensorFlow uses float32 matrices.
- jdietrich 7y ago
  It's quite common to train at float32 but quantize down to int8 for inference. The loss of accuracy is negligible, but you make huge performance gains. https://medium.com/apache-mxnet/model-quantization-for-production-level-neural-network-inference-f54462ebba05 https://medium.com/apache-mxnet/model-quantization-for-produ...
tachyonbeam 7y ago
The thing is, you still need 32-bit or 64-bit values for pointers, loop counters, etc. You're also going to want your CPU to interact and convert between richer data types. What you probably want is to have a 64-bit CPU and control unit, with some SIMD ALUs that operate on multiple 8-bit values at once. Sidenote: I think that ML people are more interested in 16-bit floats than 8-bit integer values. Though none of that is mainstream yet, most ML code still uses 32-bit floating point numbers at the moment.
rbanffy 7y ago
Since the processing elements seem to be unable to reach memory outside their own 64 MB block, 32 bit integers seem sufficient for pointers. Also, it has a 64-bit wide access to its memory block, but can't map it - all transfers are via DMA. It must feel like a Cell processor with thousands of SPUs to play with.

by Zenst 7y ago

radarsat1 7y ago

In what domains do people do 8 bit machine learning? Most example code I've seen e.g. for TensorFlow uses float32 matrices.

jdietrich 7y ago
It's quite common to train at float32 but quantize down to int8 for inference. The loss of accuracy is negligible, but you make huge performance gains. https://medium.com/apache-mxnet/model-quantization-for-production-level-neural-network-inference-f54462ebba05 https://medium.com/apache-mxnet/model-quantization-for-produ...

tachyonbeam 7y ago

The thing is, you still need 32-bit or 64-bit values for pointers, loop counters, etc. You're also going to want your CPU to interact and convert between richer data types. What you probably want is to have a 64-bit CPU and control unit, with some SIMD ALUs that operate on multiple 8-bit values at once. Sidenote: I think that ML people are more interested in 16-bit floats than 8-bit integer values. Though none of that is mainstream yet, most ML code still uses 32-bit floating point numbers at the moment.

rbanffy 7y ago

Since the processing elements seem to be unable to reach memory outside their own 64 MB block, 32 bit integers seem sufficient for pointers. Also, it has a 64-bit wide access to its memory block, but can't map it - all transfers are via DMA. It must feel like a Cell processor with thousands of SPUs to play with.