2 ms·
You won't see mainframes doing AI training, but there is a lot of value in being able to do extremely low-latency inference (which is why they have their NPU on
by rbanffy 2y ago
You won't see mainframes doing AI training, but there is a lot of value in being able to do extremely low-latency inference (which is why they have their NPU on the same chip as the CPUs, just a few clock-cycles from the cores) during on-line transaction processing, and less timing-critical inference work on the dedicated cards (which are a few more nanoseconds away).