3 ms·
You need at least 10x over an established approach to have any hope of surviving in the mid-term. The results are for batch-size 1 as well. GPUs are throughput
by orbifold 6y ago
You need at least 10x over an established approach to have any hope of surviving in the mid-term. The results are for batch-size 1 as well. GPUs are throughput optimized. So I wouldn't be surprised if an A100 chip would be able to outperform this for batch size >4. I still think this is a really cool architecture though.
- ZeroCool2u 6y agoThat's a good point, though these folks do specifically call this a "Streaming Processor", so I'm not sure that using anything besides a batch size of 1 for inference is entirely fair, but perhaps I'm misunderstanding what they mean by streaming. I was also thinking that the A100 comparison would be very interesting.