3 ms·
Transformers are heavily memory bandwidth bound on modern hardware, and these chips only have 60% higher memory bandwidth.
by zaptrem 3y ago
Transformers are heavily memory bandwidth bound on modern hardware, and these chips only have 60% higher memory bandwidth.
- shaklee3 3y agoTheir own slides show a 20-30% speedup on attention tasks.