3 ms·
A billion parameter matrix, if it was square, is 32,000 by 32,000. NVidia's tensor units are basically 4x4 matrix multipliers.
by xscott 3y ago
A billion parameter matrix, if it was square, is 32,000 by 32,000. NVidia's tensor units are basically 4x4 matrix multipliers.