3 ms·
Yes but in matrix multiplication there are O(N²) numbers and O(N³) multiplications, so it might be possible that you are bounded by compute speed.
by codedokode 1y ago
Yes but in matrix multiplication there are O(N²) numbers and O(N³) multiplications, so it might be possible that you are bounded by compute speed.
- electroglyph 1y agoboth are equally important. compute for prefill and mem bandwidth for generation