3 ms·
definitely! it has the advantage that it can run CUDA kernels but on the other hand it has lower memory bandwidth and probably loses a token/s fight for many LL
by spullara 4mo ago
definitely! it has the advantage that it can run CUDA kernels but on the other hand it has lower memory bandwidth and probably loses a token/s fight for many LLMs.