3 ms·
I wonder what inference of big LLMs might look like with that much cache and memory bandwidth. Not trivial to get a benchmark for that but I wonder
by freeqaz 2y ago
I wonder what inference of big LLMs might look like with that much cache and memory bandwidth. Not trivial to get a benchmark for that but I wonder
- wmf 2y agoFor LLMs you'd want to use the MI300X variant and benchmarks should already be available.