4 ms·
Great read. Have you compared performance with other Llama models (3, 3.2) or have you just done benchmarking with 3.1? Is there some intuition as to why 3.1 m
by fngarrett 2y ago
Great read. Have you compared performance with other Llama models (3, 3.2) or have you just done benchmarking with 3.1?
Is there some intuition as to why 3.1 might outperform 3.2 on MI300X?
- cheptsov 2y agoIn this one we were only using 3.1 405B FP8. We took one model to simplify the setup and were mostly looking at the memory saturation effect. So basically we compared inference metrics of the same model. I suppose comparing 3.1 and 3.2 will be difficult as they are different models entirely. But open to ideas