5 ms·
The numbers in this article don't make sense. They aren't consistent with the hardware (they seem to show the weights being loaded faster than mi300x peak memor
by fancyfredbot 2y ago
The numbers in this article don't make sense. They aren't consistent with the hardware (they seem to show the weights being loaded faster than mi300x peak memory bandwidth) and they aren't self consistent (70B models running only 2x slower than 7B models).
I'm not sure what's going on here but something is not right.
- Lindon4290 2y agoYeah, I've asked questions along those lines as well. Something sketchy is going on. See here - https://news.ycombinator.com/item?id=40833109 https://news.ycombinator.com/item?id=40833109
- deleted 2y ago[deleted]