3 ms·
If you want to load up 405B @ FP_16 into a single H100 box, how do you do it? You get two boxes. 2x the price. Models are getting larger, not smaller. This is
by latchkey 2y ago
If you want to load up 405B @ FP_16 into a single H100 box, how do you do it? You get two boxes. 2x the price.
Models are getting larger, not smaller. This is why H200 has more memory, but the same exact compute. MI300x vs. MI325x... more memory, same compute.