2 ms·
As many others in this conversation have asked, can we have some sources on the idea that the model is spread across chips? You keep making the claim, but no o
by 9cb14c1ec0 8mo ago
As many others in this conversation have asked, can we have some sources on the idea that the model is spread across chips? You keep making the claim, but no one (myself included) else has any idea where that information comes from or if it is correct.
- aurareturn 8mo agoI was indeed wrong about 10 chips. I thought they would use llama 8B 16bit and a few thousand context size. It turns out, they used llama 8B 3bit with only 1k context size. That made me assume they must have chained multiple chips together since the max SRAM on TSMC n6 for reticle sized chip is only around 3GB.