3 ms·
I'm confused, though, how then do they claim it needs the same resources as 13B? Is that amortised over parallel usage or something?
by irthomasthomas 3y ago
I'm confused, though, how then do they claim it needs the same resources as 13B? Is that amortised over parallel usage or something?
- gorbypark 3y agoThe same compute resources, but not the same VRAM. It will more or less get you the same tokens per second as a ~13B model but should have significantly "higher quality output" than a single 13B model.