14 ms·
Anyone know what kind of performance, in terms of token per second, one could expect with such a system running llama-2-70b?
by rthnbgrredf 3y ago
Anyone know what kind of performance, in terms of token per second, one could expect with such a system running llama-2-70b?
- EVa5I7bHFq9mnYK 3y agoThat system is for training, it will probably spew tokens like a Terminator's machine gun.