4 ms·
I think you're conflating speed of inference/generation with optimization. gpt-3.5-turbo does not fit on a single GPU unlike the "toy" models.
by ukuina 4y ago
I think you're conflating speed of inference/generation with optimization.
gpt-3.5-turbo does not fit on a single GPU unlike the "toy" models.