4 ms·
I'm using model performance because inference is definitely not comparable to a 17B model when you're streaming model weights on and off disk storage.
by BoorishBears 7mo ago
I'm using model performance because inference is definitely not comparable to a 17B model when you're streaming model weights on and off disk storage.