3 ms·
Given how close it is to 405B in performance it would be interesting to see which has the edge comparing an unquantized 3.3-70B against 405B quantized to be the
by Lerc 2y ago
Given how close it is to 405B in performance it would be interesting to see which has the edge comparing an unquantized 3.3-70B against 405B quantized to be the same size.
- vletal 2y agoThat would be 1.38 bits per weight on average, which I can confidently guess would not perform well.
- swalsh 2y agoIt's kind of amazing how there seems to be a wall where sizing up the model starts to diminish in terms of intelligence gains. I guess that's why we can still compete with whales even though their brains are like twice as big as ours.
- vessenes 2y agoThere is a line of thinking that the required tokens shoved through training also needs to go up, perhaps super-linearly to the model size. And of course, there's a line of thinking that there are diminishing returns in all things, including these models. Both could be true!
- int_19h 2y agoI have tried 405B at 1-bit quantization. It remains coherent, but didn't seem to be any better than 3.1-70B.