4 ms·
34B is grouped query attention, right? Does that make it the smallest model with grouped attention? I can see some people fine-tuning it again for general prop
by born-jre 3y ago
34B is grouped query attention, right? Does that make it the smallest model with grouped attention?
I can see some people fine-tuning it again for general propose instruct.