4 ms·
The model meets/beats Llama despite having an order-of-magnitude fewer active parameters (52B vs 405B). Absolutely bonkers. AI is moving so fast with these brea
by a_wild_dandan 2y ago
The model meets/beats Llama despite having an order-of-magnitude fewer active parameters (52B vs 405B). Absolutely bonkers. AI is moving so fast with these breakthroughs -- synthetic data, distillation, alt. architectures (e.g. MoE/SSM), LoRA, RAG, curriculum learning, etc.
We've come so astonishingly far in like two years. I have no idea what AI will do in another year, and it's thrilling.
- csomar 2y agoIt is insane because 52B can run on my current 3 years old laptop. 3B LLMA 3.2 from Facebook can already autocomplete for me. I didn't try this model but if the scores are to be believed, this can give useful and actionable insights into a project source code. Probably not as good as Claude 3.5 but I can run it locally. This is a game changer.
- z3ncyberpunk 2y agoMoving fast or just completely inefficient