4 ms·
A decent walkthrough of the paper is here: https://youtu.be/Dt_UNg7Mchg https://youtu.be/Dt_UNg7Mchg I’m surprised this hasn’t received more comments. Orca is
by ttul 3y ago
A decent walkthrough of the paper is here: https://youtu.be/Dt_UNg7Mchg https://youtu.be/Dt_UNg7Mchg
I’m surprised this hasn’t received more comments. Orca is an important result. An 11B-parameter model has reached near-parity with a 175B-parameter model on reasoning tasks. You don’t have to squint hard to see where this work is headed.
- ryneandal 3y agoThis is where I heard of Orca last night. Love his breakdowns. The idea (and model) is very exciting. I figured efficiency was going to be the next area of breakthrough, since monolith LLMs seem to be VRAM-limited these days. I hope this is a sign of things to come and ML becomes more democratized!