>All optimizations inevitably end up just being funneled into larger and larger models. Well of course if you're trying to beat the SOTA, bigger sizes allow fo

>All optimizations inevitably end up just being funneled into larger and larger models. Well of course if you're trying to beat the SOTA, bigger sizes allow for better models, but not everyone is trying to use or train the latest SOTA model, maybe llama-3 8b is perfect for what you need to do and having better optimizations to run it locally is gold.