4 ms·
Very good result, and awesome to see such great progress happen so fast. Quantizing/pruning/deduplicating/compressing models and embeddings is still a vast orc
by gorkish 3y ago
Very good result, and awesome to see such great progress happen so fast.
Quantizing/pruning/deduplicating/compressing models and embeddings is still a vast orchard of low hanging fruit.
I personally think there are still quite a few multiple-orders-of-magnitude scale opportunities to accelerate inference, and we are fortunate to have strong economic incentives aligned with the problem.
- Der_Einzige 3y agoSo much low hanging fruit for those willing to pick it. It's a great time to be an LLM researcher.
- bitL 3y agoYeah, writing a thesis on it right now; this + adapters give so many options to play with and Meta was nice to give me access to their research models.