2 ms·
It’s not that crazy, just the architecture of differently quantized models and so on that you’d need to do that is impressive considering.
by BriggyDwiggs42 1y ago
It’s not that crazy, just the architecture of differently quantized models and so on that you’d need to do that is impressive considering.
- layer8 1y agoThe models are the same, it's the surrounding processing like "thinking" iterations that are adjusted.
- BriggyDwiggs42 1y agoThat only works for LRMs no? Not traditional LLM inference.