3 ms·
If you knew that the model never changed it might be very helpful, but most of the big providers constantly mess with their models.
by BriggyDwiggs42 1y ago
If you knew that the model never changed it might be very helpful, but most of the big providers constantly mess with their models.
- cwillu 1y agoEven if you used a local copy of a model, it would still just be a semi-quantitative version of “everyone knows ‹thing-you-don't-have-a-grounded-argument-for›”
- layer8 1y agoTheir performance also varies depending on load (concurrent users).
- BriggyDwiggs42 1y agoDear god does it really? That’s very funny.
- wiseowise 1y agoWhy are you surprised? It’s a computational thing, after all.
- BriggyDwiggs42 1y agoIt’s not that crazy, just the architecture of differently quantized models and so on that you’d need to do that is impressive considering.
- layer8 1y agoThe models are the same, it's the surrounding processing like "thinking" iterations that are adjusted.
- BriggyDwiggs42 1y agoThat only works for LRMs no? Not traditional LLM inference.