2 ms·
Somehow I never got the hype around Groq. They served models with fast inference speed - that sounded great in theory - and as a user I was looking forward to u
by conshama 4mo ago
Somehow I never got the hype around Groq. They served models with fast inference speed - that sounded great in theory - and as a user I was looking forward to use them. But, when I did try, I discovered that they are quantizing the models underneath. And they dont even disclose it. So I stopped using them.
The whole thing never made any sense to me - but I guess AI hype is a thing.
- adityashankar 4mo agoI believe despite quantisation they were still extremely fast, which is still incredibly useful if you don't need high precision/accuracy (which is good enough for many use cases)