3 ms·
Pure speculation: most likely quantizing models to 8bit to make them more efficient (and cheaper to run), and losing some accuracy on the way. I still use it,
by ardit33 3y ago
Pure speculation: most likely quantizing models to 8bit to make them more efficient (and cheaper to run), and losing some accuracy on the way.
I still use it, but I have found it to be less useful for coding than it was earlier. So, something is up.
- djangelic 3y agoI agree, this is exactly my experience as well. My earlier requests for code were more thorough then they are now. I can still use it if I give it the documentation in the prompt, but it has trouble remembering context across prompts.