4 ms·
Of course they aren't, but it's still pretty evident that most opensource models are miles ahead of GTP-3, even the ones that are only a fraction of its size so
by littlestymaar 3y ago
Of course they aren't, but it's still pretty evident that most opensource models are miles ahead of GTP-3, even the ones that are only a fraction of its size so there's still some massive improvement that doesn't depend on the model size itself.
- int_19h 3y agoGPT-3 sure. There's no model that's even close to GPT-4, though. And we don't really know much about the actual size of that, only lots of speculation.
- littlestymaar 3y agoMy entire point since the beginning is that the models out there improved a ton compared to what used to be the state of the art just 4 years ago, with an equivalent when not much smaller number of parameters (GPT-3 was 175B params and on MMLU it performs worse than Mistral 7B [1]). [1]: https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu https://paperswithcode.com/sota/multi-task-language-understa...