3 ms·
if not open source or API is not available, benchmarks can't be independently reproduced, its hard to take at face value any claim of outperforming GPT3.5, whic
by swyx 3y ago
if not open source or API is not available, benchmarks can't be independently reproduced, its hard to take at face value any claim of outperforming GPT3.5, which is a major claim that must be verified first (Falcon has had similar reproduction issues https://twitter.com/Francis_YAO_/status/1667245675447468034?s=20 https://twitter.com/Francis_YAO_/status/1667245675447468034?...)
- ilaksh 3y agoYou can very easily test it on their pi website. I tried a coding exercise. It's not as good at programming as gpt-3.5-turbo. From my short test, what it really seems to excel at is smugness.