2 ms·
GPT4 vs. GPT-4-0613: Worse reasoning, but improves if you use the function API
- diegodotta 3y agoYeap. We noticed some issues with the new version as well. We went back to the previous model.
- reissbaker 3y agoThere's been a bunch of discussion of gpt-4 "getting worse"; I tried gpt-4 vs gpt-4-0613 (the latest release with function calling), and noticed that while the latest release is much faster, its generalized instruction following capability indeed declined compared to the original... But you can get it to regain some lost reasoning capability by using the new function call API. Kinda weird!
- ruined 3y agoask gpt the same question three times, you'll get three different answers. a 'correct' answer one day and an 'incorrect' answer later demonstrates nothing.
- PeterStuer 3y agoZero the temperature if you want the same answer every time.
- reissbaker 3y agoI redid the test several times! GPT4 consistently got it right, and gpt-0613 got it wrong every time. Anecdotally gpt-0613 has generally been "worse" for me on non-function-calls (but equally-ish good and much faster on function calls), but I wanted a consistent test for it. Feel free to try and duplicate on the Playground though!
- throwawayadvsec 3y agoit's just a question of temperature that's a choice
- Madmallard 3y agoReally wish they didn’t block the population out of more real value
- PeterStuer 3y agoWe desperately need more advances on open models. Having this space controlled by corps that decide what we can or can not use of the SOTA is very frustrating