14 ms·
Absolutely! I will certainly be experimenting with the new functions as you suggest. One thing which I haven't seen discussed elsewhere is the tension between
by anotherpaulg 3y ago
Absolutely! I will certainly be experimenting with the new functions as you suggest.
One thing which I haven't seen discussed elsewhere is the tension between the output format and the underlying task. When I ask GPT to use a simple, natural output format it does *better* at the actual code editing task. If I ask it to output using a more technical format like `diff -c` or heavily structured json formats... it "gets distracted" and does worse at the underlying coding request.
It writes worse code if you ask it to output edits in a terse, machine-readable diff format. It writes better code if you let it show you the code edits in a simple way.
For GPT3.5 this means I have to let it just type the whole source file back to me, with the edits included. GPT-4 is able to output a very simple diff-like format, but struggles with `diff -c`, etc.
So it will be interesting to see how the new function capabilities affect this tradeoff. Perhaps it is now "fluent" in function-json and so won't be "distracted" by that output formatting.
- ilaksh 3y agoMy guess is the new function training might overall slightly take away from the capacity for other types of reasoning, since it's a fixed capacity. But hopefully significantly less impact than including the instructions on the fly, since it's mostly baked in to the model now which should be a more efficient encoding.
- vorticalbox 3y agoIf you're using the API and not the website have you tried setting the temperature to something like 0.2? That may help with the distraction part.