4 ms·
The conclusions start from https://github.com/E-xyza/Exonerate/blob/master/bench/reports/gpt-bench.md#accuracy-evaluation https://github.com/E-xyza/Exonerate/bl
by dwohnitmok 4y ago
The conclusions start from https://github.com/E-xyza/Exonerate/blob/master/bench/reports/gpt-bench.md#accuracy-evaluation https://github.com/E-xyza/Exonerate/blob/master/bench/report...
This is particularly impressive for Elixir, which is not a language that is a particular focus of GPT-4. I imagine the accuracy for Python is extremely good. Maybe near perfect for this kind of benchmark if allowed to see error messages.
- naet 4y agoIt's also possible GPT-4 is better at writing Elixir since there are less beginners/students writing Elixir code and polluting the training data with bad practice or faulty code.
- coder-3 4y agoThis might not be an issue for the same (somewhat inscrutable) reason that GPT-4 has quasi-perfect grammar.