3 ms·
You don't have to wonder. In their paper: https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165 they state it has 0.7% accuracy on zero shot 5 digit
by ja3k 6y ago
You don't have to wonder. In their paper: https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165
they state it has 0.7% accuracy on zero shot 5 digit addition problems and 9.3% accuracy on few shot 5 digit addition problems.
- gwern 6y agoYou do have to wonder, because as that section states, the BPEs may impede arithmetic, and as we've found using the API, if you use commas, the accuracy (zero and few-shot) goes way up.
- ralfd 6y agoBy the way: Arithmetic accuracy is better if dollar sign and commas are added (financial data in the training set): http://gptprompts.wikidot.com/logic:math http://gptprompts.wikidot.com/logic:math