You have to be careful when analyzing GPT-3 and derivatives due to BPE encoding. It's accuracy greatly improves when adding commas to strings of numbers: https:

You have to be careful when analyzing GPT-3 and derivatives due to BPE encoding. It's accuracy greatly improves when adding commas to strings of numbers: https://www.gwern.net/GPT-3#fn21 https://www.gwern.net/GPT-3#fn21