Your first point is actually covered in the computerphile video on GPT-3. They discuss a paper that studies the ability of GPT-3 to do arithmetic as it's part

Your first point is actually covered in the computerphile video on GPT-3. They discuss a paper that studies the ability of GPT-3 to do arithmetic as it's part of the learned text database Check a look https://www.youtube.com/watch?v=_8yVOC4ciXc https://www.youtube.com/watch?v=_8yVOC4ciXc