You might like the paper "Do NLP Models Know Numbers? Probing Numeracy in Embeddings": https://arxiv.org/abs/1909.07940 https://arxiv.org/abs/1909.07940 Neural

You might like the paper "Do NLP Models Know Numbers? Probing Numeracy in Embeddings": https://arxiv.org/abs/1909.07940 https://arxiv.org/abs/1909.07940 Neural network models seem to encode an approximate notion of quantity in their representations. This paper is pre-GPT-3, but I would think the larger training set and larger model capacity would help the model learn quantity more easily.