Check out my project https://github.com/Thopliterce/transformer-arithmetic https://github.com/Thopliterce/transformer-arithmetic. This is a concrete implementat

Check out my project https://github.com/Thopliterce/transformer-arithmetic https://github.com/Thopliterce/transformer-arithmetic. This is a concrete implementation based on GPT-2 model that does multiplication accurately, digit by digit. It does so by generating a dataset that tells the model how to do multiplication step by step. Doing arithmetic actually works with just GPT-2, without an oracle.