We've been there already, before transformers, with AlphaGo. The AlphaGo model doesn't lose symbolic inference of the Q states/map it learned whereas transform

We've been there already, before transformers, with AlphaGo. The AlphaGo model doesn't lose symbolic inference of the Q states/map it learned whereas transformer based models do. Whether or not it can learn the coordinates of a more complex function (like f(x) = 123xe^3 + 12x^2 + f2(x)) depends on the amount of neurons in each hidden layer. Depending on how detailed the boosting method is, it is able to learn integrals (just as LSTMs can), the limitations about the detail of the result is limited by the implementation and the amount of neurons, hidden layers, and the boosting method.