Gpt3 is a graph neural network with added recursive information through positioninal encoding. It outputs sequentially, but I am not sure why a RNN would be req

Gpt3 is a graph neural network with added recursive information through positioninal encoding. It outputs sequentially, but I am not sure why a RNN would be required beyond that. I would agree that the manner of reasoning must differ, since the sequence follows bpe tokens rather than logical steps, however who is to say that another form of mathematical reasoning could not lead to valid results? For instance, Gpt might solve the problem at each output step insofar as required to generate that token. It certainly iterates over each output token, and the encoding of the problem is equivalent so iterating over the characters of the math problem, roughly speaking. But yes, the iterated output does not follow a logical graph externally, it is token by token. But internally, the network can absolutely follow a more complex graph. Could you say what you mean by single network path when we speak about attention based architectures? What sort of operation or information is missing in such architecture? I am aware of of some results relating to certain graphs, but I do not think this would apply to a text describing a math problem, say.