3 ms·

>As there is no hidden state, no "memory" or "intention" can be passed to the future This is probably your misconception : "Transformers are RNNs" https://arxi

>As there is no hidden state, no "memory" or "intention" can be passed to the future This is probably your misconception : "Transformers are RNNs" https://arxiv.org/pdf/2006.16236.pdf https://arxiv.org/pdf/2006.16236.pdf (see section 3.4) Even though an inner state is not specified explicitly like in a LSTM, for a transformer it's possible to view the inner state as implicitly defined as the inner features (deterministically derived from the layers weights) applied to the past inputs. Therefore transformers can learn to do the task by learning to remember a,b,ab to this inner state and write them in order ab,a,b (Though to train it you'll probably have to give it an initial random context to be able to produce different sequences : From this source of entropy it will learn to produce 2 hidden variables which it will use to produce a,b,and ab) I concede that it will not be an easy task to learn in this way, and that it's probably easier to train a chain of thought model where you will allow it to use its output to remember the state (by writing it down), instead of having to memorize in its weights. So if you train it to produce a,b,ab,a,b and then another transformer to extract the last 3 outputs a*b,a,b it become very easy.

golol 4y ago
This is very surprising to me and I don't completely understand it. Let me ask; Does this crucially rely on deterministic behavior of the transformer? The only way I can imagine what you say is that the transformer can see the past inputs, and if it is deterministic know what (internal) chain of thought it must have had and hence what its plan (factorization) would have been when it generated the output (first prime number). In this case, I have the impression that any amount of non-determinicity, i.e. randomness, breaks this mechanism and hence transformers with noise are weaker than RNNs with noise. Would you agree? Furthermore, if I understand correctly it does seem like it would be strange for a model to learn to use its knowledge of its own deterministic behavior to predict what the only strategy it could have chosen must have been. I just can't imagine a transformer solving the prime product generation task with this method, even if generating large primes and muktiplying them was easy for it. (And I can easily imagine an RNN solving the task). It reminds me of solutions to infinite prisoner hat problems where you agree on some abstract well-ordering of possibilities...
- GistNoesis 4y ago
  No it doesn't crucially rely on deterministic behavior (you can add some dropout layers if you want). Transformers does see the past inputs. But it won't really use them, except to "serialize the answer" that it will form on its first time-step : From the random initial context vector, splitting this space in regions it will generate two "numbers" as features, and multiply them as features and from this feature vector containing the answer it will just serialize it. At no times will the network ever factorize anything. As an exercise you can probably manually specify the weights that will do exactly the task I describe, by considering each feature as an individual bit, and specifying layers weights such like they behave as multiplier logical circuit (with or-gates, and-gates and not-gates ...). (Therefore the architecture can at least express the solution even though the network will probably have a really hard time reaching it) Alternatively, a transformer is capable of over-fitting exactly a dataset : so if you give it a dataset ab,a,b it is perfectly capable of memorizing it exactly so it will be able to produce the task you want, even though it will not have understood anything. Alternatively, if you want for the network to implicitly discover chain of thought reasoning then you will have to train the transformer to produce ????????ab,a,b this is a much harder task to learn because the gradient has to flow across various time-steps and through discrete sampling (instead of just trying to predict from imitation using known previous character). This is a Reinforcement Learning problem, that has to deal with credit assignment and long time horizon. You can use decision transformer. Or use other larger LLM that has access to the task description to help sample intelligent ???????, and train standard transformer on this sampled inputs, define a reward model, and build datasets of increasing rewards. Even in these reinforcement learning, you can reach the optimal state without having to "plan". You can learn optimal controller that plan implicitly by just solving for the Bellman equation. This controller are living in the present and not anticipating the future outcomes, but you can also build controller like Model Predictive Control, that at each time-step make a plan, project it and solve for the best next move, but it's much more computationally expensive.
  - golol 4y ago
    Hmm I would love for someone else to give their opinion because I find it very interesting but don't quite understand it yet. The way I see it, at the very beginnig the transformer has a large number of choices for numbers a, b that allow it to solve the problem. If there is randomness present then it will (pseudo-)randomly choose a pair a, b, with the intention to write ab,a,b After writing the first digit of ab, as I understand it wanta to recover its features by doing the same operations as before on the past sequence. But as the computation of the features is non-deterministic, it can't arrive at the same pair a, b. Let me try to specifiy a more difficult task for a tranaformer with randomness: I want you to generate exactly three numbers in the form c,a,b where a and b are prime numbers with 250-300 digits and c=ab. I want these numbers to be randomly chosen with a distribution so that the range of primes 250-300 is approximately uniformly covered. Suppose now a transformer has uniformly picked a and b and generated the first digit of ab. Let's in fact say it has already generated all the digits of ab. If the transformer has weights that make it now successfully print a, b (say with probability > 0.99), then you have constructed a method of factoring products of prime numbers in the 250-300 digit range, i.e. you just initialize the context window with the desired number 500-600 number ab to be factorized, and let the transformer do its work. I.e. such a transformer has to be computationally powerful enough to factorize large prome products with >0.99 percent accuracy. On the other hand an RNN or a human both with randomness can solve this task without having to be computationally powerful enough to do the factorization.
theGnuMe 4y ago
You can pass context in as another input, so encoder->decoder cross attention models. People are working on data storage and retrieval as well with transformers (so you would query out to load in different attention vectors). So you can just view that as a memory bank with paging. You do however need to teach the network the task you want to solve. So you'd have to train it on the math part.

golol 4y ago

This is very surprising to me and I don't completely understand it. Let me ask; Does this crucially rely on deterministic behavior of the transformer? The only way I can imagine what you say is that the transformer can see the past inputs, and if it is deterministic know what (internal) chain of thought it must have had and hence what its plan (factorization) would have been when it generated the output (first prime number). In this case, I have the impression that any amount of non-determinicity, i.e. randomness, breaks this mechanism and hence transformers with noise are weaker than RNNs with noise. Would you agree? Furthermore, if I understand correctly it does seem like it would be strange for a model to learn to use its knowledge of its own deterministic behavior to predict what the only strategy it could have chosen must have been. I just can't imagine a transformer solving the prime product generation task with this method, even if generating large primes and muktiplying them was easy for it. (And I can easily imagine an RNN solving the task). It reminds me of solutions to infinite prisoner hat problems where you agree on some abstract well-ordering of possibilities...

GistNoesis 4y ago
No it doesn't crucially rely on deterministic behavior (you can add some dropout layers if you want). Transformers does see the past inputs. But it won't really use them, except to "serialize the answer" that it will form on its first time-step : From the random initial context vector, splitting this space in regions it will generate two "numbers" as features, and multiply them as features and from this feature vector containing the answer it will just serialize it. At no times will the network ever factorize anything. As an exercise you can probably manually specify the weights that will do exactly the task I describe, by considering each feature as an individual bit, and specifying layers weights such like they behave as multiplier logical circuit (with or-gates, and-gates and not-gates ...). (Therefore the architecture can at least express the solution even though the network will probably have a really hard time reaching it) Alternatively, a transformer is capable of over-fitting exactly a dataset : so if you give it a dataset ab,a,b it is perfectly capable of memorizing it exactly so it will be able to produce the task you want, even though it will not have understood anything. Alternatively, if you want for the network to implicitly discover chain of thought reasoning then you will have to train the transformer to produce ????????ab,a,b this is a much harder task to learn because the gradient has to flow across various time-steps and through discrete sampling (instead of just trying to predict from imitation using known previous character). This is a Reinforcement Learning problem, that has to deal with credit assignment and long time horizon. You can use decision transformer. Or use other larger LLM that has access to the task description to help sample intelligent ???????, and train standard transformer on this sampled inputs, define a reward model, and build datasets of increasing rewards. Even in these reinforcement learning, you can reach the optimal state without having to "plan". You can learn optimal controller that plan implicitly by just solving for the Bellman equation. This controller are living in the present and not anticipating the future outcomes, but you can also build controller like Model Predictive Control, that at each time-step make a plan, project it and solve for the best next move, but it's much more computationally expensive.
- golol 4y ago
  Hmm I would love for someone else to give their opinion because I find it very interesting but don't quite understand it yet. The way I see it, at the very beginnig the transformer has a large number of choices for numbers a, b that allow it to solve the problem. If there is randomness present then it will (pseudo-)randomly choose a pair a, b, with the intention to write ab,a,b After writing the first digit of ab, as I understand it wanta to recover its features by doing the same operations as before on the past sequence. But as the computation of the features is non-deterministic, it can't arrive at the same pair a, b. Let me try to specifiy a more difficult task for a tranaformer with randomness: I want you to generate exactly three numbers in the form c,a,b where a and b are prime numbers with 250-300 digits and c=ab. I want these numbers to be randomly chosen with a distribution so that the range of primes 250-300 is approximately uniformly covered. Suppose now a transformer has uniformly picked a and b and generated the first digit of ab. Let's in fact say it has already generated all the digits of ab. If the transformer has weights that make it now successfully print a, b (say with probability > 0.99), then you have constructed a method of factoring products of prime numbers in the 250-300 digit range, i.e. you just initialize the context window with the desired number 500-600 number ab to be factorized, and let the transformer do its work. I.e. such a transformer has to be computationally powerful enough to factorize large prome products with >0.99 percent accuracy. On the other hand an RNN or a human both with randomness can solve this task without having to be computationally powerful enough to do the factorization.

theGnuMe 4y ago

You can pass context in as another input, so encoder->decoder cross attention models. People are working on data storage and retrieval as well with transformers (so you would query out to load in different attention vectors). So you can just view that as a memory bank with paging. You do however need to teach the network the task you want to solve. So you'd have to train it on the math part.