3 ms·

They may do that for that reason, and it might also happen to help them produce the right output code. My point is just that (aiui), for any transformer model,

by drdeca 2y ago

They may do that for that reason, and it might also happen to help them produce the right output code. My point is just that (aiui), for any transformer model, holding the network constant but increasing the length of the code that is the input, at some point they will be unable to produce the output of the code without first preceding it with other output tokens first. Of course, this statement is kind of trivial as applied to current transformer models because they have a limited context length anyway (they don’t properly generalize to ~arbitrary context lengths, but I imagine there is a way to fix that… idk how though). And, also, “output the tokens for the output of the code immediately, without other tokens first” is I think a bit restrictive. That they are limited in this way when required to produce the output immediately is, to my mind, not that significant of a limitation. Just wrap the model in a simpler (designed by hand) program which has the model produce whatever tokens it needs to produce to get the answer, and then extract the answer from that and yield that as the output. (Though, while it seems likely to me that there is a possible transformer model that would be capable of serving this role inside such a wrapper, obtaining such a model through training seems more complicated.)

dartos 2y ago
Just some thoughts: > they don’t properly generalize to ~arbitrary context lengths, but I imagine there is a way to fix that… idk how though There’s no way to fix that for transformers architectures as the limited context length is inherent in their attention mechanisms (it’s a big reason they are so efficient at inference time,) but RNN architectures like RWKV have no such limitation, but haven’t seen success at a large scale yet. > They may do that for that reason, and it might also happen to help them produce the right output code We can only really spitball at this, but I’m poverty confident that it’s a result of how they’re trained. You could train a model that just takes natural language and outputs only code, with no conversion. Those models would be less general, and less interesting to non Technical people, which is why we don’t see them as much (imo.)
- drdeca 2y ago
  > You could train a model that just takes natural language and outputs only code, with no conversion. Those models would be less general, and less interesting to non Technical people, which is why we don’t see them as much (imo.) Right, I’m not saying “the reason they output other text first is because they need to in order to produce the correct output.”. I agree that the reason they put the other text first is due to how they were trained. I’m just saying that if they didn’t do that, that this would limit the theoretical maximum average performance they could achieve at the “producing the output of some long code” task. But, I don’t expect that this was part of the motivation that led to them being trained in the way they have been. > There’s no way to fix that for transformers architectures as the limited context length is inherent in their attention mechanisms Hmm… the paper showing that transformers can recognize TC^0 has a definition for attention that is a bit different from the usual attention, and seems a bit restricted.., it seemed to me that the paper involved the idea that this could be applied to arbitrary length sequences, but I could be mistaken.
  - dartos 2y ago
    Oh? Do you have a link to that paper? I’m no linalg expert, but it was my understanding that attention worked (very basically) because each token attends to all other tokens in the sequence, but you can’t do that with an unbounded sequence length
    - drdeca 2y ago
      I think it was https://aclanthology.org/2022.tacl-1.49/ https://aclanthology.org/2022.tacl-1.49/ But am now thinking I may have made a mistake, with this showing they can be simulated in TC^0 when I thought the result was both that and that they can simulate TC^0 .

by drdeca 2y ago

dartos 2y ago

Just some thoughts: > they don’t properly generalize to ~arbitrary context lengths, but I imagine there is a way to fix that… idk how though There’s no way to fix that for transformers architectures as the limited context length is inherent in their attention mechanisms (it’s a big reason they are so efficient at inference time,) but RNN architectures like RWKV have no such limitation, but haven’t seen success at a large scale yet. > They may do that for that reason, and it might also happen to help them produce the right output code We can only really spitball at this, but I’m poverty confident that it’s a result of how they’re trained. You could train a model that just takes natural language and outputs only code, with no conversion. Those models would be less general, and less interesting to non Technical people, which is why we don’t see them as much (imo.)

drdeca 2y ago
> You could train a model that just takes natural language and outputs only code, with no conversion. Those models would be less general, and less interesting to non Technical people, which is why we don’t see them as much (imo.) Right, I’m not saying “the reason they output other text first is because they need to in order to produce the correct output.”. I agree that the reason they put the other text first is due to how they were trained. I’m just saying that if they didn’t do that, that this would limit the theoretical maximum average performance they could achieve at the “producing the output of some long code” task. But, I don’t expect that this was part of the motivation that led to them being trained in the way they have been. > There’s no way to fix that for transformers architectures as the limited context length is inherent in their attention mechanisms Hmm… the paper showing that transformers can recognize TC^0 has a definition for attention that is a bit different from the usual attention, and seems a bit restricted.., it seemed to me that the paper involved the idea that this could be applied to arbitrary length sequences, but I could be mistaken.
- dartos 2y ago
  Oh? Do you have a link to that paper? I’m no linalg expert, but it was my understanding that attention worked (very basically) because each token attends to all other tokens in the sequence, but you can’t do that with an unbounded sequence length
  - drdeca 2y ago
    I think it was https://aclanthology.org/2022.tacl-1.49/ https://aclanthology.org/2022.tacl-1.49/ But am now thinking I may have made a mistake, with this showing they can be simulated in TC^0 when I thought the result was both that and that they can simulate TC^0 .