3 ms·

I did not read the entire linked article but For instance, I tested whether asking a model to produce "a b c d e" before solving a math problem improves perfor

I did not read the entire linked article but For instance, I tested whether asking a model to produce "a b c d e" before solving a math problem improves performance. is of course not what you would expect to work. It is not about dumping meaningless stuff into the input but intermediate results. If you have to add two long numbers but are only capable to add single digits, you want to dump the carry into the output and get this fed back as input so that you can take it into account when adding up the next two digits. Dumping unrelated stuff into the output will of course not help. Chain of thought does exactly this, it solves part of the overall problem and feeds that back for consideration when tackling the next piece of the problem.

famouswaffles 3y ago
Yes and I'm saying even that doesn't work as robustly as CoT across different tasks. CoT is so obviously not just about padding for extra compute. Certain tokens will guide it to more accurate responses just as much as others derail it out of distribution. This should be obvious. Literally this paper, you can see the model only really learns how to use the padding tokens effectively after training.
- danbruc 3y ago
  Yes and I'm saying even that doesn't work as robustly as CoT across different tasks. What does not work? CoT is so obviously not just about padding for extra compute. In a certain way it is but the crucial point is that an autoregressive neural network is pretty much a recurrent neural network and chain of thought encourages the use of the feedback path.
  - famouswaffles 3y ago
    >What does not work? I'm saying that even if you do what you suggest, it simply doesn't work that well. If you read the paper, you'd see extra computation from padding is not used effectively till after training. Yes, extra compute helps with CoT but it's not even the reason it works. I believe CoT works moreso because it nudges the computation in the right(er) direction.
    - danbruc 3y ago
      List the digit sum mod 7 of the first 10 primes larger than 53. This starts with 0 0 6 1. What do you have to do to generate the next number? Count the numbers in the output to figure out that you have done 4 numbers, add 1 because the 5th prime after 53 is next, figure out that this prime is 73, calculate the digit sum 10 and then find the remainder 3. There is just no way that a neural network has learned to do this. But ask for this as a table with prime, digit sum and digit sum mod 7 and it becomes trivial. First prime after 53? Output 59. Digit sum? Output 14. Mod 7? Output 0. Next prime after 59? Output 61. Digit sum? Output 7. Mod 7? Output 0. Next prime after 61? Output 67. Digit sum? Output... This is what chain of thought does, it allows the neural network to only do simple tasks it knows how to do, output the result and continue from there with the next step. You would keep those intermediate results in your short-term memory but a large language model has none and so the best thing it can do is to dump them into the output and read them back when generating the next token. The proposed pause token has essentially the same goal, the neural network can dump some state into the output to work on it but this output is not considered part of the response. It is essentially an attempt to hide the chain of thought process and only respond with the final answer. The hard part will of course be to train the neural network to make efficient use of this, to actually output useful intermediate results before the pause is over and the response gets extracted.

famouswaffles 3y ago

Yes and I'm saying even that doesn't work as robustly as CoT across different tasks. CoT is so obviously not just about padding for extra compute. Certain tokens will guide it to more accurate responses just as much as others derail it out of distribution. This should be obvious. Literally this paper, you can see the model only really learns how to use the padding tokens effectively after training.

danbruc 3y ago
Yes and I'm saying even that doesn't work as robustly as CoT across different tasks. What does not work? CoT is so obviously not just about padding for extra compute. In a certain way it is but the crucial point is that an autoregressive neural network is pretty much a recurrent neural network and chain of thought encourages the use of the feedback path.
- famouswaffles 3y ago
  >What does not work? I'm saying that even if you do what you suggest, it simply doesn't work that well. If you read the paper, you'd see extra computation from padding is not used effectively till after training. Yes, extra compute helps with CoT but it's not even the reason it works. I believe CoT works moreso because it nudges the computation in the right(er) direction.
  - danbruc 3y ago
    List the digit sum mod 7 of the first 10 primes larger than 53. This starts with 0 0 6 1. What do you have to do to generate the next number? Count the numbers in the output to figure out that you have done 4 numbers, add 1 because the 5th prime after 53 is next, figure out that this prime is 73, calculate the digit sum 10 and then find the remainder 3. There is just no way that a neural network has learned to do this. But ask for this as a table with prime, digit sum and digit sum mod 7 and it becomes trivial. First prime after 53? Output 59. Digit sum? Output 14. Mod 7? Output 0. Next prime after 59? Output 61. Digit sum? Output 7. Mod 7? Output 0. Next prime after 61? Output 67. Digit sum? Output... This is what chain of thought does, it allows the neural network to only do simple tasks it knows how to do, output the result and continue from there with the next step. You would keep those intermediate results in your short-term memory but a large language model has none and so the best thing it can do is to dump them into the output and read them back when generating the next token. The proposed pause token has essentially the same goal, the neural network can dump some state into the output to work on it but this output is not considered part of the response. It is essentially an attempt to hide the chain of thought process and only respond with the final answer. The hard part will of course be to train the neural network to make efficient use of this, to actually output useful intermediate results before the pause is over and the response gets extracted.