3 ms·

> How do the results of your work inform these predictions that stack-more-layers token-prediction scaling will keep unlocking new abilities? The short answer

> How do the results of your work inform these predictions that stack-more-layers token-prediction scaling will keep unlocking new abilities? The short answer is that scaling up alone probably won't fix these shortcomings. Better decoding strategies like CoT may help, but the issue we're identifying is a limitation of any transformer (no matter the size) arising from its parallelism (the property that enables scaling to large model size/data volume). As some evidence of this, we were able to use these results to find simple problems that ChatGPT and GPT4 get chance performance on. Moreover, the questions seem to lead these models to hallucinate supporting evidence for the incorrect answers: https://arxiv.org/abs/2305.13534 https://arxiv.org/abs/2305.13534

ftxbro 3y ago
Thanks that one is very interesting. I especially like how GPT4 performs worse (zero shot greedy decoding) than ChatGPT on those questions. I wonder have you thought about how it relates to 'inverse scaling' (https://github.com/inverse-scaling/prize https://github.com/inverse-scaling/prize)? As I was reading it I was thinking how that kind of 'hallucination' is exactly the problem that beam search (as opposed to greedy decoding) is designed to solve, and I see this was addressed later in the paper - unfortunately you guys didn't have access to it on the API and also no access to other things like log likelihoods. A related question I had, is to decompose the blame for the problems these LLMs were having into two parts: (1) the weaknesses of the model size and data size and architecture caused for example by parallelism tradeoffs versus (2) the inherent problems of greedy token prediction. For example I could imagine that even an 'oracular' (kolmogorov or solomonoff or whoever absolute minimum theoretical perplexity) single-token-at-a-time-predictor could have related errors purely as a result of greedy decoding, which would be in no way attributable to any kind of size issue or architectural property or parallelism property of the model. As I guess you already know, the reason is just that 'the most likely next token of the response' isn't necessarily the same as 'the next token of the most likely response'.

ftxbro 3y ago

Thanks that one is very interesting. I especially like how GPT4 performs worse (zero shot greedy decoding) than ChatGPT on those questions. I wonder have you thought about how it relates to 'inverse scaling' (https://github.com/inverse-scaling/prize https://github.com/inverse-scaling/prize)? As I was reading it I was thinking how that kind of 'hallucination' is exactly the problem that beam search (as opposed to greedy decoding) is designed to solve, and I see this was addressed later in the paper - unfortunately you guys didn't have access to it on the API and also no access to other things like log likelihoods. A related question I had, is to decompose the blame for the problems these LLMs were having into two parts: (1) the weaknesses of the model size and data size and architecture caused for example by parallelism tradeoffs versus (2) the inherent problems of greedy token prediction. For example I could imagine that even an 'oracular' (kolmogorov or solomonoff or whoever absolute minimum theoretical perplexity) single-token-at-a-time-predictor could have related errors purely as a result of greedy decoding, which would be in no way attributable to any kind of size issue or architectural property or parallelism property of the model. As I guess you already know, the reason is just that 'the most likely next token of the response' isn't necessarily the same as 'the next token of the most likely response'.