3 ms·

Can you explain it with reasoning? The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates th

Can you explain it with reasoning? The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates that function to get a set of tokens, and returns the response. Where is there reasoning? If that is sufficient for reasoning, is my calculator reasoning when I input 2+2 and it returns 4?

xeckr 2y ago
>Can you explain it with reasoning? If you were asked to make an HTML button that looks like a watermelon, you would start by considering what a watermelon looks like: a green shell, a red interior, and a circular cross-section. You would then take this information and apply it when writing the CSS for your button. What word would you use to describe this process? >The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates that function to get a set of tokens, and returns the response. Where is there reasoning? The "formula" is dependent on prior knowledge about the world, which is used to successfully solve a problem that does not appear in the training data. Why assume that reasoning is somehow impervious to mathematical modelling?
- elicksaur 2y ago
  Sure, that’s essentially what a person would do. Describing that process says nothing about what an LLM does, which we know by definition of the system is applying a set of weights to a text string input. If you want to reduce all reasoning to just math, then LLMs reason. But then saying something like the OP of “we’ve made computers that reason” is not very novel or useful. Under that framework, computers always reasoned. Does ChatGPT even “solve” this problem? How do you know it’s not in the training data? Example of a tutorial for making a watermelon in css from 2017: https://dev.to/munamohamed94/easy-css-watermelon-slice-animation-tutorial-3ngm https://dev.to/munamohamed94/easy-css-watermelon-slice-anima...
- AnimalMuppet 2y ago
  In this case, it's more like "find a bunch of images that have been associated with the word 'watermelon', then generate some pixels that kind of look like those images". That's not a very high level of reasoning. [Edit: Though I guess it's somewhat decent to realize that, given the request for a button that "looks like X", it needs to go find images of X.]

xeckr 2y ago

>Can you explain it with reasoning? If you were asked to make an HTML button that looks like a watermelon, you would start by considering what a watermelon looks like: a green shell, a red interior, and a circular cross-section. You would then take this information and apply it when writing the CSS for your button. What word would you use to describe this process? >The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates that function to get a set of tokens, and returns the response. Where is there reasoning? The "formula" is dependent on prior knowledge about the world, which is used to successfully solve a problem that does not appear in the training data. Why assume that reasoning is somehow impervious to mathematical modelling?

elicksaur 2y ago
Sure, that’s essentially what a person would do. Describing that process says nothing about what an LLM does, which we know by definition of the system is applying a set of weights to a text string input. If you want to reduce all reasoning to just math, then LLMs reason. But then saying something like the OP of “we’ve made computers that reason” is not very novel or useful. Under that framework, computers always reasoned. Does ChatGPT even “solve” this problem? How do you know it’s not in the training data? Example of a tutorial for making a watermelon in css from 2017: https://dev.to/munamohamed94/easy-css-watermelon-slice-animation-tutorial-3ngm https://dev.to/munamohamed94/easy-css-watermelon-slice-anima...
AnimalMuppet 2y ago
In this case, it's more like "find a bunch of images that have been associated with the word 'watermelon', then generate some pixels that kind of look like those images". That's not a very high level of reasoning. [Edit: Though I guess it's somewhat decent to realize that, given the request for a button that "looks like X", it needs to go find images of X.]