3 ms·

LLMs are like perfect, brilliant high-performance cheating students on steroids. They are not optimized for understanding and reasoning, they optimized to give

LLMs are like perfect, brilliant high-performance cheating students on steroids. They are not optimized for understanding and reasoning, they optimized to give you plausible-looking answers. So yes, they will cheat their way to good scores and it will be progressively harder to catch them on that.

williamcotton 3y ago
An LLM has a very deep and broad model of textual language as used by humans. It allows for a prompt of a sequence of text to be completed in a manner that can translate one human language into another, even formal languages like programming languages. It is inconsequential whether or not we consider this understanding or reasoning to the point where such a conversation seems useless. It is a categorical error to attribute cheating to such a process. Just ask if such a tool is useful and then choose to learn how the tool could work to your advantage.
- mecsred 3y ago
  Pro tip: anyone this argument would work on is currently learning about and tinkering with the tech. The people stuck in the intelligence speculation are tautologically the people you can't stop speculating.
  - williamcotton 3y ago
    What if I hit them in the head with a copy of Wittgenstein’s Philosophical Investigations until they stop?
- enragedcacti 3y ago
  > Just ask if such a tool is useful and then choose to learn how the tool could work to your advantage. Wouldn't a great first step to understanding a tools capabilities, advantages, and disadvantages be mapping it to familiar concepts like understanding, reasoning, or cheating? It is clear from examples of misuse that many users do think that LLMs are capable of things that they are not so maybe conversations about the proper analogies aren't completely useless.
mo_42 3y ago
I disagree. The way I imagine LLMs is that we feed huge amounts of text to a ANN. This ANN doesn't have enough weights to just memorize these huge amounts. So it has to find meaningful abstractions from the texts to make the loss function small. Let's even ignore regularization for now. For example, it will not memorize that all the instances in the context of earth fall on the ground. It will learn a the meaningful abstraction gravitation. From that it infers that objects should also fall on the ground. This should be very easy to verify. Moreover, I have also seen some interesting examples of spacial reasoning in ChatGPT, which seems much more complicated than my example. Edit: I think understanding and finding suitable abstractions is the same thing. That's what we do with children and students. They may try to memorize everything we teach them. But they should find abstractions and commonalities that are useful in more than one instance.

williamcotton 3y ago

An LLM has a very deep and broad model of textual language as used by humans. It allows for a prompt of a sequence of text to be completed in a manner that can translate one human language into another, even formal languages like programming languages. It is inconsequential whether or not we consider this understanding or reasoning to the point where such a conversation seems useless. It is a categorical error to attribute cheating to such a process. Just ask if such a tool is useful and then choose to learn how the tool could work to your advantage.

mecsred 3y ago
Pro tip: anyone this argument would work on is currently learning about and tinkering with the tech. The people stuck in the intelligence speculation are tautologically the people you can't stop speculating.
- williamcotton 3y ago
  What if I hit them in the head with a copy of Wittgenstein’s Philosophical Investigations until they stop?
enragedcacti 3y ago
> Just ask if such a tool is useful and then choose to learn how the tool could work to your advantage. Wouldn't a great first step to understanding a tools capabilities, advantages, and disadvantages be mapping it to familiar concepts like understanding, reasoning, or cheating? It is clear from examples of misuse that many users do think that LLMs are capable of things that they are not so maybe conversations about the proper analogies aren't completely useless.

mo_42 3y ago

I disagree. The way I imagine LLMs is that we feed huge amounts of text to a ANN. This ANN doesn't have enough weights to just memorize these huge amounts. So it has to find meaningful abstractions from the texts to make the loss function small. Let's even ignore regularization for now. For example, it will not memorize that all the instances in the context of earth fall on the ground. It will learn a the meaningful abstraction gravitation. From that it infers that objects should also fall on the ground. This should be very easy to verify. Moreover, I have also seen some interesting examples of spacial reasoning in ChatGPT, which seems much more complicated than my example. Edit: I think understanding and finding suitable abstractions is the same thing. That's what we do with children and students. They may try to memorize everything we teach them. But they should find abstractions and commonalities that are useful in more than one instance.