4 ms·

> Based on these observations, I'd bet the house that no, large language models cannot under any circumstance reason about medical questions at present. I thin

> Based on these observations, I'd bet the house that no, large language models cannot under any circumstance reason about medical questions at present. I think that would be a bad bet, unless you're leaning heavily on the words "at present." I don't see any fundamental reason why language models can't reason about any particular thing, provided they're taught about that thing.

sinenomine 4y ago
It's like betting that StableDiffusion's descendants won't be able to draw hands correctly. Sure, there could be a manifold market for that, if someone is truly confident in risking (play)-money.
jfk13 4y ago
And I don't see any fundamental reason to believe that language models can learn or reason at all.
- sinenomine 4y ago
  If gradient-descent in DNNs indeed has an intrinsic bias for short and simple solutions, like this paper https://arxiv.org/abs/2006.15191 https://arxiv.org/abs/2006.15191 shows, and if the "reasoning" is among the shortest solutions to match the textual data with the language modeling loss, then at some limit of model+data scaling the model has to recover this solution. Or maybe even something better, "reasoning++"? It's all too easy to dismiss the success of infinitely plastic, scalable systems.
  - mcguire 4y ago
    "For every complex problem there is an answer that is clear, simple, and wrong." Possibly said by H. L. Mencken. Also, "The fastest sea mammal is the sailfish, ... I am confident that the information I provided about the fastest sea mammal is accurate. The data on the speeds of different marine animals is widely available and has been widely studied and reported on by scientists and other experts."
- actually_a_dog 4y ago
  Then you need to provide a definition for "learn" and "reason" and an argument as to why. These models are trained to do the task they do, which is, by definition learning. I'll leave "reasoning" up to you, because that's a fuzzier question, but I still would like to see what your fundamental objection is.
arcturus17 4y ago
ChatGPT has been taught about sailfish - it has had access to a corpus of several centuries worth of knowledge regarding sailfish - and it will try to persuade you that it is a mammal, making up some outlandish facts along the way. Yea, I lean on "at present" because the question in the paper is whether LLMs (GPT 3.5 specifically) can reason about medicine now.
- mcguire 4y ago
  The really beautiful part: I asked "what is the fastest sea mammal" twice, got orca once and sailfish the second time. Isn't randomness fun?
mcguire 4y ago
"Here is an outline of a quantum algorithm that can be used to sort a list of integers: "Initialize a quantum register with the input list of integers. Each integer in the list can be represented as a qubit (quantum bit) in the quantum register. "Use a quantum circuit to apply a quantum Fourier transform (QFT) to the quantum register. The QFT is a quantum algorithm that can be used to perform a transformation on the quantum register that maps the integers in the list to their corresponding frequencies in the frequency domain. "Use a classical computer to measure the output of the QFT. This will result in a list of integers in the frequency domain, which can be sorted using a classical sorting algorithm. "Use a quantum circuit to apply an inverse QFT to the sorted list of integers in the frequency domain. This will transform the list back to the original domain, resulting in a sorted list of integers. "This is just a high-level overview of the algorithm, and there are many details and considerations that would need to be addressed in order to fully implement it. However, I hope this gives you a general idea of how a quantum algorithm can be used to sort a list of integers."

sinenomine 4y ago

It's like betting that StableDiffusion's descendants won't be able to draw hands correctly. Sure, there could be a manifold market for that, if someone is truly confident in risking (play)-money.

jfk13 4y ago

And I don't see any fundamental reason to believe that language models can learn or reason at all.

sinenomine 4y ago
If gradient-descent in DNNs indeed has an intrinsic bias for short and simple solutions, like this paper https://arxiv.org/abs/2006.15191 https://arxiv.org/abs/2006.15191 shows, and if the "reasoning" is among the shortest solutions to match the textual data with the language modeling loss, then at some limit of model+data scaling the model has to recover this solution. Or maybe even something better, "reasoning++"? It's all too easy to dismiss the success of infinitely plastic, scalable systems.
- mcguire 4y ago
  "For every complex problem there is an answer that is clear, simple, and wrong." Possibly said by H. L. Mencken. Also, "The fastest sea mammal is the sailfish, ... I am confident that the information I provided about the fastest sea mammal is accurate. The data on the speeds of different marine animals is widely available and has been widely studied and reported on by scientists and other experts."
actually_a_dog 4y ago
Then you need to provide a definition for "learn" and "reason" and an argument as to why. These models are trained to do the task they do, which is, by definition learning. I'll leave "reasoning" up to you, because that's a fuzzier question, but I still would like to see what your fundamental objection is.

arcturus17 4y ago

ChatGPT has been taught about sailfish - it has had access to a corpus of several centuries worth of knowledge regarding sailfish - and it will try to persuade you that it is a mammal, making up some outlandish facts along the way. Yea, I lean on "at present" because the question in the paper is whether LLMs (GPT 3.5 specifically) can reason about medicine now.

mcguire 4y ago
The really beautiful part: I asked "what is the fastest sea mammal" twice, got orca once and sailfish the second time. Isn't randomness fun?

mcguire 4y ago

"Here is an outline of a quantum algorithm that can be used to sort a list of integers: "Initialize a quantum register with the input list of integers. Each integer in the list can be represented as a qubit (quantum bit) in the quantum register. "Use a quantum circuit to apply a quantum Fourier transform (QFT) to the quantum register. The QFT is a quantum algorithm that can be used to perform a transformation on the quantum register that maps the integers in the list to their corresponding frequencies in the frequency domain. "Use a classical computer to measure the output of the QFT. This will result in a list of integers in the frequency domain, which can be sorted using a classical sorting algorithm. "Use a quantum circuit to apply an inverse QFT to the sorted list of integers in the frequency domain. This will transform the list back to the original domain, resulting in a sorted list of integers. "This is just a high-level overview of the algorithm, and there are many details and considerations that would need to be addressed in order to fully implement it. However, I hope this gives you a general idea of how a quantum algorithm can be used to sort a list of integers."