8 ms·

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading a book". Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works. It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw in any complex interactions it has not seen in the training data and it's all over.

deleted 3y ago
[deleted]
sshumaker 3y ago
You’re being downvoted because this is a hot take that isn’t supported by evidence. I just tried exactly that with dalle-3 and it worked well. More to the point, it’s pretty clear LLMs do form a model of the world, that’s exactly how they reason about things. There was some good experiments on this a while back - check out the Othello experiment. https://thegradient.pub/othello/ https://thegradient.pub/othello/
- godelski 3y ago
  It's a hot take, but it is supported by a ton of evidence.
HDThoreaun 3y ago
I think the situation is a lot more complicated than youre making it out to be. GPT4 for example can be very good at tasks it has not seen in the training data. The philosophy of mind is much more open ended and less understood than you seem to think.
- godelski 3y ago
  > not seen in the training data Do you have some evidence for this?
- renegade-otter 3y ago
  What's the evidence? OpenAI's claims? They do have an inherent interest is making investors believe this technology is magic.
  - xetplan 3y ago
    There is no evidence. This entire line of thought is really stupid when chatGPT itself will tell you it doesn't understand the responses it gives. Try the prompt in chatGPT4 "Can you explain to a human how you don't understand anything in the response you give to this question?" For those that don't have chatGPT4. chatGPT4: "Imagine you're using a very advanced calculator that can help you solve all kinds of math problems, from simple additions to complex equations. Now, let's say you ask this calculator a question, like what's the sum of 2 plus 2. The calculator quickly tells you the answer is 4. But here's the key part: the calculator doesn't really "understand" what 4 means, or what adding numbers is all about. It doesn't know that if you have two apples and get two more, you'll have four apples. It just follows a set of rules (its programming) to come up with the answer based on the input you gave it. I work in a similar way. When you ask me a question, I search through a huge amount of information I've been trained on to find the best possible response based on patterns and data. However, I don't "understand" this information in the way humans do. I don't have personal experiences, emotions, or consciousness. I can't feel joy when I solve a problem or frustration when I can't find a solution. I'm like a very advanced book that can find and present information on its own but doesn't know what any of it truly means." I am sure at some point people will start arguing that chatGPT is just bullshitting about this. This line of thought is so moronic and dangerous.
    - godelski 3y ago
      To save people time, here's the inverse > Can you explain to a human how you understand things and respond to this question? GPT: As an AI language model, I don't have understanding in the way humans do. My "responses" are generated based on statistical patterns and relationships in the data I've been trained on. When you ask a question, I analyze the text, identify keywords and context, and then generate a response based on the most probable language patterns that align with the input received. My process is purely computational and lacks consciousness or genuine comprehension. It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat. I used to use the feathers/bricks question but give it different units (kilograms and pounds), but they patched that. But you only need a single example to show it doesn't understand. People will try to prove it understands by asking follow-up questions, but they don't realize that the questions they act spoil the answer. It is very hard to ask follow-ups and not give away the answer.
- goatlover 3y ago
  How do you know what's in the training data? Has OpenAI made the dataset searchable so we can see that GPT4 is performing tasks not in there?
HeatrayEnjoyer 3y ago
Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.
- godelski 3y ago
  Neither of these comments are accurate. (edit: but renegade-otter is more correct) Here's 1.5 EMA https://imgur.com/mJPKuIb https://imgur.com/mJPKuIb Here's 2.0 EMA https://imgur.com/KrPVUGy https://imgur.com/KrPVUGy No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1. Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV https://imgur.com/Dmx78xV, https://imgur.com/HBTitWm https://imgur.com/HBTitWm But I changed the prompt and switched to DPM++ 3M SDE Karras Positive: beautiful woman sitting on a bus reading a book,(detailed [face|eyes],detailed [hands|fingers]:1.2),Tokyo city,sitting next to a window with the city outside,detailed book,(8k HDR RAW Fuji film:0.9),perfect reflections,best quality,(masterpiece:1.2),beautiful Negative: ugly,low quality,worst quality,medium quality,deformed,bad hands,ugly face,deformed book,bad text,extra fingers We can do even better if we use LoRAs and textual inversions, or better checkpoints. But there's a lot of work that goes into making really high quality photos with these models. Edit: here is switching to Cyberrealistic checkpoint: https://imgur.com/gFMkg0J https://imgur.com/gFMkg0J, And here's adding some LoRAs, TIs, and prompt engineering: https://imgur.com/VklfVVC https://imgur.com/VklfVVC (https://imgur.com/ZrAtluS https://imgur.com/ZrAtluS, https://imgur.com/cYQajMN https://imgur.com/cYQajMN), https://imgur.com/ci2JTJl https://imgur.com/ci2JTJl (https://imgur.com/9tEhzHF https://imgur.com/9tEhzHF, https://imgur.com/4Ck03P7 https://imgur.com/4Ck03P7). I can get better, but I don't feel too much like it just to prove a point.
  - TacticalCoder 3y ago
    > I can get better, but I don't feel too much like it just to prove a point. Honestly these pictures you posted do prove GP's point...
    - godelski 3y ago
      > Honestly these pictures you posted do prove GP's point... Sorry, which person's? HeatrayEnjoyer's? I don't think it does since there are a ton of mistakes. And the better ones come with a lot of work and a whole lot of experience. Or renegade-otter's (GGP)? I wouldn't call it a horror show, but I can see how others would. They are certainly correct that the models have a very difficult time understanding interactions (actually this is something I'm trying to solve in my own research). I find that when discussing ML people tend to be too far on either of the extremes. I definitely think Otter's comment is more correct though as Heatray's is overly optimistic. Images are often fantastic if you only look at them with a glance. Scrolling through twitter or a blog or whatever. Often that's good enough though. But if we are to actually look with care, I think you start to see a strange unrealistic world. Sora's demos have been a great example of exactly this phenomena. They are all great. But if you look with care, all have errors that you'll probably be surprised you didn't notice before. You'll probably be surprised how bad of errors slipped right by. I think that's actually interesting in itself.

deleted 3y ago

[deleted]

sshumaker 3y ago

You’re being downvoted because this is a hot take that isn’t supported by evidence. I just tried exactly that with dalle-3 and it worked well. More to the point, it’s pretty clear LLMs do form a model of the world, that’s exactly how they reason about things. There was some good experiments on this a while back - check out the Othello experiment. https://thegradient.pub/othello/ https://thegradient.pub/othello/

godelski 3y ago
It's a hot take, but it is supported by a ton of evidence.

HDThoreaun 3y ago

I think the situation is a lot more complicated than youre making it out to be. GPT4 for example can be very good at tasks it has not seen in the training data. The philosophy of mind is much more open ended and less understood than you seem to think.

godelski 3y ago
> not seen in the training data Do you have some evidence for this?
renegade-otter 3y ago
What's the evidence? OpenAI's claims? They do have an inherent interest is making investors believe this technology is magic.
- xetplan 3y ago
  There is no evidence. This entire line of thought is really stupid when chatGPT itself will tell you it doesn't understand the responses it gives. Try the prompt in chatGPT4 "Can you explain to a human how you don't understand anything in the response you give to this question?" For those that don't have chatGPT4. chatGPT4: "Imagine you're using a very advanced calculator that can help you solve all kinds of math problems, from simple additions to complex equations. Now, let's say you ask this calculator a question, like what's the sum of 2 plus 2. The calculator quickly tells you the answer is 4. But here's the key part: the calculator doesn't really "understand" what 4 means, or what adding numbers is all about. It doesn't know that if you have two apples and get two more, you'll have four apples. It just follows a set of rules (its programming) to come up with the answer based on the input you gave it. I work in a similar way. When you ask me a question, I search through a huge amount of information I've been trained on to find the best possible response based on patterns and data. However, I don't "understand" this information in the way humans do. I don't have personal experiences, emotions, or consciousness. I can't feel joy when I solve a problem or frustration when I can't find a solution. I'm like a very advanced book that can find and present information on its own but doesn't know what any of it truly means." I am sure at some point people will start arguing that chatGPT is just bullshitting about this. This line of thought is so moronic and dangerous.
  - godelski 3y ago
    To save people time, here's the inverse > Can you explain to a human how you understand things and respond to this question? GPT: As an AI language model, I don't have understanding in the way humans do. My "responses" are generated based on statistical patterns and relationships in the data I've been trained on. When you ask a question, I analyze the text, identify keywords and context, and then generate a response based on the most probable language patterns that align with the input received. My process is purely computational and lacks consciousness or genuine comprehension. It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat. I used to use the feathers/bricks question but give it different units (kilograms and pounds), but they patched that. But you only need a single example to show it doesn't understand. People will try to prove it understands by asking follow-up questions, but they don't realize that the questions they act spoil the answer. It is very hard to ask follow-ups and not give away the answer.
goatlover 3y ago
How do you know what's in the training data? Has OpenAI made the dataset searchable so we can see that GPT4 is performing tasks not in there?

HeatrayEnjoyer 3y ago

Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

godelski 3y ago
Neither of these comments are accurate. (edit: but renegade-otter is more correct) Here's 1.5 EMA https://imgur.com/mJPKuIb https://imgur.com/mJPKuIb Here's 2.0 EMA https://imgur.com/KrPVUGy https://imgur.com/KrPVUGy No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1. Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV https://imgur.com/Dmx78xV, https://imgur.com/HBTitWm https://imgur.com/HBTitWm But I changed the prompt and switched to DPM++ 3M SDE Karras Positive: beautiful woman sitting on a bus reading a book,(detailed [face|eyes],detailed [hands|fingers]:1.2),Tokyo city,sitting next to a window with the city outside,detailed book,(8k HDR RAW Fuji film:0.9),perfect reflections,best quality,(masterpiece:1.2),beautiful Negative: ugly,low quality,worst quality,medium quality,deformed,bad hands,ugly face,deformed book,bad text,extra fingers We can do even better if we use LoRAs and textual inversions, or better checkpoints. But there's a lot of work that goes into making really high quality photos with these models. Edit: here is switching to Cyberrealistic checkpoint: https://imgur.com/gFMkg0J https://imgur.com/gFMkg0J, And here's adding some LoRAs, TIs, and prompt engineering: https://imgur.com/VklfVVC https://imgur.com/VklfVVC (https://imgur.com/ZrAtluS https://imgur.com/ZrAtluS, https://imgur.com/cYQajMN https://imgur.com/cYQajMN), https://imgur.com/ci2JTJl https://imgur.com/ci2JTJl (https://imgur.com/9tEhzHF https://imgur.com/9tEhzHF, https://imgur.com/4Ck03P7 https://imgur.com/4Ck03P7). I can get better, but I don't feel too much like it just to prove a point.
- TacticalCoder 3y ago
  > I can get better, but I don't feel too much like it just to prove a point. Honestly these pictures you posted do prove GP's point...
  - godelski 3y ago
    > Honestly these pictures you posted do prove GP's point... Sorry, which person's? HeatrayEnjoyer's? I don't think it does since there are a ton of mistakes. And the better ones come with a lot of work and a whole lot of experience. Or renegade-otter's (GGP)? I wouldn't call it a horror show, but I can see how others would. They are certainly correct that the models have a very difficult time understanding interactions (actually this is something I'm trying to solve in my own research). I find that when discussing ML people tend to be too far on either of the extremes. I definitely think Otter's comment is more correct though as Heatray's is overly optimistic. Images are often fantastic if you only look at them with a glance. Scrolling through twitter or a blog or whatever. Often that's good enough though. But if we are to actually look with care, I think you start to see a strange unrealistic world. Sora's demos have been a great example of exactly this phenomena. They are all great. But if you look with care, all have errors that you'll probably be surprised you didn't notice before. You'll probably be surprised how bad of errors slipped right by. I think that's actually interesting in itself.