4 ms·

Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

godelski 3y ago
Neither of these comments are accurate. (edit: but renegade-otter is more correct) Here's 1.5 EMA https://imgur.com/mJPKuIb https://imgur.com/mJPKuIb Here's 2.0 EMA https://imgur.com/KrPVUGy https://imgur.com/KrPVUGy No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1. Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV https://imgur.com/Dmx78xV, https://imgur.com/HBTitWm https://imgur.com/HBTitWm But I changed the prompt and switched to DPM++ 3M SDE Karras Positive: beautiful woman sitting on a bus reading a book,(detailed [face|eyes],detailed [hands|fingers]:1.2),Tokyo city,sitting next to a window with the city outside,detailed book,(8k HDR RAW Fuji film:0.9),perfect reflections,best quality,(masterpiece:1.2),beautiful Negative: ugly,low quality,worst quality,medium quality,deformed,bad hands,ugly face,deformed book,bad text,extra fingers We can do even better if we use LoRAs and textual inversions, or better checkpoints. But there's a lot of work that goes into making really high quality photos with these models. Edit: here is switching to Cyberrealistic checkpoint: https://imgur.com/gFMkg0J https://imgur.com/gFMkg0J, And here's adding some LoRAs, TIs, and prompt engineering: https://imgur.com/VklfVVC https://imgur.com/VklfVVC (https://imgur.com/ZrAtluS https://imgur.com/ZrAtluS, https://imgur.com/cYQajMN https://imgur.com/cYQajMN), https://imgur.com/ci2JTJl https://imgur.com/ci2JTJl (https://imgur.com/9tEhzHF https://imgur.com/9tEhzHF, https://imgur.com/4Ck03P7 https://imgur.com/4Ck03P7). I can get better, but I don't feel too much like it just to prove a point.
- TacticalCoder 3y ago
  > I can get better, but I don't feel too much like it just to prove a point. Honestly these pictures you posted do prove GP's point...
  - godelski 3y ago
    > Honestly these pictures you posted do prove GP's point... Sorry, which person's? HeatrayEnjoyer's? I don't think it does since there are a ton of mistakes. And the better ones come with a lot of work and a whole lot of experience. Or renegade-otter's (GGP)? I wouldn't call it a horror show, but I can see how others would. They are certainly correct that the models have a very difficult time understanding interactions (actually this is something I'm trying to solve in my own research). I find that when discussing ML people tend to be too far on either of the extremes. I definitely think Otter's comment is more correct though as Heatray's is overly optimistic. Images are often fantastic if you only look at them with a glance. Scrolling through twitter or a blog or whatever. Often that's good enough though. But if we are to actually look with care, I think you start to see a strange unrealistic world. Sora's demos have been a great example of exactly this phenomena. They are all great. But if you look with care, all have errors that you'll probably be surprised you didn't notice before. You'll probably be surprised how bad of errors slipped right by. I think that's actually interesting in itself.
    - justinclift 3y ago
      > Sorry, which person's? Probably this one: https://news.ycombinator.com/item?id=39502539 https://news.ycombinator.com/item?id=39502539
- renegade-otter 3y ago
  You kind of proved my point. Of course the "finger situation" is getting better but people handling complex objects is still where these tools trip. They can't reason about it - they just need to see enough data of people handling books. On a bus. Now do this for ALL possible objects in the world. I have generated hundreds of these - the bus cabin LOOKS like a bus cabin, but it's a plausible fake - the poles abruptly terminate, the seats are in weird unrealistic configurations, unnatural single-row isles, etc. Which is why I called it a super-convincing autocomplete.
  - mr_toad 3y ago
    Put a brain in a jar and expose it only to photos and you’d get the same results. It’s hard to learn what holding a book is like if you’ve never held anything.
    - godelski 3y ago
      I'm not sure I see the utility of this "thought experiment." It's not really provable and not even possible to do. Not to mention highly unethical. I mean just make the embodiment argument instead. But we don't know if embodiment is necessary, or even what kind of body is even necessary.
  - godelski 3y ago
    > You kind of proved my point. Yeah I did say you were more right. But it was difficult to distinguish exaggeration from actual intent. You can check my comment history of me battling the common ML mindset. I love the area of study (I'm a researcher myself) but there's a lot of problems that even in the research community a lot want to ignore. It's odd to me. It's been hilarious to watch big names claim Sora understands physics. Or people think just because it doesn't understand physics that the videos aren't still impressive and even useful. But with how you updated your language, I think we are in a very high level of agreement. You are perfectly right: no ML model "understands" anything. GPT doesn't understand how to code and image models don't understand how to... art(?) or do physics or whatever. They don't have world models. And I'm deeply frustrated that people think a single example of a accurately acting like a world model is proof and will do gymnastics to say a single counter example isn't. A single counter does disprove a world model and to understand you need to be able to self-correct. Hallucinations are fine but "are you sure?" should be enough to get it to reconsider, not double down or just switch. We can be fooled with setups, but we laugh at ourselves quickly because we self-correct fairly easily (or rather, we can).
paulmd 3y ago
> Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new. it's true that most people do not actually understand the problem/limitation, but it's a discussion that is statistically likely to occur on the internet and therefore people tend to regurgitate the words without understanding the concept. I'm being facetious but honestly it's a major theme of this whole AI revolution, people do not want to accept that humans are just another kind of machine and that their own cognition resembles AI/ML in virtually every aspect. People confabulate. People overreach the bounds of their expertise. People repeat words and concepts without properly understanding the larger context in which they need to be applied. Etc etc. Has nobody ever watched someone get asked a big question or an unexpected question and "watched the wheels turn", or watched them stammer out some slop of incoherent words while they're processing? Does nobody have "canned responses" that summarize a topic that you can give pretty much the same (but not exactly, of course) every time you are asked it? Is that not "stochastic word chains"? By design neural nets work almost exactly the same as your brain. But a lot of people are trapped in the idea that there must be some kind of "soul" or something that makes human cognition fundamentally different. By design, it's not. And we don't fully understand the exact modalities to encode information in it usefully and process it yet, but that's what the whole process here is about. (I commented about this maybe 6 months ago, but the real hot take is that what we think of as "consciousness" isn't a real thing, or even an "overseer" within the mind - "consciousness" may be exactly the thing people mean when they say that "LLMs have to write a word every time they think about a concept". "Consciousness" may in fact be a low-dimensional projection of the actual computation occurring in the brain itself, rationalizing and explicating the symbolic computations of the brain in some form that can be written down and communicated to other humans. "Language" and "consciousness" as top-level concepts may actually only be an annex that our brain has built for storing and communicating those symbolic computations, rather than a primary driver of the computations itself. It's not in control, it's only explaining decisions that we already have made... we see the shadows on the wall of plato's cave and think that's the entire world, but it's really only a low-dimensional projection.) (or in other words - everyone assumes consciousness is the OS, or at least the application. But consciousness may actually be the json serializer/deserializer - ie not actually the thing in control at all. Our entire lives and decisionmaking processes may in fact be simple rationalizations and explanations around "what the subconscious mind thinks should happen next".)

godelski 3y ago

Neither of these comments are accurate. (edit: but renegade-otter is more correct) Here's 1.5 EMA https://imgur.com/mJPKuIb https://imgur.com/mJPKuIb Here's 2.0 EMA https://imgur.com/KrPVUGy https://imgur.com/KrPVUGy No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1. Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV https://imgur.com/Dmx78xV, https://imgur.com/HBTitWm https://imgur.com/HBTitWm But I changed the prompt and switched to DPM++ 3M SDE Karras Positive: beautiful woman sitting on a bus reading a book,(detailed [face|eyes],detailed [hands|fingers]:1.2),Tokyo city,sitting next to a window with the city outside,detailed book,(8k HDR RAW Fuji film:0.9),perfect reflections,best quality,(masterpiece:1.2),beautiful Negative: ugly,low quality,worst quality,medium quality,deformed,bad hands,ugly face,deformed book,bad text,extra fingers We can do even better if we use LoRAs and textual inversions, or better checkpoints. But there's a lot of work that goes into making really high quality photos with these models. Edit: here is switching to Cyberrealistic checkpoint: https://imgur.com/gFMkg0J https://imgur.com/gFMkg0J, And here's adding some LoRAs, TIs, and prompt engineering: https://imgur.com/VklfVVC https://imgur.com/VklfVVC (https://imgur.com/ZrAtluS https://imgur.com/ZrAtluS, https://imgur.com/cYQajMN https://imgur.com/cYQajMN), https://imgur.com/ci2JTJl https://imgur.com/ci2JTJl (https://imgur.com/9tEhzHF https://imgur.com/9tEhzHF, https://imgur.com/4Ck03P7 https://imgur.com/4Ck03P7). I can get better, but I don't feel too much like it just to prove a point.

TacticalCoder 3y ago
> I can get better, but I don't feel too much like it just to prove a point. Honestly these pictures you posted do prove GP's point...
- godelski 3y ago
  > Honestly these pictures you posted do prove GP's point... Sorry, which person's? HeatrayEnjoyer's? I don't think it does since there are a ton of mistakes. And the better ones come with a lot of work and a whole lot of experience. Or renegade-otter's (GGP)? I wouldn't call it a horror show, but I can see how others would. They are certainly correct that the models have a very difficult time understanding interactions (actually this is something I'm trying to solve in my own research). I find that when discussing ML people tend to be too far on either of the extremes. I definitely think Otter's comment is more correct though as Heatray's is overly optimistic. Images are often fantastic if you only look at them with a glance. Scrolling through twitter or a blog or whatever. Often that's good enough though. But if we are to actually look with care, I think you start to see a strange unrealistic world. Sora's demos have been a great example of exactly this phenomena. They are all great. But if you look with care, all have errors that you'll probably be surprised you didn't notice before. You'll probably be surprised how bad of errors slipped right by. I think that's actually interesting in itself.
  - justinclift 3y ago
    > Sorry, which person's? Probably this one: https://news.ycombinator.com/item?id=39502539 https://news.ycombinator.com/item?id=39502539
renegade-otter 3y ago
You kind of proved my point. Of course the "finger situation" is getting better but people handling complex objects is still where these tools trip. They can't reason about it - they just need to see enough data of people handling books. On a bus. Now do this for ALL possible objects in the world. I have generated hundreds of these - the bus cabin LOOKS like a bus cabin, but it's a plausible fake - the poles abruptly terminate, the seats are in weird unrealistic configurations, unnatural single-row isles, etc. Which is why I called it a super-convincing autocomplete.
- mr_toad 3y ago
  Put a brain in a jar and expose it only to photos and you’d get the same results. It’s hard to learn what holding a book is like if you’ve never held anything.
  - godelski 3y ago
    I'm not sure I see the utility of this "thought experiment." It's not really provable and not even possible to do. Not to mention highly unethical. I mean just make the embodiment argument instead. But we don't know if embodiment is necessary, or even what kind of body is even necessary.
- godelski 3y ago
  > You kind of proved my point. Yeah I did say you were more right. But it was difficult to distinguish exaggeration from actual intent. You can check my comment history of me battling the common ML mindset. I love the area of study (I'm a researcher myself) but there's a lot of problems that even in the research community a lot want to ignore. It's odd to me. It's been hilarious to watch big names claim Sora understands physics. Or people think just because it doesn't understand physics that the videos aren't still impressive and even useful. But with how you updated your language, I think we are in a very high level of agreement. You are perfectly right: no ML model "understands" anything. GPT doesn't understand how to code and image models don't understand how to... art(?) or do physics or whatever. They don't have world models. And I'm deeply frustrated that people think a single example of a accurately acting like a world model is proof and will do gymnastics to say a single counter example isn't. A single counter does disprove a world model and to understand you need to be able to self-correct. Hallucinations are fine but "are you sure?" should be enough to get it to reconsider, not double down or just switch. We can be fooled with setups, but we laugh at ourselves quickly because we self-correct fairly easily (or rather, we can).

paulmd 3y ago

> Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new. it's true that most people do not actually understand the problem/limitation, but it's a discussion that is statistically likely to occur on the internet and therefore people tend to regurgitate the words without understanding the concept. I'm being facetious but honestly it's a major theme of this whole AI revolution, people do not want to accept that humans are just another kind of machine and that their own cognition resembles AI/ML in virtually every aspect. People confabulate. People overreach the bounds of their expertise. People repeat words and concepts without properly understanding the larger context in which they need to be applied. Etc etc. Has nobody ever watched someone get asked a big question or an unexpected question and "watched the wheels turn", or watched them stammer out some slop of incoherent words while they're processing? Does nobody have "canned responses" that summarize a topic that you can give pretty much the same (but not exactly, of course) every time you are asked it? Is that not "stochastic word chains"? By design neural nets work almost exactly the same as your brain. But a lot of people are trapped in the idea that there must be some kind of "soul" or something that makes human cognition fundamentally different. By design, it's not. And we don't fully understand the exact modalities to encode information in it usefully and process it yet, but that's what the whole process here is about. (I commented about this maybe 6 months ago, but the real hot take is that what we think of as "consciousness" isn't a real thing, or even an "overseer" within the mind - "consciousness" may be exactly the thing people mean when they say that "LLMs have to write a word every time they think about a concept". "Consciousness" may in fact be a low-dimensional projection of the actual computation occurring in the brain itself, rationalizing and explicating the symbolic computations of the brain in some form that can be written down and communicated to other humans. "Language" and "consciousness" as top-level concepts may actually only be an annex that our brain has built for storing and communicating those symbolic computations, rather than a primary driver of the computations itself. It's not in control, it's only explaining decisions that we already have made... we see the shadows on the wall of plato's cave and think that's the entire world, but it's really only a low-dimensional projection.) (or in other words - everyone assumes consciousness is the OS, or at least the application. But consciousness may actually be the json serializer/deserializer - ie not actually the thing in control at all. Our entire lives and decisionmaking processes may in fact be simple rationalizations and explanations around "what the subconscious mind thinks should happen next".)