3 ms·

In my opinion, image generation models will replace some jobs - the low-value stuff that's basically "I need somebody to draw a bunch of random things and I don

In my opinion, image generation models will replace some jobs - the low-value stuff that's basically "I need somebody to draw a bunch of random things and I don't particularly care about the outcome". Now, moving up the value chain we find ourselves at design agencies that do 2 things: they gather requirements and then they convert those requirements into an output. Many times a client themselves don't know what they want, and part of a good agency's job is turning nebulous requirements into something usable (and charging through the roof for it). The technology will make images much more accessible - the same way Google translate is crap, but it opened up me communicating with a random foreign dude when travelling in a new country that nobody would've ever hired a translator for. The translators moved up market to industries where it's important that a human verified everything looks good like official government/corporate publications. Same thing here; one day you'll have these models in powerpoint and people will click on the model when they need some kind of image there. The high-end won't go away, but the low-value work almost certainly will.

godelski 4y ago
As a computer vision researcher who has worked with these T2I generators, this is more spot on. I think a lot of people are making a lot of assumptions without having actually worked with these networks. There's also a highly fundamental problem that is not only unsolved, but we don't even have great directions to solve: alignment. Alignment is even a difficult task when you're communicating to another person. Everyone has had an experience of working with someone and they are asking you to do something (or vise versa) and there is a gross misinterpretation. That's alignment. Now try properly aligning with an "entity" that thinks nothing like you, has a vastly different set of experiences, doesn't know how to count, and has no real concept of physical mechanisms (and more). The technology will definitely help us see a lot more unique art and a lot more amateur artists (a good thing). But it would be naive to think that artists are going away anytime soon. I'd even argue that there will be new jobs from this (prompt engineer comes to mind). It is really difficult to speculate how this technology (even as it progresses rapidly) will change the future. But considering that no one is creating machines that think like humans we will always have alignment issues. This isn't a fact that can be ignored. It is also worth noting that if you haven't worked with these systems, that what you are seeing is extremely biased. People aren't showing their failures. But there are some twitter accounts that do: WeirdStableAI and weirddalle are probably the most famous. Of course, you can also follow Gary Marcus who is going to retweet every example of failure that he can get his hands on. Other side note: some of these conversations remind me about how people used to talk about 3D printers
- boredtofears 4y ago
  This confirms some of the suspicions I've had after playing with DALL-E for a few months. In the good examples I see from Dall-E, oftentimes the things that it frequently does wrong (face symmetry, hands, feet, limbs) are just missing altogether because they're not needed to produce the prompt. Even in some of the good ones, a lot of elements out of focus seem to get smeared into other objects quite often, or the "smudginess" contributes to its surreal style. People keep saying that sort of problem with be optimized away, but people often underestimate that when a technology plateaus at a certain point it usually requires a completely new innovation to get past some of the hurdles - not just optimization. It can take decades for those innovations to arrive. Of course I don't have enough specialized knowledge to know if that's really the case here but it's an observation I've noticed about hyped technologies in my lifetime.
  - godelski 4y ago
    Well IIRC DALL-E specifically in their public release tries to censor humans. But other works like Stable Diffusion and Midjourney do not have these restrictions. Still, both are quite terrible at human faces. This is always interesting because generative tasks like FFHQ or CelebA have always been considered easy (despite humans being quite adept at recognizing when something is wrong with a face). But being able to create whole scenes requires a substantial amount of diversity. A good network can both memorize and create. There are some technical details that I am certain will be resolved (like long range features) simply due to increased hardware power/memory. But alignment will still be quite difficult as the entire problem is ill-defined to begin with. But to give you some hints of how to use DALL-E better, there are magic keywords and this is why people suggest pretending like you're writing a prompt as if the thing exists. Some of these magic words are: screenshot, unreal engine, photorealistic, studio Ghibli. It does better with anime styles, probably due to training and human interpretations. Try to write longer prompts too. For example "flying otters" will give you otters in the water but "a photorealistic unreal engine render of otters flying through a beautiful sunset" will give you something much closer to what you actually want.
    - boredtofears 4y ago

godelski 4y ago

As a computer vision researcher who has worked with these T2I generators, this is more spot on. I think a lot of people are making a lot of assumptions without having actually worked with these networks. There's also a highly fundamental problem that is not only unsolved, but we don't even have great directions to solve: alignment. Alignment is even a difficult task when you're communicating to another person. Everyone has had an experience of working with someone and they are asking you to do something (or vise versa) and there is a gross misinterpretation. That's alignment. Now try properly aligning with an "entity" that thinks nothing like you, has a vastly different set of experiences, doesn't know how to count, and has no real concept of physical mechanisms (and more). The technology will definitely help us see a lot more unique art and a lot more amateur artists (a good thing). But it would be naive to think that artists are going away anytime soon. I'd even argue that there will be new jobs from this (prompt engineer comes to mind). It is really difficult to speculate how this technology (even as it progresses rapidly) will change the future. But considering that no one is creating machines that think like humans we will always have alignment issues. This isn't a fact that can be ignored. It is also worth noting that if you haven't worked with these systems, that what you are seeing is extremely biased. People aren't showing their failures. But there are some twitter accounts that do: WeirdStableAI and weirddalle are probably the most famous. Of course, you can also follow Gary Marcus who is going to retweet every example of failure that he can get his hands on. Other side note: some of these conversations remind me about how people used to talk about 3D printers

boredtofears 4y ago
This confirms some of the suspicions I've had after playing with DALL-E for a few months. In the good examples I see from Dall-E, oftentimes the things that it frequently does wrong (face symmetry, hands, feet, limbs) are just missing altogether because they're not needed to produce the prompt. Even in some of the good ones, a lot of elements out of focus seem to get smeared into other objects quite often, or the "smudginess" contributes to its surreal style. People keep saying that sort of problem with be optimized away, but people often underestimate that when a technology plateaus at a certain point it usually requires a completely new innovation to get past some of the hurdles - not just optimization. It can take decades for those innovations to arrive. Of course I don't have enough specialized knowledge to know if that's really the case here but it's an observation I've noticed about hyped technologies in my lifetime.
- godelski 4y ago
  Well IIRC DALL-E specifically in their public release tries to censor humans. But other works like Stable Diffusion and Midjourney do not have these restrictions. Still, both are quite terrible at human faces. This is always interesting because generative tasks like FFHQ or CelebA have always been considered easy (despite humans being quite adept at recognizing when something is wrong with a face). But being able to create whole scenes requires a substantial amount of diversity. A good network can both memorize and create. There are some technical details that I am certain will be resolved (like long range features) simply due to increased hardware power/memory. But alignment will still be quite difficult as the entire problem is ill-defined to begin with. But to give you some hints of how to use DALL-E better, there are magic keywords and this is why people suggest pretending like you're writing a prompt as if the thing exists. Some of these magic words are: screenshot, unreal engine, photorealistic, studio Ghibli. It does better with anime styles, probably due to training and human interpretations. Try to write longer prompts too. For example "flying otters" will give you otters in the water but "a photorealistic unreal engine render of otters flying through a beautiful sunset" will give you something much closer to what you actually want.
  - boredtofears 4y ago