3 ms·

"it just blends many images of arms together and create things like 3 arms. Also why there are so many eyes, arms, legs and other things in other generative pro

by nullc 4y ago

"it just blends many images of arms together and create things like 3 arms. Also why there are so many eyes, arms, legs and other things in other generative programs. It just spits out the training set in random configurations" That is thoroughly confused to the point of uselessness. The reason you get structural issues is because it's hard for the architecture to express large scale structure, but they get better and better at it simply by scaling up the network.

ffwd 4y ago
It was poorly communicated in my post but what I was referring to there were earlier programs like in 2015 and not SD and newer ones. If you put in an image of a landscape it could fill out the landscape with eyes and elbows all over the generated image because it had no information or context for what an eye was or where it should go. But now you get SD, dalle and others which add more information not just by scaling, but also by mapping sentences/words to pre-existing images that already have cohesion. That way when you write in sentences to the text prompt, the model has more semantic information about what an eye is, but (IMO) only _indirectly_ because it will map a sentence to images that match that sentence. The question is always what information is actually contained in the training set and what is missing from it and when it creates an image where is the information from etc. In some ways, that means I think that meaning to us as humans, is different from scaling which is almost like pixel resolution except resolution of patterns and differentiation of patterns. Meaning in this sense is things like creating a doorway with no actual door, but still the doorway itself looks super realistic is rendered. You can fix it by scaling and increasing the differentiation of patterns I guess, but you can never fix all instances completely with scaling. That's why in some ways I think meaning is sort of orthogonal to scale, however on a philosophical level, they should converge but that's for another topic. I may have missed something in my thoughts here because this is sort of difficult to talk about without writing a book eventually.

by nullc 4y ago

ffwd 4y ago

It was poorly communicated in my post but what I was referring to there were earlier programs like in 2015 and not SD and newer ones. If you put in an image of a landscape it could fill out the landscape with eyes and elbows all over the generated image because it had no information or context for what an eye was or where it should go. But now you get SD, dalle and others which add more information not just by scaling, but also by mapping sentences/words to pre-existing images that already have cohesion. That way when you write in sentences to the text prompt, the model has more semantic information about what an eye is, but (IMO) only _indirectly_ because it will map a sentence to images that match that sentence. The question is always what information is actually contained in the training set and what is missing from it and when it creates an image where is the information from etc. In some ways, that means I think that meaning to us as humans, is different from scaling which is almost like pixel resolution except resolution of patterns and differentiation of patterns. Meaning in this sense is things like creating a doorway with no actual door, but still the doorway itself looks super realistic is rendered. You can fix it by scaling and increasing the differentiation of patterns I guess, but you can never fix all instances completely with scaling. That's why in some ways I think meaning is sort of orthogonal to scale, however on a philosophical level, they should converge but that's for another topic. I may have missed something in my thoughts here because this is sort of difficult to talk about without writing a book eventually.