6 ms·

I gave it a photo I had Stable Diffusion 1.4 generate from the prompt "avatar for saurik". If you dig through the CLIP database, you will find that the model wa

by saurik 4y ago

I gave it a photo I had Stable Diffusion 1.4 generate from the prompt "avatar for saurik". If you dig through the CLIP database, you will find that the model was trained on a ridiculously large number of copies of my Twitter profile photo due to it being included when people screenshot popular tweets I've posted (which, notably, also means that it is rather low resolution). https://www.stableattribution.com/?image=e89f1e94-067b-4ab8-b1f5-d601bb55825d https://www.stableattribution.com/?image=e89f1e94-067b-4ab8-... https://pbs.twimg.com/profile_images/1434747464/square_400x400.png https://pbs.twimg.com/profile_images/1434747464/square_400x4... Given that I only said "saurik" and SD came up with something that not only looks more than quite a bit like me but is more than quite a bit similar to the pose of my profile photo, I'd say clearly that photo would be one of the most important photographs in the database to show up when asking "which human-made source images were used by AI to generate this image"... ...and yet, whatever algorithm is being used here--which I'm guessing is merely "(full) images similar to this (full) image" as opposed to "images that were used to make this image"--isn't finding it; which, to me, means this website is adding more noise than signal to the discussion (in that I think people might learn the wrong lessons or draw the wrong conclusions).

codetrotter 4y ago
Am I seeing the same thing you are in those two images you linked to? The first one, generated by AI has you looking at the camera. The other one has you looking at an instrument. They don’t look much like each other to me, in terms of pose or anything. The first images suggested by Stable Attribution looks a lot more like the AI image to me, in terms of pose and everything.
- csande17 4y ago
  That's the point. The website is finding images that look similar, not the images the algorithm actually used to generate that picture for the prompt "avatar for saurik".
- Retr0id 4y ago
  I think you're missing the point. How does Stable Diffusion know "what does saurik look like?". The answer is of course that it's seen saurik's profile pic in training data. Stable Attribution is not showing that. As another comment[1] points out: > This appears to be just looking for the nearest neighbors of the image in embedding space and calling those the source data. Stylistically similar images are not the same as source images. [1] https://news.ycombinator.com/item?id=34670483 https://news.ycombinator.com/item?id=34670483
  - killjoywashere 4y ago
    [flagged]
  - ForestCritter 4y ago
    I think you may have a point. I can see familiar references and recognize certain artists in ai art without help, yet when I look at the supposed source photos, for the most part, I dont really see it.
  - orangeoxidation 4y ago
    Interesting. So it only shows images that were transformed / mixed to get the output, but does not show images used to learn how to transform / select them? Sounds very much like a human would do it. If I 'know' how to recognize saurik and I know how anime is supposed to look like, I can check my digital photo library for a picture of saurik and than use that picture as a template to draw an anime version of saurik. If someone later asked me what pictures I used the photo is the only one I'd present. Not the thousands of anime pictures I have seen teaching me what anime looks like, nor the picture my eyes took meeting saurik.
    - space_fountain 4y ago
      I think saying exactly what an ai of sufficient complexity is doing is a mater for philosophy more than science. But idk if transforming or mixing is how I’d describe what this is doing. In particular it truly does not have a complete representation of any of the images it was trained on. They just wouldn’t fit. It does of course have an understanding of how embeddings relate to images that is informed by the images it’s seen so maybe that counts, but I’m not sure if it’s useful in understanding the limitations or how to improve models like it
- saurik 4y ago
  The slight adjustment of where I'm looking is minor. The instrument is uncommon and honestly difficult to get stable diffusion to generate correctly at all; though, from my experience playing with this (I spent a lot of time trying to figure out why it knew who I was before discovering the CLIP database browser), I'm going to argue that the reason that hand is showing up in the position it is is because of my hand holding the violin there. Frankly, we might also as well start nitpicking that the suit-like thing being worn in the generated image is not a tuxedo and the generated person looks a tad bit rounded and has more hair ;P. However, the key thing here is: do you really think "avatar for saurik" is going to come up with something out of the blue--out of all the numerous random images of people that exist and who have avatar/profile photos... people wearing lots of different clothing and in different orientations--where we can even be talking about such silly things as props and gaze direction? I will assert that would be ridiculous. Clearly seeing at least one photo of me (and AFAIK it was only trained on thousands of copies of that single photo) was absolutely crucial to the construction of this image, and yet this website isn't finding any such to show as it isn't really doing what it is claiming to do (and it isn't clear to me how it could without the original prompt). From there, SD is upscaling from memory a lot and filling in a ton of details (as almost all of the copies of my photo it was trained on are very small, embedded in screenshots of tweets), but the cornerstone of that construction is clearly my Twitter profile photo... and then, once you do that, you can go back and attempt to rationalize "these were the photos that were most similar to the photo that we generated, and so I guess those were what the algorithm used to generate this image", but--at least in this case--it is pretty obvious that that isn't how this worked as there's no way you start with "avatar of saurik" and arbitrarily pluck an image that is this similar to the one photo of saurik you have.
  - throwaway1851 4y ago
    > Clearly seeing at least one photo of me (and AFAIK it was only trained on thousands of copies of that single photo) was absolutely crucial to the construction of this image, and yet this website isn't finding any I think there are two separate ideas that are being conflated here. The first idea is that there is a mapping between text input and a joint text/image embedding space. For that mapping, yes, your profile photo (paired with some caption text that says saurik) would have to be the most important training input because, as you say, how else does SD know what “saurik” looks like? But the second idea is the mapping from the shared latent space to an image output. It is NOT necessarily true that your profile photo is the most significant training example for the generated image. That’s because the text “saurik” can map to a neighborhood in the latent space that has all the photos this tool retrieved — photos which are indeed very similar to the generated image. As an example, let’s say that I resemble Ben Affleck. And you type in “throwaway1851” and you get back an image that does look like me, but also looks an awful lot like Ben Affleck. In this case I really wouldn’t be surprised if my one profile photo didn’t contribute much to the generated image as compared to a dozen different photos of Ben Affleck. Perhaps the original mapping just pointed to Affleck anyway, because I wasn’t significant enough to end up taking space in the model’s parameters.
heliumcraft 4y ago
relevant: https://rom1504.github.io/clip-retrieval/ https://rom1504.github.io/clip-retrieval/ https://github.com/rom1504/clip-retrieval https://github.com/rom1504/clip-retrieval Ironically, the stableattribution.com authors didn't give any attribution or credit to clip-retrieval and its developer
- rcme 4y ago
  They don’t need to, as clip-retrieval is licensed under the MIT license.
  - deleted 4y ago
    [deleted]
  - HDevo 4y ago
    The MIT license disagrees: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. Besides, even if you don't interpret this as "attribution", it's still ironic coming from this particular project.
    - janekm 4y ago
      It is ironic, but MIT license definitely does not require attribution from websites using the code on the backend (otherwise you'd see a rather large attribution list at the bottom of Google.com or any website...)
Thorentis 4y ago
Are you the guy who made Cydia for jailbroken iPhones all those years ago? Had no idea you were on HN!
- sogen 4y ago
  Yes, the one and only Saurik!
louison11 4y ago
This. They probably simply used an “image similarity” algorithm, of which many are readily available and do not require much computing power. The credit at the bottom of the website for “Chroma”, a startup with no live product and hiring, suggests this could be a growth hack. Like “let’s ride the buzz of the moment with a barely useful tool quick to develop, hit front page of HN, and get some visibility.” Good for them, it worked, but yeah the product is probably bogus as you’ve demonstrated.
- bemmu 4y ago
  This will spread further than HN, as this is a current hot topic, and journalists are dying for more things to write about it. Whether or not this tool works, expect it to make the rounds on any tech publications that have previously written “stable diffusion bad” -type stories.
- chaostheory 4y ago
  It’s not for nothing. Like Twitter, the idea is so obvious, but only when someone introduces it to the masses. This just needs to be done at the source. Ie whatever AI service is generating the image should generate attribution as part of the result I was surprised that they even had a prototype. I thought this was just a really good CTA
junon 4y ago
This was my suspicion. It just seems like a glorified image similarity search instead of a real reversal of the stable diffusion process.
irthomasthomas 4y ago
Perhaps it's confused because the real you looks exactly like Leonidas Kavakos! http://thelistenersclub.timothyjuddviolin.com/wp-content/uploads/sites/2/2016/05/2017_Leonidas_Kavakos_annoncering_wide.jpg http://thelistenersclub.timothyjuddviolin.com/wp-content/upl...

by saurik 4y ago

codetrotter 4y ago

Am I seeing the same thing you are in those two images you linked to? The first one, generated by AI has you looking at the camera. The other one has you looking at an instrument. They don’t look much like each other to me, in terms of pose or anything. The first images suggested by Stable Attribution looks a lot more like the AI image to me, in terms of pose and everything.

csande17 4y ago
That's the point. The website is finding images that look similar, not the images the algorithm actually used to generate that picture for the prompt "avatar for saurik".
Retr0id 4y ago
I think you're missing the point. How does Stable Diffusion know "what does saurik look like?". The answer is of course that it's seen saurik's profile pic in training data. Stable Attribution is not showing that. As another comment[1] points out: > This appears to be just looking for the nearest neighbors of the image in embedding space and calling those the source data. Stylistically similar images are not the same as source images. [1] https://news.ycombinator.com/item?id=34670483 https://news.ycombinator.com/item?id=34670483
- killjoywashere 4y ago
  [flagged]
- ForestCritter 4y ago
  I think you may have a point. I can see familiar references and recognize certain artists in ai art without help, yet when I look at the supposed source photos, for the most part, I dont really see it.
- orangeoxidation 4y ago
  Interesting. So it only shows images that were transformed / mixed to get the output, but does not show images used to learn how to transform / select them? Sounds very much like a human would do it. If I 'know' how to recognize saurik and I know how anime is supposed to look like, I can check my digital photo library for a picture of saurik and than use that picture as a template to draw an anime version of saurik. If someone later asked me what pictures I used the photo is the only one I'd present. Not the thousands of anime pictures I have seen teaching me what anime looks like, nor the picture my eyes took meeting saurik.
  - space_fountain 4y ago
    I think saying exactly what an ai of sufficient complexity is doing is a mater for philosophy more than science. But idk if transforming or mixing is how I’d describe what this is doing. In particular it truly does not have a complete representation of any of the images it was trained on. They just wouldn’t fit. It does of course have an understanding of how embeddings relate to images that is informed by the images it’s seen so maybe that counts, but I’m not sure if it’s useful in understanding the limitations or how to improve models like it
saurik 4y ago
The slight adjustment of where I'm looking is minor. The instrument is uncommon and honestly difficult to get stable diffusion to generate correctly at all; though, from my experience playing with this (I spent a lot of time trying to figure out why it knew who I was before discovering the CLIP database browser), I'm going to argue that the reason that hand is showing up in the position it is is because of my hand holding the violin there. Frankly, we might also as well start nitpicking that the suit-like thing being worn in the generated image is not a tuxedo and the generated person looks a tad bit rounded and has more hair ;P. However, the key thing here is: do you really think "avatar for saurik" is going to come up with something out of the blue--out of all the numerous random images of people that exist and who have avatar/profile photos... people wearing lots of different clothing and in different orientations--where we can even be talking about such silly things as props and gaze direction? I will assert that would be ridiculous. Clearly seeing at least one photo of me (and AFAIK it was only trained on thousands of copies of that single photo) was absolutely crucial to the construction of this image, and yet this website isn't finding any such to show as it isn't really doing what it is claiming to do (and it isn't clear to me how it could without the original prompt). From there, SD is upscaling from memory a lot and filling in a ton of details (as almost all of the copies of my photo it was trained on are very small, embedded in screenshots of tweets), but the cornerstone of that construction is clearly my Twitter profile photo... and then, once you do that, you can go back and attempt to rationalize "these were the photos that were most similar to the photo that we generated, and so I guess those were what the algorithm used to generate this image", but--at least in this case--it is pretty obvious that that isn't how this worked as there's no way you start with "avatar of saurik" and arbitrarily pluck an image that is this similar to the one photo of saurik you have.
- throwaway1851 4y ago
  > Clearly seeing at least one photo of me (and AFAIK it was only trained on thousands of copies of that single photo) was absolutely crucial to the construction of this image, and yet this website isn't finding any I think there are two separate ideas that are being conflated here. The first idea is that there is a mapping between text input and a joint text/image embedding space. For that mapping, yes, your profile photo (paired with some caption text that says saurik) would have to be the most important training input because, as you say, how else does SD know what “saurik” looks like? But the second idea is the mapping from the shared latent space to an image output. It is NOT necessarily true that your profile photo is the most significant training example for the generated image. That’s because the text “saurik” can map to a neighborhood in the latent space that has all the photos this tool retrieved — photos which are indeed very similar to the generated image. As an example, let’s say that I resemble Ben Affleck. And you type in “throwaway1851” and you get back an image that does look like me, but also looks an awful lot like Ben Affleck. In this case I really wouldn’t be surprised if my one profile photo didn’t contribute much to the generated image as compared to a dozen different photos of Ben Affleck. Perhaps the original mapping just pointed to Affleck anyway, because I wasn’t significant enough to end up taking space in the model’s parameters.

heliumcraft 4y ago

relevant: https://rom1504.github.io/clip-retrieval/ https://rom1504.github.io/clip-retrieval/ https://github.com/rom1504/clip-retrieval https://github.com/rom1504/clip-retrieval Ironically, the stableattribution.com authors didn't give any attribution or credit to clip-retrieval and its developer

rcme 4y ago
They don’t need to, as clip-retrieval is licensed under the MIT license.
- deleted 4y ago
  [deleted]
- HDevo 4y ago
  The MIT license disagrees: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. Besides, even if you don't interpret this as "attribution", it's still ironic coming from this particular project.
  - janekm 4y ago
    It is ironic, but MIT license definitely does not require attribution from websites using the code on the backend (otherwise you'd see a rather large attribution list at the bottom of Google.com or any website...)

Thorentis 4y ago

Are you the guy who made Cydia for jailbroken iPhones all those years ago? Had no idea you were on HN!

sogen 4y ago
Yes, the one and only Saurik!

louison11 4y ago

This. They probably simply used an “image similarity” algorithm, of which many are readily available and do not require much computing power. The credit at the bottom of the website for “Chroma”, a startup with no live product and hiring, suggests this could be a growth hack. Like “let’s ride the buzz of the moment with a barely useful tool quick to develop, hit front page of HN, and get some visibility.” Good for them, it worked, but yeah the product is probably bogus as you’ve demonstrated.

bemmu 4y ago
This will spread further than HN, as this is a current hot topic, and journalists are dying for more things to write about it. Whether or not this tool works, expect it to make the rounds on any tech publications that have previously written “stable diffusion bad” -type stories.
chaostheory 4y ago
It’s not for nothing. Like Twitter, the idea is so obvious, but only when someone introduces it to the masses. This just needs to be done at the source. Ie whatever AI service is generating the image should generate attribution as part of the result I was surprised that they even had a prototype. I thought this was just a really good CTA

junon 4y ago

This was my suspicion. It just seems like a glorified image similarity search instead of a real reversal of the stable diffusion process.

irthomasthomas 4y ago

Perhaps it's confused because the real you looks exactly like Leonidas Kavakos! http://thelistenersclub.timothyjuddviolin.com/wp-content/uploads/sites/2/2016/05/2017_Leonidas_Kavakos_annoncering_wide.jpg http://thelistenersclub.timothyjuddviolin.com/wp-content/upl...