6 ms·

Nyx.gallery – AI-generated photography

56 points by MasterScrat 4y ago

MasterScrat 4y ago
Authors here! Stable Diffusion and DALL·E have made it simpler to create new images. But getting photorealistic results is still a challenge… This is a continuation of our work from 3 months ago on “This Food Does Not Exist” (https://news.ycombinator.com/item?id=32167704 https://news.ycombinator.com/item?id=32167704). We are now using both Stable Diffusion and GANs depending on the subjects we want to render.
- MasterScrat 4y ago
  Models for the 256px food images were previously released here: https://github.com/nyx-ai/stylegan2-flax-tpu https://github.com/nyx-ai/stylegan2-flax-tpu
- lm28469 4y ago
  > But getting photorealistic results is still a challenge… Understatement of the year, I wonder if/when we'll get there
- LegitShady 4y ago
  these are AI generated 'photorealistic' images, not 'ai generated photography'. Photography is a particular process you are not involved in during the course of these AI activities, as far as I can tell. If you somehow have Machine Learning algorithms operating cameras it would be correct.
  - thih9 4y ago
    You’re right; then again, my money’s on “ai photo” becoming common enough to acquire meaning on its own. We already have a glow-worm (not a worm), funny bone (not a bone), Baby Yoda (not the Yoda), and many more.
    - LegitShady 4y ago
      that other people are incorrect is no reason to volunteer to be incorrect too. In fact I'd say its the opposite of good to use the wrong terms when you know better. I suppose 'AI Generated Photography' sounds better than 'yet another stable diffusion model' but one is correct and the other isn't. This has nothing to do with photography.
      - DemocracyFTW2 4y ago
        "ai fauxto" (or "phauxto"?)
        thih9 4y ago
        With lowercase “ai fauxto” it’s almost like iPhoto [1]. The name is now available I guess… [1]: https://en.m.wikipedia.org/wiki/IPhoto https://en.m.wikipedia.org/wiki/IPhoto
  - CharlesW 4y ago
    > these are AI generated 'photorealistic' images, not 'ai generated photography'. Amen, brother. Also, "fauxtography" was RIGHT THERE.
    - pessimizer 4y ago
      I will repeat the word "fauxtography" whenever I have an opportunity.
    - sam1r 4y ago
      Wow. Mind blown faux-sho.
  - macrolime 4y ago
    These are photos of Latent Space taken with a Autoencoder Decoder camera :p
- itronitron 4y ago
  I was expecting to also see some photographs of traffic lights.
O__________O 4y ago
>> From FAQs: “we are using both diffusion models and GANs in combination with an extensive filtering and quality assessment pipeline that allows us to generate photorealistic images at scale.” For every image that reaches the site, how many were generated that were filtered out by the pipeline? For example, for every photo that reaches site, 1000 were generated but did not pass the quality assessment pipeline.
- MasterScrat 4y ago
  It really depends on the subjects. For example cookies are quite forgiving - up to 30% of results are good. For things like coral or sushi we keep less than 1 in 10 (which is already much better than where we were at a few months ago!) For now we still keep a close manual look at the output, but the goal as we scale up is to fully automate this selection process. Right now we have a pipeline that ranks the outputs and we select from the top results.
MontyCarloHall 4y ago
I see an uncharacteristically high number of malformed images. For example, Six legged gecko: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_9f0475078b391da2a381ee97a6b9e400/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm... Dog with warped face: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_f3feac216f6dfe61395c674a8c3010dd/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm... Bizarrely proportioned lion: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_b8a9c03e8492a8744f6a98b7a8524946/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm... Rabbit with fur and whisker artifacts, misplaced hind leg, and weird front paw: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_b1cadd303c6547f472878c5b6199efd9/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm...
- lurquer 4y ago
  Perhaps the ‘natural’ gecko is malformed, and the AI mind is showing us what they ought to look like.
- deleted 4y ago
  [deleted]
- ok_dad 4y ago
  I liked a few weirdo pictures: - Weird-grown broccolli (it should be more tree-like, not have stems going every which way): https://nyx.gallery/#c590eb57ef4e33ef93c147b1b705feee https://nyx.gallery/#c590eb57ef4e33ef93c147b1b705feee - I don't know what kind of fish this weird-ass sushi is, but I wouldn't eat it: https://nyx.gallery/#8a77524d034b2733a34ae4aabe8a97f0 https://nyx.gallery/#8a77524d034b2733a34ae4aabe8a97f0 - A very unstable glass: https://nyx.gallery/#2dd470a9f3ab82f04c4a4391bb06df53 https://nyx.gallery/#2dd470a9f3ab82f04c4a4391bb06df53 - If I ordered a "hot dog with catsup and mustard" and got this, with jalapenos, I would return it to the kitchen: https://nyx.gallery/#ab5a523bcc5da8689ad945af775f662b https://nyx.gallery/#ab5a523bcc5da8689ad945af775f662b I make fun, but this type of tech is pretty amazing to me.
aimor 4y ago
I find Stable Diffusion is pretty good at generating single-subject images like this. It's really mind blowing and the novelty hasn't worn off for me yet. But after dozens of attempts I still haven't managed to get it to show me a photograph of a duck eating a hoagie at Niagara Falls. I think it would be really interesting to try to find the simplest query that these tools cannot produce.
- iamgopal 4y ago
  Because of too much use of proper noun ?
  - aimor 4y ago
    I had considered this, I'm not sure it's the issue since Niagara Falls is pretty iconic. I'll give it a shot with surrogates in place of Niagara Falls and see what happens. The bigger difficulty it seems is getting a duck to eat a hoagie. I can get a hoagie at Niagara Falls, I can get a duck at Niagara Falls, I can even get a hoagie and a duck together near some water, I can almost get a duck eating a hoagie (I've gotten the duck near the hoagie with its mouth open), but I can't get a duck eating a hoagie at Niagara Falls. Update: Stable Diffusion 1.5 is getting there. I have a duck, hoagie, and Niagara Falls all together, and I think this picture might be a success: https://i.ibb.co/Mhrv0sD/1894667935-A-duck-eating-a-hoagie-at-Niagara-Falls.png https://i.ibb.co/Mhrv0sD/1894667935-A-duck-eating-a-hoagie-a...
abhorrence 4y ago
It's interesting to see what types of features these models don't distinguish well. For example, I've noticed that a lot of models have trouble with giving lady bugs distinct spots. Instead they usually end up with a big black splotch.
yalogin 4y ago
So at this point is the stock photo companies more or less dead or do they still have business?
- cercatrova 4y ago
  The stock photo companies simply pivot to AI, see companies like StockAI popping up [0]. Getty should also get in this business but it seems they're going in the opposite direction, banning AI generated images. [0] https://www.stockai.com/ https://www.stockai.com/
celeritascelery 4y ago
One thing I think would be super fascinating to have is a system that can take AI image training sets and reverse engineer which pictures were used to make the output. Take the bunny. I bet there were a lot of bunny pictures in the training set that looked very similar to the generated one. It would be interesting to have a system pick the one that is closest and display it next to it. It would be show how original (or unoriginal) these images are.
ridgeguy 4y ago
As a meatspace photographer, I take some comfort that the photograph in column 3, row 25 has an issue. The mountain peak has abundant snow. Its reflection in the lake doesn't. There are similar snow reflection disparities in several of the mountains-reflected-in-water pix.
- jerrysievert 4y ago
  reflections seem to be something that is struggled with. there are more examples where the reflection is not quite right: /\_ / \ -------- \ / \/
  - mattigames 4y ago
    current AI frequently fail with any hard expectation of structure, like when it tries to genarate human bodies and it doesn't grasp at all how bones work, bending limba in unreal ways or adding extra ones, or creating heads with wrong proportions/spacing. It is pretty good at grasping how to fade textures/colors but it has little grasp on structure and "the big picture".

56 points by MasterScrat 4y ago

Authors here! Stable Diffusion and DALL·E have made it simpler to create new images. But getting photorealistic results is still a challenge… This is a continuation of our work from 3 months ago on “This Food Does Not Exist” (https://news.ycombinator.com/item?id=32167704 https://news.ycombinator.com/item?id=32167704). We are now using both Stable Diffusion and GANs depending on the subjects we want to render.

MasterScrat 4y ago
Models for the 256px food images were previously released here: https://github.com/nyx-ai/stylegan2-flax-tpu https://github.com/nyx-ai/stylegan2-flax-tpu
lm28469 4y ago
> But getting photorealistic results is still a challenge… Understatement of the year, I wonder if/when we'll get there
LegitShady 4y ago
these are AI generated 'photorealistic' images, not 'ai generated photography'. Photography is a particular process you are not involved in during the course of these AI activities, as far as I can tell. If you somehow have Machine Learning algorithms operating cameras it would be correct.
- thih9 4y ago
  You’re right; then again, my money’s on “ai photo” becoming common enough to acquire meaning on its own. We already have a glow-worm (not a worm), funny bone (not a bone), Baby Yoda (not the Yoda), and many more.
  - LegitShady 4y ago
    that other people are incorrect is no reason to volunteer to be incorrect too. In fact I'd say its the opposite of good to use the wrong terms when you know better. I suppose 'AI Generated Photography' sounds better than 'yet another stable diffusion model' but one is correct and the other isn't. This has nothing to do with photography.
    - DemocracyFTW2 4y ago
      "ai fauxto" (or "phauxto"?)
      - thih9 4y ago
        With lowercase “ai fauxto” it’s almost like iPhoto [1]. The name is now available I guess… [1]: https://en.m.wikipedia.org/wiki/IPhoto https://en.m.wikipedia.org/wiki/IPhoto
- CharlesW 4y ago
  > these are AI generated 'photorealistic' images, not 'ai generated photography'. Amen, brother. Also, "fauxtography" was RIGHT THERE.
  - pessimizer 4y ago
    I will repeat the word "fauxtography" whenever I have an opportunity.
  - sam1r 4y ago
    Wow. Mind blown faux-sho.
- macrolime 4y ago
  These are photos of Latent Space taken with a Autoencoder Decoder camera :p
itronitron 4y ago
I was expecting to also see some photographs of traffic lights.

O__________O 4y ago

>> From FAQs: “we are using both diffusion models and GANs in combination with an extensive filtering and quality assessment pipeline that allows us to generate photorealistic images at scale.” For every image that reaches the site, how many were generated that were filtered out by the pipeline? For example, for every photo that reaches site, 1000 were generated but did not pass the quality assessment pipeline.

MasterScrat 4y ago
It really depends on the subjects. For example cookies are quite forgiving - up to 30% of results are good. For things like coral or sushi we keep less than 1 in 10 (which is already much better than where we were at a few months ago!) For now we still keep a close manual look at the output, but the goal as we scale up is to fully automate this selection process. Right now we have a pipeline that ranks the outputs and we select from the top results.

MontyCarloHall 4y ago

I see an uncharacteristically high number of malformed images. For example, Six legged gecko: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_9f0475078b391da2a381ee97a6b9e400/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm... Dog with warped face: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_f3feac216f6dfe61395c674a8c3010dd/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm... Bizarrely proportioned lion: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_b8a9c03e8492a8744f6a98b7a8524946/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm... Rabbit with fur and whisker artifacts, misplaced hind leg, and weird front paw: https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnmqHMQ/v0_b1cadd303c6547f472878c5b6199efd9/public https://nyx.gallery/cdn-cgi/imagedelivery/16bz3hOZjq1MqWAKnm...

lurquer 4y ago
Perhaps the ‘natural’ gecko is malformed, and the AI mind is showing us what they ought to look like.
deleted 4y ago
[deleted]
ok_dad 4y ago
I liked a few weirdo pictures: - Weird-grown broccolli (it should be more tree-like, not have stems going every which way): https://nyx.gallery/#c590eb57ef4e33ef93c147b1b705feee https://nyx.gallery/#c590eb57ef4e33ef93c147b1b705feee - I don't know what kind of fish this weird-ass sushi is, but I wouldn't eat it: https://nyx.gallery/#8a77524d034b2733a34ae4aabe8a97f0 https://nyx.gallery/#8a77524d034b2733a34ae4aabe8a97f0 - A very unstable glass: https://nyx.gallery/#2dd470a9f3ab82f04c4a4391bb06df53 https://nyx.gallery/#2dd470a9f3ab82f04c4a4391bb06df53 - If I ordered a "hot dog with catsup and mustard" and got this, with jalapenos, I would return it to the kitchen: https://nyx.gallery/#ab5a523bcc5da8689ad945af775f662b https://nyx.gallery/#ab5a523bcc5da8689ad945af775f662b I make fun, but this type of tech is pretty amazing to me.

aimor 4y ago

I find Stable Diffusion is pretty good at generating single-subject images like this. It's really mind blowing and the novelty hasn't worn off for me yet. But after dozens of attempts I still haven't managed to get it to show me a photograph of a duck eating a hoagie at Niagara Falls. I think it would be really interesting to try to find the simplest query that these tools cannot produce.

iamgopal 4y ago
Because of too much use of proper noun ?
- aimor 4y ago
  I had considered this, I'm not sure it's the issue since Niagara Falls is pretty iconic. I'll give it a shot with surrogates in place of Niagara Falls and see what happens. The bigger difficulty it seems is getting a duck to eat a hoagie. I can get a hoagie at Niagara Falls, I can get a duck at Niagara Falls, I can even get a hoagie and a duck together near some water, I can almost get a duck eating a hoagie (I've gotten the duck near the hoagie with its mouth open), but I can't get a duck eating a hoagie at Niagara Falls. Update: Stable Diffusion 1.5 is getting there. I have a duck, hoagie, and Niagara Falls all together, and I think this picture might be a success: https://i.ibb.co/Mhrv0sD/1894667935-A-duck-eating-a-hoagie-at-Niagara-Falls.png https://i.ibb.co/Mhrv0sD/1894667935-A-duck-eating-a-hoagie-a...

abhorrence 4y ago

It's interesting to see what types of features these models don't distinguish well. For example, I've noticed that a lot of models have trouble with giving lady bugs distinct spots. Instead they usually end up with a big black splotch.

yalogin 4y ago

So at this point is the stock photo companies more or less dead or do they still have business?

cercatrova 4y ago
The stock photo companies simply pivot to AI, see companies like StockAI popping up [0]. Getty should also get in this business but it seems they're going in the opposite direction, banning AI generated images. [0] https://www.stockai.com/ https://www.stockai.com/

celeritascelery 4y ago

One thing I think would be super fascinating to have is a system that can take AI image training sets and reverse engineer which pictures were used to make the output. Take the bunny. I bet there were a lot of bunny pictures in the training set that looked very similar to the generated one. It would be interesting to have a system pick the one that is closest and display it next to it. It would be show how original (or unoriginal) these images are.

ridgeguy 4y ago

As a meatspace photographer, I take some comfort that the photograph in column 3, row 25 has an issue. The mountain peak has abundant snow. Its reflection in the lake doesn't. There are similar snow reflection disparities in several of the mountains-reflected-in-water pix.

jerrysievert 4y ago
reflections seem to be something that is struggled with. there are more examples where the reflection is not quite right: /\_ / \ -------- \ / \/
- mattigames 4y ago
  current AI frequently fail with any hard expectation of structure, like when it tries to genarate human bodies and it doesn't grasp at all how bones work, bending limba in unreal ways or adding extra ones, or creating heads with wrong proportions/spacing. It is pretty good at grasping how to fade textures/colors but it has little grasp on structure and "the big picture".