5 ms·

To be fair, to me, title should be more like are you able to tell cherry picked AI images from cherry picked human created images, since both clearly were selec

To be fair, to me, title should be more like are you able to tell cherry picked AI images from cherry picked human created images, since both clearly were selected to appear similarly picked to confuse a human in my opinion. No idea if that’s accurate, or even if it is, how to fix it, but one idea would be to get a list of the top ten search queries on a popular stock image site, then randomly pick an AI image and human image; problem being that in my experience it would become painful obvious on average which was real and which was AI, which to me points out that AI images as they’re currently “generated” heavily depend on humans to create prompts and curate which are shared. How did you filter, select, etc the images?

ronsor 4y ago
Actually no, they were not cherry picked. Except for generations which did not match the prompt at all, I took every image that was created. For the human images, I just took things from Unsplash's and ArtStation's front pages. I clicked on those images without looking. Early next year I'll do a better quiz with more variety. This was a weekend project.
- O__________O 4y ago
  >> Except for generations which did not match the prompt at all, I took every image that was created. While I believe I understand what you mean, to me, this is not what AI generated art means, hence why I suggested limiting prompts to existing top queries to a top stock image site and the using that as the prompt. Also, I have to say that I very surprised none of the AI images were cherry picked, since I personally have run 100s if not 1000s of “high quality” prompts, and in my experience volume of misfits is normal pretty high; for sure read and reviewed 1000s of prompts and their outputs which were shared and in my experience even those have artificial artifacts and are likely cherry picked from the ones that were much worse. Another alternative might be to get the top best selling stock images, run them through CLIP, then use those as the prompts, but then you’re evaluating CLIP + plus whatever AI image generation system you’re using. Another huge give away is the resolution. Obviously you’re able to get ultra super resolution human generated images, but to my knowledge that’s not possible yet for AI images. Which AI generation tool were you using and what were the prompts/settings?
  - ronsor 4y ago
    Two models were used for image generation: the not-yet-released AniPlus v2 and vanilla Stable Diffusion 1.5 AniPlus v2 generally creates good if not great images consistently. All images were generated in AUTOMATIC1111's web UI, with the DPM++ 2S A sampler @ 15 steps. I'll soon release the metadata for the AI images.
- matkoniecz 4y ago
  > Actually no, they were not cherry picked. Except for generations which did not match the prompt at all Second sentence describes process of selection. Is there difference in English between "filtered" / "selected" and "cherry picked"? Is "cherry picked" implying extreme filtering?
  - dec0dedab0de 4y ago
    Filtered is the process of removing parts from the whole. Like how a water filter removes unwanted chemicals. Selected means it was chosen. So after filtering you could make a selection from all or part of the remaining choices. Cherry picked, means selected individually to meet a goal. Selecting very bad results to be removed is almost the opposite of cherry picking. Cherry picking would be picking only the best images and presenting them, here they removed the worst images and showed the rest.
    - Xelynega 4y ago
      > picking only the best images and presenting them > they removed the worst images and showed the rest Is removing rejects not a way to pick only the best?
      - O__________O 4y ago
        To me, there’s no right answer, hence why I listed set of examples and ended list with etc. Based on my own experience, selecting images that fit the prompt and images that just happened to be generally speaking a fit for the creator are different modes of producing outputs to share. That said, as it relates to the topic of this HN post, understanding the systems, assumption, workflow, etc used to render the survey, its content, and results to me are vital to understanding what actually going on.
      - dec0dedab0de 4y ago
        Is removing rejects not a way to pick only the best? It is not, unless your rejection criteria is "everything that is not the best." However, if your rejection criteria is "not meeting basic requirements" then you are left with the best, and the second best and the middle, all the way down to the bottom of adequate.
constantlm 4y ago
In reality, the use of AI images would be cherry picked, so this is a good test.
- O__________O 4y ago
  Good test would document its hypothesis, procedures, results, etc - and let the reader make their own decisions on its merits and their experience. 100% agree cherry picking is part of the process of generating AI content, though there’s no standard for evaluating the effectiveness of a system, creator, or audience at cherry picking good from bad generations. By pushing as much of the cherry picking to the audience, in my opinion it’s currently the best way to reduce assumptions and get better sense of what current state of the art is. As I mentioned before though, not sure what the answer is, just know it’s an issue, especially in this context in my opinion.

ronsor 4y ago

Actually no, they were not cherry picked. Except for generations which did not match the prompt at all, I took every image that was created. For the human images, I just took things from Unsplash's and ArtStation's front pages. I clicked on those images without looking. Early next year I'll do a better quiz with more variety. This was a weekend project.

O__________O 4y ago
>> Except for generations which did not match the prompt at all, I took every image that was created. While I believe I understand what you mean, to me, this is not what AI generated art means, hence why I suggested limiting prompts to existing top queries to a top stock image site and the using that as the prompt. Also, I have to say that I very surprised none of the AI images were cherry picked, since I personally have run 100s if not 1000s of “high quality” prompts, and in my experience volume of misfits is normal pretty high; for sure read and reviewed 1000s of prompts and their outputs which were shared and in my experience even those have artificial artifacts and are likely cherry picked from the ones that were much worse. Another alternative might be to get the top best selling stock images, run them through CLIP, then use those as the prompts, but then you’re evaluating CLIP + plus whatever AI image generation system you’re using. Another huge give away is the resolution. Obviously you’re able to get ultra super resolution human generated images, but to my knowledge that’s not possible yet for AI images. Which AI generation tool were you using and what were the prompts/settings?
- ronsor 4y ago
  Two models were used for image generation: the not-yet-released AniPlus v2 and vanilla Stable Diffusion 1.5 AniPlus v2 generally creates good if not great images consistently. All images were generated in AUTOMATIC1111's web UI, with the DPM++ 2S A sampler @ 15 steps. I'll soon release the metadata for the AI images.
matkoniecz 4y ago
> Actually no, they were not cherry picked. Except for generations which did not match the prompt at all Second sentence describes process of selection. Is there difference in English between "filtered" / "selected" and "cherry picked"? Is "cherry picked" implying extreme filtering?
- dec0dedab0de 4y ago
  Filtered is the process of removing parts from the whole. Like how a water filter removes unwanted chemicals. Selected means it was chosen. So after filtering you could make a selection from all or part of the remaining choices. Cherry picked, means selected individually to meet a goal. Selecting very bad results to be removed is almost the opposite of cherry picking. Cherry picking would be picking only the best images and presenting them, here they removed the worst images and showed the rest.
  - Xelynega 4y ago
    > picking only the best images and presenting them > they removed the worst images and showed the rest Is removing rejects not a way to pick only the best?
    - O__________O 4y ago
      To me, there’s no right answer, hence why I listed set of examples and ended list with etc. Based on my own experience, selecting images that fit the prompt and images that just happened to be generally speaking a fit for the creator are different modes of producing outputs to share. That said, as it relates to the topic of this HN post, understanding the systems, assumption, workflow, etc used to render the survey, its content, and results to me are vital to understanding what actually going on.
    - dec0dedab0de 4y ago
      Is removing rejects not a way to pick only the best? It is not, unless your rejection criteria is "everything that is not the best." However, if your rejection criteria is "not meeting basic requirements" then you are left with the best, and the second best and the middle, all the way down to the bottom of adequate.

constantlm 4y ago

In reality, the use of AI images would be cherry picked, so this is a good test.

O__________O 4y ago
Good test would document its hypothesis, procedures, results, etc - and let the reader make their own decisions on its merits and their experience. 100% agree cherry picking is part of the process of generating AI content, though there’s no standard for evaluating the effectiveness of a system, creator, or audience at cherry picking good from bad generations. By pushing as much of the cherry picking to the audience, in my opinion it’s currently the best way to reduce assumptions and get better sense of what current state of the art is. As I mentioned before though, not sure what the answer is, just know it’s an issue, especially in this context in my opinion.