3 ms·

It is not possible to manually label hundreds of millions of images to train a model on them, CFG exists to deal with this problem, also Unstable Diffusion will

by GaggiX 4y ago

It is not possible to manually label hundreds of millions of images to train a model on them, CFG exists to deal with this problem, also Unstable Diffusion will just finetune a Stable Diffusion model, so you cannot simply change the licence to what you want.

operator-name 4y ago
Boorus [0] contain millions of images, manually labeled to a pretty high quality. Notably defusion models trained on booru datasets have had good success. This is not the only example of well curated image-tag pairs, especially in artistic circles. It's just that most of them are not CC. [0]: https://en.wiktionary.org/wiki/booru https://en.wiktionary.org/wiki/booru
- GaggiX 4y ago
  Booru use tags instead of captions, so a model trained on them is really limited; moreover, Danbooru has only 5 million images, while other booru such as gelbooru and sankaku have lower quality.
  - operator-name 4y ago
    Tags are limited how exactly? Prompt crafting becomes a case of selecting the relevant tags, and the embedding space will still capture the dataset. Danbooru is only one such example of well curated tagging and if we ignore copyright there are far more examples. These example just serve as evidence that refining poor labeling is not outside of the relm of possibility as you suggested.
    - GaggiX 4y ago
      A tag-based system would completely lack any kind of contextual information and it would not be possible to create any relationship between words; natural language is much more powerful. An example, an image is tagged: kanna_kamui, kimono and torhu_(maiddragon), who has the kimono? Kanna, Torhu or both? It cannot be known, but with natural language it is possible to describe who is wearing what.
      - throwaway1851 4y ago
        I think this is mainly theoretical at this point. In my experience, current technology doesn’t seem to be utilizing the additional information that comes from natural language all that well. For example: prompt Dalle2 for “a dinner plate on a stack of pancakes” and you will get ordinary images of pancakes on plates, not the other way around. Edit: an experiment comparing tags/BOW vs natural language sequences in image generation tasks would be interesting to see.
        GaggiX 4y ago
        I think this does not work mainly because of the unusual situation you describe, such as "a horse riding a person"; most of the time Dalle 2 is really good at following the prompt.
devmor 4y ago
>It is not possible to manually label hundreds of millions of images to train a model on them Citation, please? I think you mean "the developers of this technology do not want to pay to have hundreds of millions of images labeled".
- GaggiX 4y ago
  It is not believable that someone would pay humans to label 400mln or 5bln images/samples to train a model on them, but I guess if you argument is "everything is possible" then gotcha
  - devmor 4y ago
    You seem to be confusing "possibility" with your personal opinion on what you think would be done by others.
    - GaggiX 4y ago
      As a human being I know human limitations, explicitly labeling 400mln/5bln images for a particular task seems absurd to me, but if you think it is realistically possible perhaps you can give an example.
  - satvikpendem 4y ago
    If it's done in a reCAPTCHA like way, it can be done fairly efficiently and for cheap. In fact Scale AI does just this, they do manual labor operations such as captioning images, as an API. Here's their product for image labeling: https://scale.com/rapid https://scale.com/rapid. Unstable Diffusion is also doing their captioning like how I mentioned, with groups of volunteers as well as hired individuals.
    - GaggiX 4y ago
      Scale seems to do, for example, image classification but not captioning as it would be hard to compare the results with others people to verify the quality (when you have a discrete number of classes is really straightforward), also can you report where you read about the Unstable Diffusion plan for manually labeling image datasets? I want to dig deeper

by GaggiX 4y ago

operator-name 4y ago

Boorus [0] contain millions of images, manually labeled to a pretty high quality. Notably defusion models trained on booru datasets have had good success. This is not the only example of well curated image-tag pairs, especially in artistic circles. It's just that most of them are not CC. [0]: https://en.wiktionary.org/wiki/booru https://en.wiktionary.org/wiki/booru

GaggiX 4y ago
Booru use tags instead of captions, so a model trained on them is really limited; moreover, Danbooru has only 5 million images, while other booru such as gelbooru and sankaku have lower quality.
- operator-name 4y ago
  Tags are limited how exactly? Prompt crafting becomes a case of selecting the relevant tags, and the embedding space will still capture the dataset. Danbooru is only one such example of well curated tagging and if we ignore copyright there are far more examples. These example just serve as evidence that refining poor labeling is not outside of the relm of possibility as you suggested.
  - GaggiX 4y ago
    A tag-based system would completely lack any kind of contextual information and it would not be possible to create any relationship between words; natural language is much more powerful. An example, an image is tagged: kanna_kamui, kimono and torhu_(maiddragon), who has the kimono? Kanna, Torhu or both? It cannot be known, but with natural language it is possible to describe who is wearing what.
    - throwaway1851 4y ago
      I think this is mainly theoretical at this point. In my experience, current technology doesn’t seem to be utilizing the additional information that comes from natural language all that well. For example: prompt Dalle2 for “a dinner plate on a stack of pancakes” and you will get ordinary images of pancakes on plates, not the other way around. Edit: an experiment comparing tags/BOW vs natural language sequences in image generation tasks would be interesting to see.
      - GaggiX 4y ago
        I think this does not work mainly because of the unusual situation you describe, such as "a horse riding a person"; most of the time Dalle 2 is really good at following the prompt.

devmor 4y ago

>It is not possible to manually label hundreds of millions of images to train a model on them Citation, please? I think you mean "the developers of this technology do not want to pay to have hundreds of millions of images labeled".

GaggiX 4y ago
It is not believable that someone would pay humans to label 400mln or 5bln images/samples to train a model on them, but I guess if you argument is "everything is possible" then gotcha
- devmor 4y ago
  You seem to be confusing "possibility" with your personal opinion on what you think would be done by others.
  - GaggiX 4y ago
    As a human being I know human limitations, explicitly labeling 400mln/5bln images for a particular task seems absurd to me, but if you think it is realistically possible perhaps you can give an example.
- satvikpendem 4y ago
  If it's done in a reCAPTCHA like way, it can be done fairly efficiently and for cheap. In fact Scale AI does just this, they do manual labor operations such as captioning images, as an API. Here's their product for image labeling: https://scale.com/rapid https://scale.com/rapid. Unstable Diffusion is also doing their captioning like how I mentioned, with groups of volunteers as well as hired individuals.
  - GaggiX 4y ago
    Scale seems to do, for example, image classification but not captioning as it would be hard to compare the results with others people to verify the quality (when you have a discrete number of classes is really straightforward), also can you report where you read about the Unstable Diffusion plan for manually labeling image datasets? I want to dig deeper