5 ms·

You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it

You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap

mpalmer 4y ago
Web scrape... what, exactly? How will "web scraping" lead to high quality results?
- tomatowurst 4y ago
  if you have to ask, the answer would be irrelevant.
  - mpalmer 4y ago
    I certainly don't want to waste the time of someone who knows what they're talking about.
- visarga 4y ago
  Already does. It works by magic, we put images on one side and text on the other side and then power it up. Let it simmer for a few months at megawatt power levels.
- Kleto 4y ago
  Reddit has plenty of nsfw categories. And tags. Insta has tags. Plenty of others have nsfw and tags
visarga 4y ago
You don't realize how hard it is to make an inclusive, cleaned up dataset. Take a look at this Notion from BigScience detailing their workgroups. Three of them are related to preparing the dataset. https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454bb8?v=92f51ef4650c4fc085cda88e23c8ebbb https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454... And this is just for a text-only scrape.
minimaxir 4y ago
Training an AI requires high quality, clean/normalized data, which is very difficult to do at millions/billions of data points and is very frequently done incorrectly with silent failures.
- tomatowurst 4y ago
  There's a section of the internet where you can easily find billions of images or can be generated from moving pictures. Even upscaled. I really didn't expect to have to spell it out. hint: they are all about one thing and there are a lot of eager volunteers to help on those websites. It would be easy to "normalize/clean/classify" because the pictures would have a consistent theme, thus reducing the amount of parameters. We are not trying to generate elephants getting railed on a SpaceX rocket flying in oil painting style (although I'm sure theres people into that and its not my place to judge), we are just trying to remove the human cost out of this necessary evil. I can't believe nobody is investing in "DALL-E-2-4-PORN". This sounds like an X amount of money thrown at a hugely sticky product that can be iterated (with the current trend in hardware) to the point where it literally generates billions of dollars in revenues for ages to come (no pun intended).
  - thorum 4y ago
    Does anyone actually want porn of fake, AI generated “people”? Seems like most of the demand would be deepfakes of real people, which is both highly unethical and a good way to get your business sued out of existence.
    - Kleto 4y ago
      Why not? Do you prefer known porn actress over non known ones? I would say that I might know a handful of names but search for them rarly.
    - sterlind 4y ago
      furry porn? vore? transformation? selfcest? there's a ton of niche fetishes that aren't really doable IRL, and only exist as art. generated images would be a game changer.
dotnet00 4y ago
In my experience, scraping the data is the easy part. Once you've scraped it you've got to get rid of all the garbage, which is where the issues arise, especially if you're just blindly scraping everything you can find. For example, in a generative model I'm working on, I have a dataset consisting of ~5M images just blindly scraped from a website. After filtering, this drops down to ~500k images, yet a model trained on that performs worse than one trained on a set of 30k curated images (picked based on a manually evaluated list of "known good" artists), where filtering brings it down to ~18k images. The larger dataset, while containing more information, also contains more errors, many of them pretty hard to filter out.
- tomatowurst 4y ago
  Which is why porn is such a great dataset for crowdsource: - lots of people are stimulated by it - lots of people want DALL-E-2 for porn - and lots of people are willing to work towards that common goal The beauty of this is that people are just going to keep coming and coming to it. Like I'm trying to be mature and serious about this. What's it going to take? - Community responsible for scraping dataset, generating image dataset from moving pictures, upscaling said dataset. - Crowdsourced labeling, cleaning, normalizing dataset - Crowdfunding to train, host, and publish. All of the above are not easy by any means but its much more achievable than trying to build a generic DALL-E that can create anything. Really my point is that the scope of the dataset has narrower range in terms of desired output where as DALL-E-2 casts a far wider net. Specialization is key here.
  - dotnet00 4y ago
    I was writing up a whole explanation for how that couldn't possibly work, but tbh you aren't really wrong with that, after all captchas these days do similar work. I guess the primary concern with a porn model would be the ethics of it, which might turn off any company from helping out on training resources (for example, Google's TPU Research Cloud requires you to follow their code of ethics on AI, which would be very difficult to do with a topic as sensitive to people as porn).
    - 4y ago

You do realize web scraping has gotten very cheap at scale and easy now too. It's an after thought for me, I'm more concerned with the economics of training, it can't be cheap

mpalmer 4y ago

Web scrape... what, exactly? How will "web scraping" lead to high quality results?

tomatowurst 4y ago
if you have to ask, the answer would be irrelevant.
- mpalmer 4y ago
  I certainly don't want to waste the time of someone who knows what they're talking about.
visarga 4y ago
Already does. It works by magic, we put images on one side and text on the other side and then power it up. Let it simmer for a few months at megawatt power levels.
Kleto 4y ago
Reddit has plenty of nsfw categories. And tags. Insta has tags. Plenty of others have nsfw and tags

visarga 4y ago

You don't realize how hard it is to make an inclusive, cleaned up dataset. Take a look at this Notion from BigScience detailing their workgroups. Three of them are related to preparing the dataset. https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454bb8?v=92f51ef4650c4fc085cda88e23c8ebbb https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454... And this is just for a text-only scrape.

minimaxir 4y ago

Training an AI requires high quality, clean/normalized data, which is very difficult to do at millions/billions of data points and is very frequently done incorrectly with silent failures.

tomatowurst 4y ago
There's a section of the internet where you can easily find billions of images or can be generated from moving pictures. Even upscaled. I really didn't expect to have to spell it out. hint: they are all about one thing and there are a lot of eager volunteers to help on those websites. It would be easy to "normalize/clean/classify" because the pictures would have a consistent theme, thus reducing the amount of parameters. We are not trying to generate elephants getting railed on a SpaceX rocket flying in oil painting style (although I'm sure theres people into that and its not my place to judge), we are just trying to remove the human cost out of this necessary evil. I can't believe nobody is investing in "DALL-E-2-4-PORN". This sounds like an X amount of money thrown at a hugely sticky product that can be iterated (with the current trend in hardware) to the point where it literally generates billions of dollars in revenues for ages to come (no pun intended).
- thorum 4y ago
  Does anyone actually want porn of fake, AI generated “people”? Seems like most of the demand would be deepfakes of real people, which is both highly unethical and a good way to get your business sued out of existence.
  - Kleto 4y ago
    Why not? Do you prefer known porn actress over non known ones? I would say that I might know a handful of names but search for them rarly.
  - sterlind 4y ago
    furry porn? vore? transformation? selfcest? there's a ton of niche fetishes that aren't really doable IRL, and only exist as art. generated images would be a game changer.

dotnet00 4y ago

In my experience, scraping the data is the easy part. Once you've scraped it you've got to get rid of all the garbage, which is where the issues arise, especially if you're just blindly scraping everything you can find. For example, in a generative model I'm working on, I have a dataset consisting of ~5M images just blindly scraped from a website. After filtering, this drops down to ~500k images, yet a model trained on that performs worse than one trained on a set of 30k curated images (picked based on a manually evaluated list of "known good" artists), where filtering brings it down to ~18k images. The larger dataset, while containing more information, also contains more errors, many of them pretty hard to filter out.

tomatowurst 4y ago
Which is why porn is such a great dataset for crowdsource: - lots of people are stimulated by it - lots of people want DALL-E-2 for porn - and lots of people are willing to work towards that common goal The beauty of this is that people are just going to keep coming and coming to it. Like I'm trying to be mature and serious about this. What's it going to take? - Community responsible for scraping dataset, generating image dataset from moving pictures, upscaling said dataset. - Crowdsourced labeling, cleaning, normalizing dataset - Crowdfunding to train, host, and publish. All of the above are not easy by any means but its much more achievable than trying to build a generic DALL-E that can create anything. Really my point is that the scope of the dataset has narrower range in terms of desired output where as DALL-E-2 casts a far wider net. Specialization is key here.
- dotnet00 4y ago
  I was writing up a whole explanation for how that couldn't possibly work, but tbh you aren't really wrong with that, after all captchas these days do similar work. I guess the primary concern with a porn model would be the ethics of it, which might turn off any company from helping out on training resources (for example, Google's TPU Research Cloud requires you to follow their code of ethics on AI, which would be very difficult to do with a topic as sensitive to people as porn).
  - 4y ago