5 ms·

In general, generative models are taking publicly available content from creators and are monetizing them for big tech. Same with stable diffusion, AI art. And

by safog 4y ago

In general, generative models are taking publicly available content from creators and are monetizing them for big tech. Same with stable diffusion, AI art. And it'll be the same with LLMs. Eventually the whole internet would be flooded with cheap AI generated content and clearly AIs need human generated content to train on so it'll be the snake eats itself.

grogenaut 4y ago
There are parallels in manufacturing and product. People who sell with high quality control, better methods. And everything else gets cheaper more common and lower (good enough) quality.
the_snooze 4y ago
>Eventually the whole internet would be flooded with cheap AI generated content and clearly AIs need human generated content to train on so it'll be the snake eats itself. Geniune human-generated content will retreat to account-gated networks and private group chats where everyone knows one another. The rest of the internet will just be incestuous AI-generated chum. In a way, it'd be an improvement. Genuine connection doesn't scale, so let's be honest about keeping it away from random parasites online.
binkHN 4y ago
> In general, generative models are taking publicly available content from creators and are monetizing them for big tech. This is fair—also in fairness, whenever my Google speaker answers a question, it always tells me which site the answer is from.
merely-unlikely 4y ago
That’s not so different from the current state of the world in which many search results are full of cheap SEO content. Now instead of appending Reddit to my search I just go to chatGPT. Probably similar levels of accuracy between Reddit and GPT anyway while the SEO content is just garbage filler.
- j16sdiz 4y ago
  Until Search-Engine-Optimization is replaced with Chat-Bot-Optimization..? A chat bot, like search index, need to updated for new and current events to stay relevant. I can’t see why it can’t be deceived by spams
- CSMastermind 4y ago
  Around of quarter of GPT's training data was Reddit so in some sense it's already a Reddit response generating API.
  - Cipater 4y ago
    This can't be right. Any place I can read about this?
    - CSMastermind 4y ago
      https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf WebText and WebText2 referenced in their papers are corpuses based on Reddit submissions which had a 22% weight in their training model. https://openwebtext2.readthedocs.io/en/latest/ https://openwebtext2.readthedocs.io/en/latest/ This is larger than Wikipedia (3% weight) or either of their two book corpuses (8% each). The only other data included was a filtered set from Common Crawl (weighted 60%). --- I was imprecise with my language before but hopefully that at least provides some clarity.
luxcem 4y ago
> clearly AIs need human generated content to train on so it'll be the snake eats itself. For the time being but there is no reason AI couldn't produce genuine original content. Real life human artists also use previous content for inspiration.
- the_snooze 4y ago
  Human art isn't only previous content though. It's filtered through the experiences of the artist themselves. Eric Clapton surely took inspiration from prior music, but "Tears in Heaven" is about a uniquely personal and human experience of losing a child. If a computer had written it, it wouldn't have the same artistic or emotional weight.
yamtaddle 4y ago
The Web never really replaced books as information sources for important things. The way to access most of the best stuff available via the Web is to... download ebooks, thus leaving the Web, if not just the "buy" button for real, physical books. The single most intellectually valuable website on the entire Web is very likely Library Genesis, where the only Web content is a catalog of books you can pirate by clicking a link, and it's, like, a lot more valuable than any other site (even Wikipedia). It may well be more valuable, in those terms, than the entire rest of the Web combined. If serious book publishers survive a while longer and if non-fiction books aren't overrun with dubiously-accurate AI bullshit, things won't actually change all that much, I think. As far as written content goes, the (public) Web is most useful for opinions or product discovery, and even those can already hardly be trusted because of all the marketing astroturfing. AI garbage barely changes that already-toxic dynamic. Video's another matter—some video content on the Web is great and has ~no at-least-as-good replacement anywhere else, in any other medium. But it's also all but completely monopolized by Youtube and hardly participates in or factors into the broader Web.

by safog 4y ago

grogenaut 4y ago

There are parallels in manufacturing and product. People who sell with high quality control, better methods. And everything else gets cheaper more common and lower (good enough) quality.

the_snooze 4y ago

>Eventually the whole internet would be flooded with cheap AI generated content and clearly AIs need human generated content to train on so it'll be the snake eats itself. Geniune human-generated content will retreat to account-gated networks and private group chats where everyone knows one another. The rest of the internet will just be incestuous AI-generated chum. In a way, it'd be an improvement. Genuine connection doesn't scale, so let's be honest about keeping it away from random parasites online.

binkHN 4y ago

> In general, generative models are taking publicly available content from creators and are monetizing them for big tech. This is fair—also in fairness, whenever my Google speaker answers a question, it always tells me which site the answer is from.

merely-unlikely 4y ago

That’s not so different from the current state of the world in which many search results are full of cheap SEO content. Now instead of appending Reddit to my search I just go to chatGPT. Probably similar levels of accuracy between Reddit and GPT anyway while the SEO content is just garbage filler.

j16sdiz 4y ago
Until Search-Engine-Optimization is replaced with Chat-Bot-Optimization..? A chat bot, like search index, need to updated for new and current events to stay relevant. I can’t see why it can’t be deceived by spams
CSMastermind 4y ago
Around of quarter of GPT's training data was Reddit so in some sense it's already a Reddit response generating API.
- Cipater 4y ago
  This can't be right. Any place I can read about this?
  - CSMastermind 4y ago
    https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf WebText and WebText2 referenced in their papers are corpuses based on Reddit submissions which had a 22% weight in their training model. https://openwebtext2.readthedocs.io/en/latest/ https://openwebtext2.readthedocs.io/en/latest/ This is larger than Wikipedia (3% weight) or either of their two book corpuses (8% each). The only other data included was a filtered set from Common Crawl (weighted 60%). --- I was imprecise with my language before but hopefully that at least provides some clarity.

luxcem 4y ago

> clearly AIs need human generated content to train on so it'll be the snake eats itself. For the time being but there is no reason AI couldn't produce genuine original content. Real life human artists also use previous content for inspiration.

the_snooze 4y ago
Human art isn't only previous content though. It's filtered through the experiences of the artist themselves. Eric Clapton surely took inspiration from prior music, but "Tears in Heaven" is about a uniquely personal and human experience of losing a child. If a computer had written it, it wouldn't have the same artistic or emotional weight.

yamtaddle 4y ago

The Web never really replaced books as information sources for important things. The way to access most of the best stuff available via the Web is to... download ebooks, thus leaving the Web, if not just the "buy" button for real, physical books. The single most intellectually valuable website on the entire Web is very likely Library Genesis, where the only Web content is a catalog of books you can pirate by clicking a link, and it's, like, a lot more valuable than any other site (even Wikipedia). It may well be more valuable, in those terms, than the entire rest of the Web combined. If serious book publishers survive a while longer and if non-fiction books aren't overrun with dubiously-accurate AI bullshit, things won't actually change all that much, I think. As far as written content goes, the (public) Web is most useful for opinions or product discovery, and even those can already hardly be trusted because of all the marketing astroturfing. AI garbage barely changes that already-toxic dynamic. Video's another matter—some video content on the Web is great and has ~no at-least-as-good replacement anywhere else, in any other medium. But it's also all but completely monopolized by Youtube and hardly participates in or factors into the broader Web.