4 ms·

I look forward for AI trained entirely on Wikipedia and classical literature with no twitter and no contemporary art in sight. It would be sublime. Let's face i

I look forward for AI trained entirely on Wikipedia and classical literature with no twitter and no contemporary art in sight. It would be sublime. Let's face it, the creators of century XXI way overestimate the importance of their stuff. It's mostly deleterious to the culture.

to11mtm 2y ago
> trained entirely on Wikipedia does the CC used by wikipedia allow training? > and no contemporary art in sight will note per above, they may be able to scrape other CC if they -can- scrape wikipedia. > It's mostly deleterious to the culture. Yep and based on how I already sometimes catch people treating LLM hallucinations as fact, it's likely gonna get worse anyway.
- regularfry 2y ago
  > does the CC used by wikipedia allow training? This is backwards. Absent something that prohibits it, training is allowed. The question should be this: what do you believe prohibits it in this case?
  - jprete 2y ago
    The author of a work has copyright automatically, so what would prohibit it by default is copyright law. In the specific case of Wikipedia I would guess it's allowed by the license, but that's not generally true.
    - regularfry 2y ago
      Training isn't copying (any more than a browser or CDN cache is) or distribution. Copyright is out of scope. Is there anything else that would prohibit training?
      - jprete 2y ago
        All of those things are copying. Browser caches probably fall under fair use. CDNs are contracted by the distributor so literally licensed.
        regularfry 2y ago
        The act of copying while training is incidental, in the same way as browser caches are incidental to the viewing of the content. Except that with training, you don't want to end up with a duplicate. The whole point is not to copy the original. Training is not copying.
- lgas 2y ago
  Wikipedia provides dumps of their data[1] and there are sources that prepare it for training as well[2]. [1]: https://dumps.wikimedia.org/ https://dumps.wikimedia.org/ [2]: https://www.tensorflow.org/datasets/catalog/wikipedia https://www.tensorflow.org/datasets/catalog/wikipedia
fakedang 2y ago
I think that's basically what models such as Mistral are doing. They mentioned that they use very sanitized datasets for training as compared to simply scraping the web.

to11mtm 2y ago

> trained entirely on Wikipedia does the CC used by wikipedia allow training? > and no contemporary art in sight will note per above, they may be able to scrape other CC if they -can- scrape wikipedia. > It's mostly deleterious to the culture. Yep and based on how I already sometimes catch people treating LLM hallucinations as fact, it's likely gonna get worse anyway.

regularfry 2y ago
> does the CC used by wikipedia allow training? This is backwards. Absent something that prohibits it, training is allowed. The question should be this: what do you believe prohibits it in this case?
- jprete 2y ago
  The author of a work has copyright automatically, so what would prohibit it by default is copyright law. In the specific case of Wikipedia I would guess it's allowed by the license, but that's not generally true.
  - regularfry 2y ago
    Training isn't copying (any more than a browser or CDN cache is) or distribution. Copyright is out of scope. Is there anything else that would prohibit training?
    - jprete 2y ago
      All of those things are copying. Browser caches probably fall under fair use. CDNs are contracted by the distributor so literally licensed.
      - regularfry 2y ago
        The act of copying while training is incidental, in the same way as browser caches are incidental to the viewing of the content. Except that with training, you don't want to end up with a duplicate. The whole point is not to copy the original. Training is not copying.
lgas 2y ago
Wikipedia provides dumps of their data[1] and there are sources that prepare it for training as well[2]. [1]: https://dumps.wikimedia.org/ https://dumps.wikimedia.org/ [2]: https://www.tensorflow.org/datasets/catalog/wikipedia https://www.tensorflow.org/datasets/catalog/wikipedia

fakedang 2y ago

I think that's basically what models such as Mistral are doing. They mentioned that they use very sanitized datasets for training as compared to simply scraping the web.