3 ms·

Closest to this would be https://www.eleuther.ai https://www.eleuther.ai whose training data is largely public and training processes are openly discussed, plan

Closest to this would be https://www.eleuther.ai https://www.eleuther.ai whose training data is largely public and training processes are openly discussed, planned, and evaluated on their Discord server. Much of their training dataset is available at https://the-eye.eu https://the-eye.eu (their onion link is considered "primary", however, due to copyright concerns)

sillysaurusx 2y ago
Is our dataset still available? I thought it was taken offline. Where do you go under that link to get it? E.g. https://the-eye.eu/public/AI/pile/readme.txt https://the-eye.eu/public/AI/pile/readme.txt says it’s gone (and "old news"? I disagree).
- wongarsu 2y ago
  There are still plenty of reliably sources for magnet links to The Pile, e.g. [1]. The DMCA takedowns are just a minor inconvenience. 1: https://web.archive.org/web/20230820001113/https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://web.archive.org/web/20230820001113/https://academict...
  - sillysaurusx 2y ago
    Thank you. How’d you dig this one up?
    - wongarsu 2y ago
      [1] is the first result if I google "the pile torrent". It doesn't link to the torrent because of a DMCA notice, so I just used the wayback machine to retrieve a version from before the date of that notice. Don't tell the publisher. 1: https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://academictorrents.com/details/0d366035664fdf51cfbe9f7...
      - sillysaurusx 2y ago
        Frustratingly, they scan my comments, so hopefully they won’t bother filing a DMCA for that. (Seeing "sillysaurusx" appear in print on official court documents was pretty amusing out of context, though.)
- robertk 2y ago
  Shawn, there is a mildly redacted version available at https://huggingface.co/datasets/monology/pile-uncopyrighted https://huggingface.co/datasets/monology/pile-uncopyrighted
  - sillysaurusx 2y ago
    Thank you.

sillysaurusx 2y ago

Is our dataset still available? I thought it was taken offline. Where do you go under that link to get it? E.g. https://the-eye.eu/public/AI/pile/readme.txt https://the-eye.eu/public/AI/pile/readme.txt says it’s gone (and "old news"? I disagree).

wongarsu 2y ago
There are still plenty of reliably sources for magnet links to The Pile, e.g. [1]. The DMCA takedowns are just a minor inconvenience. 1: https://web.archive.org/web/20230820001113/https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://web.archive.org/web/20230820001113/https://academict...
- sillysaurusx 2y ago
  Thank you. How’d you dig this one up?
  - wongarsu 2y ago
    [1] is the first result if I google "the pile torrent". It doesn't link to the torrent because of a DMCA notice, so I just used the wayback machine to retrieve a version from before the date of that notice. Don't tell the publisher. 1: https://academictorrents.com/details/0d366035664fdf51cfbe9f733953ba325776e667 https://academictorrents.com/details/0d366035664fdf51cfbe9f7...
    - sillysaurusx 2y ago
      Frustratingly, they scan my comments, so hopefully they won’t bother filing a DMCA for that. (Seeing "sillysaurusx" appear in print on official court documents was pretty amusing out of context, though.)
robertk 2y ago
Shawn, there is a mildly redacted version available at https://huggingface.co/datasets/monology/pile-uncopyrighted https://huggingface.co/datasets/monology/pile-uncopyrighted
- sillysaurusx 2y ago
  Thank you.