3 ms·

The biggest theft in history with governments as complices

sillysaurusx 3y ago
With the most valuable haul of all: not gold or coin, but knowledge. How terrible. But no, I don’t think governments are accomplices. The smartest move would be for China to wait and see how this plays out. It could significantly shift the epicenter of AI work to China. The single biggest limiting factor for open source LLMs is data, and no one in America or Europe wants to be sued.
- fragmede 3y ago
  So fund it with crypto and release it on 4chan.
- pilimi_anna 3y ago
  For what it's worth, we also have millions of previously unreleased (in bulk) books in English, mostly non-fiction, that are available for torrent on our website. High-speed access available for anyone who can do at-scale text extraction, or who can supply us with new collections.
  - sillysaurusx 3y ago
    Anna! I just want to say, I love you. Everything about what you’re doing is heroic. Whoever and wherever you are, thank you. Please focus on your opsec. The more visible you become, the progressively angrier people will get. Don’t do anything silly like edit your Wikipedia page from your house. With that out of the way, someone I know happens to have the original books3 epub files. I think they can be convinced to send them to you. It’s only 200,000 books, but that could theoretically grow your collection by 10% or so. I don’t know whether that would be helpful to you (you’ve far surpassed books3 at this point), but if so, let me know. Given the legal risks, the best course of action for AI companies is probably to ignore English and European books entirely. There is plenty of Chinese data, and the models would learn all the same concepts without exposing anyone to lawsuits.
    - pilimi_anna 3y ago
      Yes please, put us in touch by email. Or feel free to email me yourself and we can set up more secure comms from there. Thanks so much for everything you are doing as well!
    - stavros 3y ago
      Since you're pretty knowledgeable about these things, I think I should ask here: I've made a fairly simple design for a program based on BitTorrent, that will allow people to "donate" their disk space to organizations like archive.org, Anna's Archive, and anything else that needs data hosted. Basically, you download a client, say "allocate 2 TB of my disks to whatever archive.org/donate/disk.rss" says, and the server/client combination ensures you download and seed the rarest 2TB of the collection. This design is also open, in the sense that the server can share the database of torrents it contains, and anyone can use it to fetch any of the files in the dataset from the swarm. Would something like this be at all useful? I've emailed a few archivists, but I got no response, and the one person I've managed to talk to about this said there have been a few attempts on this, but they always fail for one reason or another.
      - sillysaurusx 3y ago
        You are literally building what I’ve been slowly working towards on my own. This seems like a very good sign. Multiple simultaneous discovery is a common occurrence in the sciences. The hard part is that those who donate their space have authority over that space. It’s the Byzantine fault tolerance problem: imagine if 4chan donated their space, then started serving CSAM instead of the expected data. You can use hashes to verify integrity, but then the question becomes who gets to decide which hashes are ok. And hashing makes it impossible to edit large files, which is a frequent occurrence in LLM work. You’re constantly tweaking your datasets and spitting out new blobs. Direct answer: yes, you’re doing good work, and you should keep doing it. I would personally use this for storing books3 transformations. The other hard part is that you’ll want at least some redundancy — see 6.824 distributed systems, or the GFS paper. It’s why I’ve been implementing Raft and toying with some kind of distributed consensus without a blockchain. (Such consensus is still possible if the researchers were granted authority over what can be stored — which is the whole reason people are donating their disk space in the first place.) Another issue is sudden bandwidth loss. Data storage is one part of it. The other half is rapid transfer. By replicating the data, you can pull it from multiple replicas at once (I.e. there are more seeders). This also protects against someone suddenly getting throttled, or just having a power outage. The protocol should prioritize donors with high bandwidth over vast storage space. Feel free to DM me on Twitter if you’d like to toss around some design ideas more seriously, and thank you for trying to build this.
deleted 3y ago
[deleted]
golol 3y ago
Any entity should be allowed to read, process and update its internal state using published media. Only your productions should be judged based on copyright law.
pixl97 3y ago
The biggest theft in history was you being taught the alphabet, numbers, and words without having to pay a creator for them. This mistake will be reversed. All knowledge will be licenced, all words will be owned. Short summary of The Right to Read, and evidently the world you want.

With the most valuable haul of all: not gold or coin, but knowledge. How terrible. But no, I don’t think governments are accomplices. The smartest move would be for China to wait and see how this plays out. It could significantly shift the epicenter of AI work to China. The single biggest limiting factor for open source LLMs is data, and no one in America or Europe wants to be sued.

fragmede 3y ago
So fund it with crypto and release it on 4chan.
pilimi_anna 3y ago
For what it's worth, we also have millions of previously unreleased (in bulk) books in English, mostly non-fiction, that are available for torrent on our website. High-speed access available for anyone who can do at-scale text extraction, or who can supply us with new collections.
- sillysaurusx 3y ago
  Anna! I just want to say, I love you. Everything about what you’re doing is heroic. Whoever and wherever you are, thank you. Please focus on your opsec. The more visible you become, the progressively angrier people will get. Don’t do anything silly like edit your Wikipedia page from your house. With that out of the way, someone I know happens to have the original books3 epub files. I think they can be convinced to send them to you. It’s only 200,000 books, but that could theoretically grow your collection by 10% or so. I don’t know whether that would be helpful to you (you’ve far surpassed books3 at this point), but if so, let me know. Given the legal risks, the best course of action for AI companies is probably to ignore English and European books entirely. There is plenty of Chinese data, and the models would learn all the same concepts without exposing anyone to lawsuits.
  - pilimi_anna 3y ago
    Yes please, put us in touch by email. Or feel free to email me yourself and we can set up more secure comms from there. Thanks so much for everything you are doing as well!
  - stavros 3y ago
    Since you're pretty knowledgeable about these things, I think I should ask here: I've made a fairly simple design for a program based on BitTorrent, that will allow people to "donate" their disk space to organizations like archive.org, Anna's Archive, and anything else that needs data hosted. Basically, you download a client, say "allocate 2 TB of my disks to whatever archive.org/donate/disk.rss" says, and the server/client combination ensures you download and seed the rarest 2TB of the collection. This design is also open, in the sense that the server can share the database of torrents it contains, and anyone can use it to fetch any of the files in the dataset from the swarm. Would something like this be at all useful? I've emailed a few archivists, but I got no response, and the one person I've managed to talk to about this said there have been a few attempts on this, but they always fail for one reason or another.
    - sillysaurusx 3y ago
      You are literally building what I’ve been slowly working towards on my own. This seems like a very good sign. Multiple simultaneous discovery is a common occurrence in the sciences. The hard part is that those who donate their space have authority over that space. It’s the Byzantine fault tolerance problem: imagine if 4chan donated their space, then started serving CSAM instead of the expected data. You can use hashes to verify integrity, but then the question becomes who gets to decide which hashes are ok. And hashing makes it impossible to edit large files, which is a frequent occurrence in LLM work. You’re constantly tweaking your datasets and spitting out new blobs. Direct answer: yes, you’re doing good work, and you should keep doing it. I would personally use this for storing books3 transformations. The other hard part is that you’ll want at least some redundancy — see 6.824 distributed systems, or the GFS paper. It’s why I’ve been implementing Raft and toying with some kind of distributed consensus without a blockchain. (Such consensus is still possible if the researchers were granted authority over what can be stored — which is the whole reason people are donating their disk space in the first place.) Another issue is sudden bandwidth loss. Data storage is one part of it. The other half is rapid transfer. By replicating the data, you can pull it from multiple replicas at once (I.e. there are more seeders). This also protects against someone suddenly getting throttled, or just having a power outage. The protocol should prioritize donors with high bandwidth over vast storage space. Feel free to DM me on Twitter if you’d like to toss around some design ideas more seriously, and thank you for trying to build this.

deleted 3y ago

[deleted]

golol 3y ago

Any entity should be allowed to read, process and update its internal state using published media. Only your productions should be judged based on copyright law.

pixl97 3y ago

The biggest theft in history was you being taught the alphabet, numbers, and words without having to pay a creator for them. This mistake will be reversed. All knowledge will be licenced, all words will be owned. Short summary of The Right to Read, and evidently the world you want.