3 ms·

Torrent sites could publish their database dumps via this extension, and consumers would automatically download their updated index. A torrent can be anything;

by sktrdie 10y ago

Torrent sites could publish their database dumps via this extension, and consumers would automatically download their updated index. A torrent can be anything; it can also be a list of other torrents ;) So it does help with content discovery.

cocotino 10y ago
An index? That could be huge. I think it's senseless. So many p2p protocols have support for p2p search (to my mind comes Kademlia), why can't BitTorrent have p2p search?
- deleted 10y ago
  [deleted]
- swolchok 10y ago
  Vuze (formerly known as Azureus) has (or at least had a prototype of) p2p search. I wrote a paper on building a torrent search index quickly using the DHT records that this feature emitted at the time. http://static.usenix.org/events/woot10/tech/full_papers/Wolchok.pdf http://static.usenix.org/events/woot10/tech/full_papers/Wolc...
  - sktrdie 10y ago
    Great work with this. Are you still in academia doing research?
- niftich 10y ago
  The Mainline DHT (the more-supported DHT vs. Azureus/Vuze DHT) traditionally only contains (iirc, hashes of) torrent infohases mapped to peer lists. With BEP 44 (arbitrary data in DHT) or some other enhancement, this may change, but right now even if you traverse the DHT there's no "interesting" metadata about the keys (or, y'know, the actual content of the .torrent file) inside the DHT. From the sibling comment (posted by swolchok)'s paper, a relevant quote: "We chose to demonstrate our proof of concept only on Vuze due to the significant additional complexity of supporting two DHTs in the crawler. While Mainline does not contain torrent descriptions, its peer lists are keyed by the torrent infohashes themselves, so no additional machinery would be needed to discover infohashes. However, some method would be needed to discover torrent names, such as downloading .torrent files directly from peers using the metadata exchange protocol. To explore this possibility, we built a prototype .torrent crawler. Since we are operating in the context of the Vuze DHT, our DHT crawler needs to obtain both a torrent description and a peer list before the .torrent crawler can contact the appropriate peers with the correct infohash. However, the .torrent files contain useful additional data, including the full listing of files contained in the torrent"
- xorcist 10y ago
  Kademlia search is also rather useless as it is flooded with false results. This is of interest not only to malware writers but also as a service to sell to rights holders.
Mizza 10y ago
You don't see a problem with that..? You still need to have the entry point - that's exactly where the censorship is occurring. Try to find a copy of the KickAssTorrents database on Google right now and tell me how easy it is. The approach you're talking about is essentially what http://bitcannon.io/ http://bitcannon.io/ is trying to do - but that requires running a separate app, a local MongoDB server, downloading many gigabytes of database data, and half an hour to import the torrent DB into your local instance, and that STILL requires you to be able to acquire the Torrent database or related infohash, which isn't easy because of the very censorship we're trying to sidestep. Quite frankly, that's a really dumb solution to this censorship and it's naive to say that it's useful. This will never be used in the fashion you're describing and it does nothing for anti-censorship. It makes so much more sense just to have the DHT be actually useful in surfacing content.
- wyager 10y ago
  > You still need to have the entry point Which will now be a short hexadecimal code that is easily shared and can't be taken down.
- sktrdie 10y ago
  If KickAss decides to share their dump using their public key, it becomes much harder to censor because of how the DHT system works. Authorities would need to shut down all the nodes that republish that public key on the DHT, which is incredibly hard to do. Much harder than asking ISPs to shutdown a domain name in DNS.

by sktrdie 10y ago

cocotino 10y ago

An index? That could be huge. I think it's senseless. So many p2p protocols have support for p2p search (to my mind comes Kademlia), why can't BitTorrent have p2p search?

deleted 10y ago
[deleted]
swolchok 10y ago
Vuze (formerly known as Azureus) has (or at least had a prototype of) p2p search. I wrote a paper on building a torrent search index quickly using the DHT records that this feature emitted at the time. http://static.usenix.org/events/woot10/tech/full_papers/Wolchok.pdf http://static.usenix.org/events/woot10/tech/full_papers/Wolc...
- sktrdie 10y ago
  Great work with this. Are you still in academia doing research?
niftich 10y ago
The Mainline DHT (the more-supported DHT vs. Azureus/Vuze DHT) traditionally only contains (iirc, hashes of) torrent infohases mapped to peer lists. With BEP 44 (arbitrary data in DHT) or some other enhancement, this may change, but right now even if you traverse the DHT there's no "interesting" metadata about the keys (or, y'know, the actual content of the .torrent file) inside the DHT. From the sibling comment (posted by swolchok)'s paper, a relevant quote: "We chose to demonstrate our proof of concept only on Vuze due to the significant additional complexity of supporting two DHTs in the crawler. While Mainline does not contain torrent descriptions, its peer lists are keyed by the torrent infohashes themselves, so no additional machinery would be needed to discover infohashes. However, some method would be needed to discover torrent names, such as downloading .torrent files directly from peers using the metadata exchange protocol. To explore this possibility, we built a prototype .torrent crawler. Since we are operating in the context of the Vuze DHT, our DHT crawler needs to obtain both a torrent description and a peer list before the .torrent crawler can contact the appropriate peers with the correct infohash. However, the .torrent files contain useful additional data, including the full listing of files contained in the torrent"
xorcist 10y ago
Kademlia search is also rather useless as it is flooded with false results. This is of interest not only to malware writers but also as a service to sell to rights holders.

Mizza 10y ago

You don't see a problem with that..? You still need to have the entry point - that's exactly where the censorship is occurring. Try to find a copy of the KickAssTorrents database on Google right now and tell me how easy it is. The approach you're talking about is essentially what http://bitcannon.io/ http://bitcannon.io/ is trying to do - but that requires running a separate app, a local MongoDB server, downloading many gigabytes of database data, and half an hour to import the torrent DB into your local instance, and that STILL requires you to be able to acquire the Torrent database or related infohash, which isn't easy because of the very censorship we're trying to sidestep. Quite frankly, that's a really dumb solution to this censorship and it's naive to say that it's useful. This will never be used in the fashion you're describing and it does nothing for anti-censorship. It makes so much more sense just to have the DHT be actually useful in surfacing content.

wyager 10y ago
> You still need to have the entry point Which will now be a short hexadecimal code that is easily shared and can't be taken down.
sktrdie 10y ago
If KickAss decides to share their dump using their public key, it becomes much harder to censor because of how the DHT system works. Authorities would need to shut down all the nodes that republish that public key on the DHT, which is incredibly hard to do. Much harder than asking ISPs to shutdown a domain name in DNS.