3 ms·

An index? That could be huge. I think it's senseless. So many p2p protocols have support for p2p search (to my mind comes Kademlia), why can't BitTorrent have

An index? That could be huge. I think it's senseless. So many p2p protocols have support for p2p search (to my mind comes Kademlia), why can't BitTorrent have p2p search?

deleted 10y ago
[deleted]
swolchok 10y ago
Vuze (formerly known as Azureus) has (or at least had a prototype of) p2p search. I wrote a paper on building a torrent search index quickly using the DHT records that this feature emitted at the time. http://static.usenix.org/events/woot10/tech/full_papers/Wolchok.pdf http://static.usenix.org/events/woot10/tech/full_papers/Wolc...
- sktrdie 10y ago
  Great work with this. Are you still in academia doing research?
niftich 10y ago
The Mainline DHT (the more-supported DHT vs. Azureus/Vuze DHT) traditionally only contains (iirc, hashes of) torrent infohases mapped to peer lists. With BEP 44 (arbitrary data in DHT) or some other enhancement, this may change, but right now even if you traverse the DHT there's no "interesting" metadata about the keys (or, y'know, the actual content of the .torrent file) inside the DHT. From the sibling comment (posted by swolchok)'s paper, a relevant quote: "We chose to demonstrate our proof of concept only on Vuze due to the significant additional complexity of supporting two DHTs in the crawler. While Mainline does not contain torrent descriptions, its peer lists are keyed by the torrent infohashes themselves, so no additional machinery would be needed to discover infohashes. However, some method would be needed to discover torrent names, such as downloading .torrent files directly from peers using the metadata exchange protocol. To explore this possibility, we built a prototype .torrent crawler. Since we are operating in the context of the Vuze DHT, our DHT crawler needs to obtain both a torrent description and a peer list before the .torrent crawler can contact the appropriate peers with the correct infohash. However, the .torrent files contain useful additional data, including the full listing of files contained in the torrent"
xorcist 10y ago
Kademlia search is also rather useless as it is flooded with false results. This is of interest not only to malware writers but also as a service to sell to rights holders.

An index? That could be huge. I think it's senseless. So many p2p protocols have support for p2p search (to my mind comes Kademlia), why can't BitTorrent have p2p search?

deleted 10y ago

[deleted]

swolchok 10y ago

Vuze (formerly known as Azureus) has (or at least had a prototype of) p2p search. I wrote a paper on building a torrent search index quickly using the DHT records that this feature emitted at the time. http://static.usenix.org/events/woot10/tech/full_papers/Wolchok.pdf http://static.usenix.org/events/woot10/tech/full_papers/Wolc...

sktrdie 10y ago
Great work with this. Are you still in academia doing research?

niftich 10y ago

The Mainline DHT (the more-supported DHT vs. Azureus/Vuze DHT) traditionally only contains (iirc, hashes of) torrent infohases mapped to peer lists. With BEP 44 (arbitrary data in DHT) or some other enhancement, this may change, but right now even if you traverse the DHT there's no "interesting" metadata about the keys (or, y'know, the actual content of the .torrent file) inside the DHT. From the sibling comment (posted by swolchok)'s paper, a relevant quote: "We chose to demonstrate our proof of concept only on Vuze due to the significant additional complexity of supporting two DHTs in the crawler. While Mainline does not contain torrent descriptions, its peer lists are keyed by the torrent infohashes themselves, so no additional machinery would be needed to discover infohashes. However, some method would be needed to discover torrent names, such as downloading .torrent files directly from peers using the metadata exchange protocol. To explore this possibility, we built a prototype .torrent crawler. Since we are operating in the context of the Vuze DHT, our DHT crawler needs to obtain both a torrent description and a peer list before the .torrent crawler can contact the appropriate peers with the correct infohash. However, the .torrent files contain useful additional data, including the full listing of files contained in the torrent"

xorcist 10y ago

Kademlia search is also rather useless as it is flooded with false results. This is of interest not only to malware writers but also as a service to sell to rights holders.