3 ms·

Nice project! Last year I made a similar PoC on Elixir based on the paper "Crawling DHTs for fun and profit"[1]. In that paper the researcher uses a pool of IP

• boramalper 10y ago
  I too read that paper, but magnetico crawls Mainline DHT (which is the de facto standard) whilst the paper is on Vuze DHT, though the idea is more or less the same. This project uses only one instance of crawler running at the same time, so one IP and one port for crawling the DHT. Whenever it receives a `GET_PEERS` query from another peer in the network, it forges itself a new ID that has the first 15 bytes in common with the info hash in the query, so that the querying node would think that it found the closest (or a close enough) node to announce. After responding back, most of the times the same querying node sends a `ANNOUNCE_PEER` query to register itself as a peer downloading that particular torrent. Upon receiving `ANNOUNCE_PEER` query, magneticod then connects the peer using BitTorrent (TCP) protocol, receives the complete metadata, and closes the connection. Keep repeating these and you'll end up with thousands of torrents at the end of each day. Possibly, utilizing multiple IPs might increase the throughput and prevent IP based bans, but I didn't bother to be honest, as the current solution seems to work well enough.