4 ms·
I stumbled across a DHT crawler, and started running it. Over time I added random things to it, like IPv6 and the ability to download metadata. I was mostly i
by banana_giraffe 5y ago
I stumbled across a DHT crawler, and started running it. Over time I added random things to it, like IPv6 and the ability to download metadata. I was mostly interested in using it to learn how DHT worked both in theory and in practice, and for that, it did it's job.
The page for the crawler I started with is long gone, but this appears to be a reasonable copy on github:
https://github.com/zlzlovezl/simDHT-1 https://github.com/zlzlovezl/simDHT-1
I'm sure you can find other copies, with features added. Looking around, it seems like quite a few people have taken this and made it their own over time.
It's far from perfect in this state, but a fun toy to play with. I will say, from a point of painful experience: The way this crawler works tends to make some of the various people that track such things think you're sharing pretty much all the things. If that's an issue where you live, don't run it from your home connection.