3 ms·

What kind of bandwidth usage should be expected from a DHT crawler like this?

• KennyBlanken 3y ago
  After ~30-60 minutes of running, still less than 100kB/sec combined in and out. However, as others have noted, nodes don't communicate much with nodes that haven't been up for a while (days.) It's using roughly 6% CPU time for the crawler and another 1-2% for postgres, on a second-gen i7. As a datapoint to set expectations: 4000 torrents have been captured so far, and somewhat surprisingly, they're not very current results, necessarily. For example, a certain wildly popular TV series about samurai in space swinging very hot swords around which just had its season ending episode last night (I think)...that ep isn't in my list so far, but the episode prior to it, and the first two episodes, are. There's a ton of random, low-seed torrents, so it's actually kind of interesting to search by type, year, etc and see what comes up.