9 ms·
Let's not forget one of the best approaches for indexing magnet hashes: https://btdig.com https://btdig.com . It basically connects to the DHT and listens to qu
by gaius_baltar 4y ago
Let's not forget one of the best approaches for indexing magnet hashes: https://btdig.com https://btdig.com . It basically connects to the DHT and listens to queries. When a new one is found, it downloads the torrent data and indexes it.
- LeoPanthera 4y agoWow, thank you for that. That's very clever and very useful. It's a shame they use recaptcha, it's means it's impossible to script searches with something like Jackett.
- vinibrito 4y agoWell, one can write a bot that defeats recaptcha. Trivial, once you know what to do.
- LeoPanthera 4y agoIf "one" can write a bot that defeats recaptcha, what is the point of recaptcha? I think extraordinary claims require extraordinary evidence.
- colecut 4y agoIt would still limit "most" =p
- zenincognito 4y agoSpot on!
- derefr 4y agoI presume by "defeat" they mean "use one of the libraries that allows you to pay $0.10 for each query, to route your CAPTCHA to people in developing countries who will solve them for you." Which is economically viable for individual use-cases, but still economically unviable for spambots.
- e12e 4y agowell, you could run your own instance: https://github.com/btdigg-org/dhtcrawler2 https://github.com/btdigg-org/dhtcrawler2
- LeoPanthera 4y agoAh, that is perfect. Thank you.
- tn1 4y agoAm I missing something or is the source not actually available in that repo? I only see the compiled .beam files
- maybe_pablo 4y agoSource code is available under the “src” branch.
- ghostkba 4y ago[flagged]
- metadat 4y agoBadass. Thank you sir, madame, or they.
- marci 4y ago> sir, madame, or they Neither, I think it's a cylon.
- metadat 4y agoI am assembling the grand mac daddy of all magnet link dbs / seeding systems. If you want to subscribe to updates or are otherwise interested please reply here so I can gauge interest. Will post to HN when ready. This is bigger than piracy, it's about free flow of information over time for humanity. I am up for collaborating and sharing.
- jazzyjackson 4y agoyour default /var/www/html/index.html is showing ;)
- r3trohack3r 4y agoDHT work is fun - you can scrape a lot of interesting content. BitTorrent and Tor both use DHTs for content discovery. If you’re interested in building a minimum viable scrapper, this is a good reference: https://github.com/retrohacker/Taboo/blob/main/lib/index.js https://github.com/retrohacker/Taboo/blob/main/lib/index.js The trick to content discovery on the DHT is to parrot any incoming request you receive. So if someone asks for a hash or announces a hash, you turn around and ask for that hash too. You can spin up multiple DHT nodes to increase your coverage of the table. Some cool user experiences can be built off of this. I’ve discovered a tonne of interesting content using “BitTorrent Roulette” applications that download random hashes from the DHT - often times finding fascinating content from languages I don’t speak.
- KennyBlanken 4y ago"BitTorrent Roulette" sounds like a wonderful way to end up with child porn on your computer.
- CyanBird 4y agoAre people still so naive as to use torrent for that? Well I'd guess I might be the naive, given the ample amount of honey pots and such
- r3trohack3r 4y agoI’ve found that, no, they aren’t. The ratio of this content on the DHT is so incredibly low you’re very unlikely to pull it. I never have, and I’ve resolved a metric tonne of random torrent hashes. You do find content labeled as such - but it’s either password protected rars (I suspect the password is to remove plausible deniability? Idk what’s inside these rars, I obviously don’t have the password and I immediately shred them when they do fall out of the roulette wheel - I’m not interested in trying to break into them) or a honeypot with a government agency video advertising a support hotline. I’ve noticed a government in LatAm has a lot of honeypot content on the public BitTorrent DHT. It’s a guy sitting at a desk in front of the state flag advertising an anonymous support hotline. I have, however found some ethically questionable things. Dumps of credit card info, sus military schematics written in an Eastern European language, etc. Ethics aside - if you live in the United States, more than 90% of what falls out of the Roulette Wheel is going to be strictly illegal because of copyright law. Downloading random hashes from the DHT is not something I’d suggest doing from a computer that can easily be traced back to your identity. I summarize this in the README of Taboo: https://github.com/retrohacker/taboo https://github.com/retrohacker/taboo
- squarefoot 4y ago> https://btdig.com https://btdig.com First hint suggested: "interracial gangbang" They might want to put those hints behind a NSFW warning; just my .2c.
- killingtime74 4y agoAnyone who has spent more than 5 minutes in torrent-land has seen NSFW
- jokoon 4y agothat link doesn't work