4 ms·
What do you mean with updated? Are you scraping the individual pages for all those torrents on Kickass (and others)? Some more technical details would be lovely
by tokenizerrr 12y ago
What do you mean with updated? Are you scraping the individual pages for all those torrents on Kickass (and others)? Some more technical details would be lovely.
- Prefinem 12y agoSeveral sites provide apis or daily dumps of their torrents, so I am loading them, indexing them, and then scraping the trackers for their information. How technical do you want?
- tokenizerrr 12y agoAs technical as you're willing to get :)
- Prefinem 12y agoDaily dumps are pulled from torrent sites. I grab each hash and title and store those. Then all the new torrents, I grab the hash, and scrape the demonii tracker to get seeder/leecher information. Also at the same time, I am constantly scraping the demonii tracker for seeder/leecher information for older torrents based on age. It currently takes 5 days to update all the torrents, but I will be implementing an auto-pruning system for torrents that are dead for so long, negative votes, etc etc