3 ms·
- am not familiar with tor except for the fact that it uses something called onion with multiple hops and is very hard to trace making it privacy friendly - ar
by vivzkestrel 2mo ago
- am not familiar with tor except for the fact that it uses something called onion with multiple hops and is very hard to trace making it privacy friendly
- are we talking here about using tor programmatically?
- what I am referring to on my end is how to go about programmatically downloading a 100 million videos one after the other on s3
- what kind of bot detection / evasion / ip address / proxy rotation mechanisms are needed
- are you familiar with libraries in the pypi ecosystem capable of tackling some of these concerns?
- is aiohttp / asyncio python enough for this paired with multiprocessing or do we have better options?
- earthnail 2mo ago- Use the python Library stem to control the Tor daemon and send sighup commands when necessary - yt dlp has builtin support for tor - write the retry logic using your favourite coding AI - spin up a cheap cloud instance and let it run