5 ms·
As URLs are provided by random users, a malicious server could send an infinite file to DoS the bot, so there has to be a limit.
by progval 5y ago
As URLs are provided by random users, a malicious server could send an infinite file to DoS the bot, so there has to be a limit.
- capableweb 5y agoCouldn't you disconnect the steps? First, start loading contents of URL and if it's below N bytes, try to parse the HTML and then query it via whatever library you can use in Python for querying HTML documents.