3 ms·
Playing with the code, I think there's actually three problems here: 1) I don't think doesn't recognise relative paths in the tag, which HN uses and is entirel
by lewiscollard 5y ago
Playing with the code, I think there's actually three problems here:
1) I don't think doesn't recognise relative paths in the tag, which HN uses and is entirely valid, e.g.:
<link rel="alternate" type="application/rss+xml" title="RSS" href="rss">
2) The code does HTTP requests in more than one way (aio, `get_response_content`, `get_request`, and whatever feedparser uses internally), and only one of those sets the User-Agent header properly, which is probably causing it to get flagged as a not-nice bot
3) ...and once you fix those problems you will likely get a 503 for requesting the feed too often during testing :)
[Edit: and no, I'm not _just_ complaining; expect some pull requests over the next few days.]