4 ms·
Author of SingleFile here, Thanks for your feedback! Feel free to contact me or create posts on Github about these bugs, I will do my best to fix them.
by gildas 6y ago
Author of SingleFile here, Thanks for your feedback! Feel free to contact me or create posts on Github about these bugs, I will do my best to fix them.
- gwern 6y agoI haven't mentioned it because logs are full of garbage from crawlers even without any bugs (there's a shocking amount of malformed, malicious, or just crazy bugs - I think there must be crawlers which use n-grams or something to predict possible URLs to speculatively fetch), and I haven't tracked down whether it's anything to do with SingleFile or just the crazy old Internet being crazy. At some point I'll have to dig into it more. It's probably something simple like "lots of websites encode addresses with absolute paths, so bots follow those; SingleFile ought to rewrite absolute paths to point to the original domain."
- gildas 6y agoThanks again for your comments. I also own a SaaS related to SEO and I can confirm what you see. Some bots request URLs that don't exist by trying to guess them. One the other hand, I also often see errors in links where the HTML contains for example <a href="www.example.com/another-page"> and is resolved as "https://www.example.com/www.example.com/another-page" https://www.example.com/www.example.com/another-page". FYI, note that SingleFile is supposed to resolve all the links and that this should be reliable since it delegates this resolution to the browser.
- gildas 6y agoI'm also realizing your issue could be related to some <noscript> tags which are not processed by SingleFile and could contain relative URLs. I should maybe add an option to remove them.