2 ms·
No, it follows every link recursively, as long as they are part of the same domain. So, if you add example.com, the crawl starts at / and all links found there
by sph 3y ago
No, it follows every link recursively, as long as they are part of the same domain.
So, if you add example.com, the crawl starts at / and all links found therein are added to the queue, and so on. For convenience www.example.com is also considered within crawl scope (and viceversa). Everything else is treated as an external link, so tested but not parsed.
Is it possible that your links are hosted on another subdomain than the one you have entered? I'll do a deeper dive into this issue tomorrow.
- schemescape 3y agoMaybe not all pages are reachable from the root.
- lwhsiao 3y agoCool! > I'll do a deeper dive into this issue tomorrow. FWIW, I don't think I'd go as far as calling this an issue, I suspect both Bernard and Lychee are doing different, but reasonable things. No need to spend much time on my account, and cheers on launching a useful project!
- sph 3y agoThere's why Bernard isn't finding as many links: a sample one from /blog <a href="https://luke.hsiao.dev/blog/2023-workstation-build/">My 2023 Workstation Build</a> I have never seen forward slashes encoded that way. These were simply not recognised. Added to the todo list.
- lwhsiao 3y agoHuh, me neither. Not an intentional encoding on my part. Looks like I have a bug to fix as well!