3 ms·

I gave this a shot and was shown > We have analysed 68 links on your website and haven't found a single issue. I'd love to see what this actual list of links

I gave this a shot and was shown > We have analysed 68 links on your website and haven't found a single issue. I'd love to see what this actual list of links was, even if they weren't broken. I'm partially curious because I personally use lychee [1] for this purpose, but when I run lychee (even while ignoring several classes of links), it is checking _significantly_ more links. E.g., the output looks like: > 1401 Total (in 7s) 1361 OK 4 Errors 35 Excluded Curious to understand what it is checking that Bernard is not. [1]: https://github.com/lycheeverse/lychee https://github.com/lycheeverse/lychee

sph 3y ago
What is the URL are you testing with? If you want also share the /demo URL and I'll check what is going on. Seems like lychee also extracts links from text, Bernard does not. Only links inside HTML tags are considered, text nodes are ignored.
- lwhsiao 3y ago
  https://bernard.app/demo/33b26fe3-6c67-4338-a6cd-dd4e9204fbfe https://bernard.app/demo/33b26fe3-6c67-4338-a6cd-dd4e9204fbf...
- lwhsiao 3y ago
  I also suspect a different is when I invoke lychee, I'm running it on ALL of the *.html files output by my static site generator. Perhaps Bernard is only testing a single page? If I only run lychee on the same single page, the numbers are closer: > 76 Total (in 1s) 73 OK 0 Errors 2 Excluded
  - sph 3y ago
    No, it follows every link recursively, as long as they are part of the same domain. So, if you add example.com, the crawl starts at / and all links found therein are added to the queue, and so on. For convenience www.example.com is also considered within crawl scope (and viceversa). Everything else is treated as an external link, so tested but not parsed. Is it possible that your links are hosted on another subdomain than the one you have entered? I'll do a deeper dive into this issue tomorrow.
    - schemescape 3y ago
      Maybe not all pages are reachable from the root.
    - lwhsiao 3y ago
      Cool! > I'll do a deeper dive into this issue tomorrow. FWIW, I don't think I'd go as far as calling this an issue, I suspect both Bernard and Lychee are doing different, but reasonable things. No need to spend much time on my account, and cheers on launching a useful project!
      - sph 3y ago
        There's why Bernard isn't finding as many links: a sample one from /blog <a href="https://luke.hsiao.dev/blog/2023-workstation-build/">My 2023 Workstation Build</a> I have never seen forward slashes encoded that way. These were simply not recognised. Added to the todo list.
        lwhsiao 3y ago
        Huh, me neither. Not an intentional encoding on my part. Looks like I have a bug to fix as well!

sph 3y ago

What is the URL are you testing with? If you want also share the /demo URL and I'll check what is going on. Seems like lychee also extracts links from text, Bernard does not. Only links inside HTML tags are considered, text nodes are ignored.

lwhsiao 3y ago
https://bernard.app/demo/33b26fe3-6c67-4338-a6cd-dd4e9204fbfe https://bernard.app/demo/33b26fe3-6c67-4338-a6cd-dd4e9204fbf...
lwhsiao 3y ago
I also suspect a different is when I invoke lychee, I'm running it on ALL of the *.html files output by my static site generator. Perhaps Bernard is only testing a single page? If I only run lychee on the same single page, the numbers are closer: > 76 Total (in 1s) 73 OK 0 Errors 2 Excluded
- sph 3y ago
  No, it follows every link recursively, as long as they are part of the same domain. So, if you add example.com, the crawl starts at / and all links found therein are added to the queue, and so on. For convenience www.example.com is also considered within crawl scope (and viceversa). Everything else is treated as an external link, so tested but not parsed. Is it possible that your links are hosted on another subdomain than the one you have entered? I'll do a deeper dive into this issue tomorrow.
  - schemescape 3y ago
    Maybe not all pages are reachable from the root.
  - lwhsiao 3y ago
    Cool! > I'll do a deeper dive into this issue tomorrow. FWIW, I don't think I'd go as far as calling this an issue, I suspect both Bernard and Lychee are doing different, but reasonable things. No need to spend much time on my account, and cheers on launching a useful project!
    - sph 3y ago
      There's why Bernard isn't finding as many links: a sample one from /blog <a href="https://luke.hsiao.dev/blog/2023-workstation-build/">My 2023 Workstation Build</a> I have never seen forward slashes encoded that way. These were simply not recognised. Added to the todo list.
      - lwhsiao 3y ago
        Huh, me neither. Not an intentional encoding on my part. Looks like I have a bug to fix as well!