4 ms·

I wonder how difficult it would be to make a website that, instead of serving 404's, a model does a semantic search using the nonexistent URL to come up with a

I wonder how difficult it would be to make a website that, instead of serving 404's, a model does a semantic search using the nonexistent URL to come up with a URL that actually does exist and most closely matches the "intent" of the invalid one.

kalleboo 2y ago
How about just using an LLM to generate a page that matches what the URL appears to try to link to? (not being serious here)
- throwaway4aday 2y ago
  That's called Websim my friend and it's a heck of a lot of fun https://websim.ai https://websim.ai This is what it generated for one of the non-existent articles https://websim.ai/c/P32MdBI15Ytxwwlth https://websim.ai/c/P32MdBI15Ytxwwlth
gwbas1c 2y ago
Not too hard, if the site already has some kind of a search engine, it can feed the URL to the search engine. The safe thing to do is return a 404 with a few suggested links. The problem is that "guessing" the intent of the URL will have unintended consequences. A URL will no longer be guaranteed to point to what you think it points to. This fundamentally breaks the assumption of what a URL is. How could this happen? 1: Site creates article foo.com/chocolate_bars 2: Someone accidentally links to foo.com/cocoa_bars, which serves foo.com/chocolate_bars 3: Site creates article foo.com/coco_bars 4: Now foo.com/cocoa_bars serves foo.com/coco_bars
- AlexandrB 2y ago
  Yes, this sounds like a nightmare. Also pity poor archive.org if they try to archive a web site set up like this - every URL will resolve to a valid page.
- netruk44 2y ago
  My thought was to serve a 307 (or some other redirect HTTP code, clearly I am not a web developer) instead of a 404, not to serve the found content “as” the url that didn’t exist. But agreed this would not be good general practice. It was mostly a thought experiment.
  - gwbas1c 2y ago
    > The safe thing to do is return a 404 with a few suggested links. The 404 communicates to search engines, archive.org, ect, that the link is broken. The human-readable text on the 404 page can contain a link to the correct page. It could even contain a few paragraphs too. Should get close enough to your original idea without too many unintended consequences.

kalleboo 2y ago

How about just using an LLM to generate a page that matches what the URL appears to try to link to? (not being serious here)

throwaway4aday 2y ago
That's called Websim my friend and it's a heck of a lot of fun https://websim.ai https://websim.ai This is what it generated for one of the non-existent articles https://websim.ai/c/P32MdBI15Ytxwwlth https://websim.ai/c/P32MdBI15Ytxwwlth

gwbas1c 2y ago

Not too hard, if the site already has some kind of a search engine, it can feed the URL to the search engine. The safe thing to do is return a 404 with a few suggested links. The problem is that "guessing" the intent of the URL will have unintended consequences. A URL will no longer be guaranteed to point to what you think it points to. This fundamentally breaks the assumption of what a URL is. How could this happen? 1: Site creates article foo.com/chocolate_bars 2: Someone accidentally links to foo.com/cocoa_bars, which serves foo.com/chocolate_bars 3: Site creates article foo.com/coco_bars 4: Now foo.com/cocoa_bars serves foo.com/coco_bars

AlexandrB 2y ago
Yes, this sounds like a nightmare. Also pity poor archive.org if they try to archive a web site set up like this - every URL will resolve to a valid page.
netruk44 2y ago
My thought was to serve a 307 (or some other redirect HTTP code, clearly I am not a web developer) instead of a 404, not to serve the found content “as” the url that didn’t exist. But agreed this would not be good general practice. It was mostly a thought experiment.
- gwbas1c 2y ago
  > The safe thing to do is return a 404 with a few suggested links. The 404 communicates to search engines, archive.org, ect, that the link is broken. The human-readable text on the 404 page can contain a link to the correct page. It could even contain a few paragraphs too. Should get close enough to your original idea without too many unintended consequences.