3 ms·

Wow, this looks awful. The cited sources are questionable at best, and on my first attempt it's already hallucinating. How can you trust something like this? A

Wow, this looks awful. The cited sources are questionable at best, and on my first attempt it's already hallucinating. How can you trust something like this? Also, how do I get it to return documents? I searched for 'RFC 793' with the intend to read the actual RFC. Instead, the chat bot summarized it for me (which is pointless) and doesn't even use the actual RFC as a source? Since this is a "Google bad" thread, I feel obligated to mention Kagi which I've been using for nearly a year and it has significantly improved my search experience (better search quality, great customizability).

hedora 2y ago
I mentioned Kagi in another response, but I love having test queries in these search engine threads. Appending ? to your kagi search will add LLM results like these: https://kagi.com/fastgpt?query=RFC+793 https://kagi.com/fastgpt?query=RFC+793 For me that gives a link to 793, a newer related RFC that wikipedia mentions, and a draft attempted replacement from 2021. (So, three links to standards and one to wikipedia.)
mdp2021 2y ago
> How can you trust something like this You could mistrust it since it is based on an uncorrected hallucinating technology, but use it either in contexts in which it is either very difficult to hallucinate, or just to return link to sources containing the sought information - as a search engine. > how do I get it to return documents Ask it to provide the link to said document, in literal natural language terms - "provide a link to document D123" -, and it will provide the link in a concise NL sentence as the direct output, plus related links in the "references" section.
- bramhaag 2y ago
  > Ask it to provide the link to said document, in literal natural language terms - "provide a link to document D123" -, and it will provide the link in a concise NL sentence as the direct output I dislike this style of "searching", but it does appear to work. I asked "provide a link to the HTML version of RFC 793" and got back the following URL: https://datatracker.ietf.org/doc/html/rfc793[3 https://datatracker.ietf.org/doc/html/rfc793[3. The URL is invalid (for some reason the reference became part of the URL), but it was easily fixable.
  - mdp2021 2y ago
    > I dislike this style of "searching" That's because you are using a tool that is not made to be a search engine interface: there is no "type keywords here to get links to web pages". The field is "what would you reply to this" - very much not the same. If you shouted "RFC 1234" or "Declaration of Independence" or "Walt Withman collected page 3" to a guy in the street, he would not reply with a reference to find the documents, directions as if you shouted "Fifth street towards the end" (and even then). Perplexity is there to give you the references on which it decided to build the reply. > The URL is invalid The one I received was perfect. Stochastics. > the reference became part of the URL Given the format of the return you pasted, it looks more like the NN encoded a figment merging the URL and a "footnote anchor". It may or may not be so, but we could keep the suspect of that class of possibilities alive.
CrankyBear 2y ago
I just did the same search, and it literally said "Sources" with seven different links to the RFC and related documentation.
- bramhaag 2y ago
  I repeated my search just now and got the following list of sources: - https://www.arc-it.net/html/standards/standard1117.html https://www.arc-it.net/html/standards/standard1117.html - https://www.youtube.com/watch?v=_WH1Z8htjMo https://www.youtube.com/watch?v=_WH1Z8htjMo - https://www.youtube.com/watch?v=7cG-GDpMA4k https://www.youtube.com/watch?v=7cG-GDpMA4k - https://www.tech-invite.com/y05/tinv-ietf-rfc-0793.html https://www.tech-invite.com/y05/tinv-ietf-rfc-0793.html - https://www.maths.tcd.ie/~eoin/index/isi_793rfc81.html https://www.maths.tcd.ie/~eoin/index/isi_793rfc81.html This list is different from my previous search yesterday, so I don't doubt that in your case you got different sources than me. It does look like the sources include mirrors of the RFC hosted by arc-it.net and tech-invite.com. I do not recognize these domains and would much prefer to see an ietf.org domain instead (which is also the first Google result in my case).

hedora 2y ago

I mentioned Kagi in another response, but I love having test queries in these search engine threads. Appending ? to your kagi search will add LLM results like these: https://kagi.com/fastgpt?query=RFC+793 https://kagi.com/fastgpt?query=RFC+793 For me that gives a link to 793, a newer related RFC that wikipedia mentions, and a draft attempted replacement from 2021. (So, three links to standards and one to wikipedia.)

mdp2021 2y ago

> How can you trust something like this You could mistrust it since it is based on an uncorrected hallucinating technology, but use it either in contexts in which it is either very difficult to hallucinate, or just to return link to sources containing the sought information - as a search engine. > how do I get it to return documents Ask it to provide the link to said document, in literal natural language terms - "provide a link to document D123" -, and it will provide the link in a concise NL sentence as the direct output, plus related links in the "references" section.

bramhaag 2y ago
> Ask it to provide the link to said document, in literal natural language terms - "provide a link to document D123" -, and it will provide the link in a concise NL sentence as the direct output I dislike this style of "searching", but it does appear to work. I asked "provide a link to the HTML version of RFC 793" and got back the following URL: https://datatracker.ietf.org/doc/html/rfc793[3 https://datatracker.ietf.org/doc/html/rfc793[3. The URL is invalid (for some reason the reference became part of the URL), but it was easily fixable.
- mdp2021 2y ago
  > I dislike this style of "searching" That's because you are using a tool that is not made to be a search engine interface: there is no "type keywords here to get links to web pages". The field is "what would you reply to this" - very much not the same. If you shouted "RFC 1234" or "Declaration of Independence" or "Walt Withman collected page 3" to a guy in the street, he would not reply with a reference to find the documents, directions as if you shouted "Fifth street towards the end" (and even then). Perplexity is there to give you the references on which it decided to build the reply. > The URL is invalid The one I received was perfect. Stochastics. > the reference became part of the URL Given the format of the return you pasted, it looks more like the NN encoded a figment merging the URL and a "footnote anchor". It may or may not be so, but we could keep the suspect of that class of possibilities alive.

CrankyBear 2y ago

I just did the same search, and it literally said "Sources" with seven different links to the RFC and related documentation.

bramhaag 2y ago
I repeated my search just now and got the following list of sources: - https://www.arc-it.net/html/standards/standard1117.html https://www.arc-it.net/html/standards/standard1117.html - https://www.youtube.com/watch?v=_WH1Z8htjMo https://www.youtube.com/watch?v=_WH1Z8htjMo - https://www.youtube.com/watch?v=7cG-GDpMA4k https://www.youtube.com/watch?v=7cG-GDpMA4k - https://www.tech-invite.com/y05/tinv-ietf-rfc-0793.html https://www.tech-invite.com/y05/tinv-ietf-rfc-0793.html - https://www.maths.tcd.ie/~eoin/index/isi_793rfc81.html https://www.maths.tcd.ie/~eoin/index/isi_793rfc81.html This list is different from my previous search yesterday, so I don't doubt that in your case you got different sources than me. It does look like the sources include mirrors of the RFC hosted by arc-it.net and tech-invite.com. I do not recognize these domains and would much prefer to see an ietf.org domain instead (which is also the first Google result in my case).