4 ms·

Actually, not very easy as the LLMs don't give you sources and literally make things up by design.

jsheard 4y ago
They do give sources if you ask for them... but have a habit of inventing plausible sounding but completely fake sources. https://news.ycombinator.com/item?id=33841672 https://news.ycombinator.com/item?id=33841672 That example is a few months old but OpenAI doesn't seem to have made much progress here, if you ask ChatGPT for "a list of academic papers about X" and it will nearly always confidently churn out a list of 5-10 papers that don't exist. Amusingly if you ask it for papers about an absurd premise it will sometimes call out the absurdity and say there are probably no papers on that subject, but then offer a more plausible variation on that premise and trip up at the last hurdle by inventing all the examples on the subject it supposedly thinks is more likely to really exist.
- gibsonf1 4y ago
  Right, they just make all things up by design, no matter what you ask about. There is a statistical chance that a pattern of words output may correspond to fact but just as a good a chance not. The LLM's literally know nothing about the world, its just statistical word pattern output.
  - mrguyorama 4y ago
    Lies and falsehoods are just as valid sentences as the truth.
- aledalgrande 4y ago
  I got so many links to github repos which didn't exist lol
  - visarga 4y ago
    Somebody went as far as contacting the author of a paper based on chatGPT suggestion. Of course the author was pretty sure he never heard of that paper.
- jerf 4y ago
  "They do give sources if you ask for them... but have a habit of inventing plausible sounding but completely fake sources." Basically, the way you want to think about it is, no, they can not give sources. That information is not in their neural net. It can't be. There isn't anywhere to encode or represent it. What they can do is make a guess what a source might look like, but even if they are right, it is only because they happened to guess correctly, not because they knew the source. They don't. They can't. It isn't that "they give sources but they might be wrong", it is "they will make up plausible-sounding sources if you ask just as they'll make up plausible-sounding anything else you ask for, and there's a small chance they'll be right by luck". For more normal factual-type questions part of the reason they are useful is that there's a good chance they'll be right by what is still essentially luck, but for sources in particular there's a particularly small chance, by the nature of the thing.
  - sam0x17 4y ago
    That said with enough training and training data, the line between "plausible sounding" and "accurate" gets thinner and thinner. This will be especially true as these AI models refine their results based on user interactions. Being right for the wrong reasons becomes less and less relevant the higher the accuracy goes up, and at a certain point, it might get so good that no one cares. Maybe human intelligence is more like that than we're willing to admit ;)
    - jerf 4y ago
      Source identification isn't a space amenable to guessing, no matter how much data you throw at it. Here's an exercise you can try: Cite some information from the next issue of Science to be published. Cite anything you like from it. You can make some plausible stuff up. You could make even more plausible stuff up if you went and scanned over the past few issues first. But without specific knowledge of the contents of the next issue, you aren't going to be able to create real citations. This is what LLMs lack, by their nature. It's not a criticism, it's a description. You can't guess sources. The possibility space is too large, the distribution too pathological, and the criteria for being correct too precise. GPT will never cite sources correctly. Some future AI that uses GPT as a component, but isn't entirely made out of a language model, will be able to, by pulling it out of the non-GPT component. Maybe it'll need to be built as an explicit feature, maybe it won't, only time can tell. But expecting language models to cite sources correctly is not sensible. It's just not a thing they can do.
sometdog 4y ago
That is true, but you can combine LLMs with other tools to get sourcing and more accurate answers overall. Instead of using the LLM to directly answer a question, you can use the question to search for relevant text in a particular knowledgebase, and then use the LLM to summarize those results.
- gibsonf1 4y ago
  If you use the LLM to summarize the results, it makes things up by design. As soon as you introduce the LLM into search, you lose.
  - raincole 4y ago
    I've personally replaced Google with Bing Chat for technical things (like searching for specific API). Does it make things up? Maybe. But in my experience it never happes even once in past whole week (>100 times of searchs). It's not "it happened but you just didn't notice..." if it uses a function call wrong I'd have noticed. My code won't compile. My test won't pass. So far it either gives me 100% correct result, or completely fails. But it doesn't generate "seemingly correct but actually wrong" things even once, unlike ChatGPT.
    - autonomousErwin 4y ago
      I find this the killer application for ChatGPT (at least for now). Answers you can very quickly verify and care little about the sources because a significant no. of answers on Stack Overflow make ChatGPT look modest in confidence comparison
  - throwuwu 4y ago
    Biased much?
- DebtDeflation 4y ago
  >use the question to search for relevant text in a particular knowledgebase, and then use the LLM to summarize those results As a consultant, based on personal experience, I can say that what you wrote above constitutes >90% of current enterprise use cases for ChatGPT. What clients REALLY want to do is be able to take a pre-trained LLM and then train it further on their own corpus of documents, but given limitations around token window size, the above is probably the best way to fake it for now.
mark_l_watson 4y ago
The web demo version of ChatGPT can sometimes, as you say, make things up! I asked it a question about an ancient war in Greece and it stated two facts and then just inventing stuff. However, I would ask you to also consider something: when you supply prompt input text (this is the "context text" in the ChatGPT API calls) and then ask questions about the context text, it is very accurate. It also does a very good job when you give it context text from a few different sources, and it integrates them nicely. It is more efficient to use embeddings for context text, as Llama-Index does.
- nico 4y ago
  Do you have shareable links to good tutorials about using embeddings with ChatGPT for this case?
  - mark_l_watson 4y ago
    I would start with https://gpt-index.readthedocs.io/en/latest/ https://gpt-index.readthedocs.io/en/latest/ and https://langchain.readthedocs.io/en/latest/ https://langchain.readthedocs.io/en/latest/
    - nico 4y ago
      Awesome, thank you!
politician 4y ago
Only about 10% of the links I ask for don't resolve.
deleted 4y ago
[deleted]
joshspankit 4y ago
For AIs to replace search, they will end up giving sources. Search for users is “how can I find the correct answer” but the part of search that runs the capitalist machine is for experts to say “how can I prove my expertise while I monetize it?” and that means those experts need to be cited.

They do give sources if you ask for them... but have a habit of inventing plausible sounding but completely fake sources. https://news.ycombinator.com/item?id=33841672 https://news.ycombinator.com/item?id=33841672 That example is a few months old but OpenAI doesn't seem to have made much progress here, if you ask ChatGPT for "a list of academic papers about X" and it will nearly always confidently churn out a list of 5-10 papers that don't exist. Amusingly if you ask it for papers about an absurd premise it will sometimes call out the absurdity and say there are probably no papers on that subject, but then offer a more plausible variation on that premise and trip up at the last hurdle by inventing all the examples on the subject it supposedly thinks is more likely to really exist.

gibsonf1 4y ago
Right, they just make all things up by design, no matter what you ask about. There is a statistical chance that a pattern of words output may correspond to fact but just as a good a chance not. The LLM's literally know nothing about the world, its just statistical word pattern output.
- mrguyorama 4y ago
  Lies and falsehoods are just as valid sentences as the truth.
aledalgrande 4y ago
I got so many links to github repos which didn't exist lol
- visarga 4y ago
  Somebody went as far as contacting the author of a paper based on chatGPT suggestion. Of course the author was pretty sure he never heard of that paper.
jerf 4y ago
"They do give sources if you ask for them... but have a habit of inventing plausible sounding but completely fake sources." Basically, the way you want to think about it is, no, they can not give sources. That information is not in their neural net. It can't be. There isn't anywhere to encode or represent it. What they can do is make a guess what a source might look like, but even if they are right, it is only because they happened to guess correctly, not because they knew the source. They don't. They can't. It isn't that "they give sources but they might be wrong", it is "they will make up plausible-sounding sources if you ask just as they'll make up plausible-sounding anything else you ask for, and there's a small chance they'll be right by luck". For more normal factual-type questions part of the reason they are useful is that there's a good chance they'll be right by what is still essentially luck, but for sources in particular there's a particularly small chance, by the nature of the thing.
- sam0x17 4y ago
  That said with enough training and training data, the line between "plausible sounding" and "accurate" gets thinner and thinner. This will be especially true as these AI models refine their results based on user interactions. Being right for the wrong reasons becomes less and less relevant the higher the accuracy goes up, and at a certain point, it might get so good that no one cares. Maybe human intelligence is more like that than we're willing to admit ;)
  - jerf 4y ago
    Source identification isn't a space amenable to guessing, no matter how much data you throw at it. Here's an exercise you can try: Cite some information from the next issue of Science to be published. Cite anything you like from it. You can make some plausible stuff up. You could make even more plausible stuff up if you went and scanned over the past few issues first. But without specific knowledge of the contents of the next issue, you aren't going to be able to create real citations. This is what LLMs lack, by their nature. It's not a criticism, it's a description. You can't guess sources. The possibility space is too large, the distribution too pathological, and the criteria for being correct too precise. GPT will never cite sources correctly. Some future AI that uses GPT as a component, but isn't entirely made out of a language model, will be able to, by pulling it out of the non-GPT component. Maybe it'll need to be built as an explicit feature, maybe it won't, only time can tell. But expecting language models to cite sources correctly is not sensible. It's just not a thing they can do.

sometdog 4y ago

That is true, but you can combine LLMs with other tools to get sourcing and more accurate answers overall. Instead of using the LLM to directly answer a question, you can use the question to search for relevant text in a particular knowledgebase, and then use the LLM to summarize those results.

gibsonf1 4y ago
If you use the LLM to summarize the results, it makes things up by design. As soon as you introduce the LLM into search, you lose.
- raincole 4y ago
  I've personally replaced Google with Bing Chat for technical things (like searching for specific API). Does it make things up? Maybe. But in my experience it never happes even once in past whole week (>100 times of searchs). It's not "it happened but you just didn't notice..." if it uses a function call wrong I'd have noticed. My code won't compile. My test won't pass. So far it either gives me 100% correct result, or completely fails. But it doesn't generate "seemingly correct but actually wrong" things even once, unlike ChatGPT.
  - autonomousErwin 4y ago
    I find this the killer application for ChatGPT (at least for now). Answers you can very quickly verify and care little about the sources because a significant no. of answers on Stack Overflow make ChatGPT look modest in confidence comparison
- throwuwu 4y ago
  Biased much?
DebtDeflation 4y ago
>use the question to search for relevant text in a particular knowledgebase, and then use the LLM to summarize those results As a consultant, based on personal experience, I can say that what you wrote above constitutes >90% of current enterprise use cases for ChatGPT. What clients REALLY want to do is be able to take a pre-trained LLM and then train it further on their own corpus of documents, but given limitations around token window size, the above is probably the best way to fake it for now.

mark_l_watson 4y ago

The web demo version of ChatGPT can sometimes, as you say, make things up! I asked it a question about an ancient war in Greece and it stated two facts and then just inventing stuff. However, I would ask you to also consider something: when you supply prompt input text (this is the "context text" in the ChatGPT API calls) and then ask questions about the context text, it is very accurate. It also does a very good job when you give it context text from a few different sources, and it integrates them nicely. It is more efficient to use embeddings for context text, as Llama-Index does.

nico 4y ago
Do you have shareable links to good tutorials about using embeddings with ChatGPT for this case?
- mark_l_watson 4y ago
  I would start with https://gpt-index.readthedocs.io/en/latest/ https://gpt-index.readthedocs.io/en/latest/ and https://langchain.readthedocs.io/en/latest/ https://langchain.readthedocs.io/en/latest/
  - nico 4y ago
    Awesome, thank you!

politician 4y ago

Only about 10% of the links I ask for don't resolve.

deleted 4y ago

[deleted]

joshspankit 4y ago

For AIs to replace search, they will end up giving sources. Search for users is “how can I find the correct answer” but the part of search that runs the capitalist machine is for experts to say “how can I prove my expertise while I monetize it?” and that means those experts need to be cited.