3 ms·

> But with the replication crisis in many sciences and political meddling abound, I‘d be wary of any kind of critical information, especially on Wikipedia. Of c

> But with the replication crisis in many sciences and political meddling abound, I‘d be wary of any kind of critical information, especially on Wikipedia. Of course, transformer models have the same issues I think this is a bad comparison. The issues OP described are not limited to tainted training data. Being critical of Wikipedia and critical of science is not equal to treating text as a black box looking for the most fitting continuation. LLMs are not fact machines. When you equate this kind of error with the scientific replication crisis or generally a crtique of scientific methods and political influence (what about commercial?), I don't think this demonstrates critical thinking.

manmal 3y ago
Can you point to where in my comment I asserted that hallucinations are equal to other sources of errors, or even compared them? Because I tend to agree with you. I tried to point out that info found externally always has to be cross-checked, no matter where it originates.
- moritzwarhier 3y ago
  I meant the sentence I quoted, or the part > [...] especially on Wikipedia. Of course, transformer models have the same issues I agree that one should be critical of all sources, including Wikipedia. I don't agree that GPT is an information source comparable to Wikipedia or scientific studies. Both can be wrong, biased or incorrect though. In other words, I would never consider the usage of any LLM worth to cross-check information found elsewhere, as opposed to e.g. Wikipedia. Which does not mean that I consider the info compiled on Wikipedia always as trsutworthy. Then again, I wouldn't use an LLM to cross-check Wikipedia. They're not equals. Sorry if I misunderstood anything in your comment.
  - manmal 3y ago
    Using Wikipedia to cross-check can be dangerous since it often is the source of the content you‘re trying to verify.
    - moritzwarhier 3y ago
      This is surely true and can be a danger in many cases. Still, Wikipedia is curated by humans and in most cases I would argue that articles improve over time, errors get corrected etc. I'm not arguing that Wikipedia is a good primary source, I still think it is very suitable as a point of entry if you want to cross-check information from other sources. A user-editable encyclopedia is not perfect, but it is not comparable to automatically generated text that has no regard for correctness, that only tries to fit its training data and prompt. For example, I wouldn't trust GPT when asking for the height of some building in my city. I would consider it likely though that it confidently gives ke a wrong number. Without all the RLHF training to refuse "best guess" answers, situation would be even more bleak.

manmal 3y ago

Can you point to where in my comment I asserted that hallucinations are equal to other sources of errors, or even compared them? Because I tend to agree with you. I tried to point out that info found externally always has to be cross-checked, no matter where it originates.

moritzwarhier 3y ago
I meant the sentence I quoted, or the part > [...] especially on Wikipedia. Of course, transformer models have the same issues I agree that one should be critical of all sources, including Wikipedia. I don't agree that GPT is an information source comparable to Wikipedia or scientific studies. Both can be wrong, biased or incorrect though. In other words, I would never consider the usage of any LLM worth to cross-check information found elsewhere, as opposed to e.g. Wikipedia. Which does not mean that I consider the info compiled on Wikipedia always as trsutworthy. Then again, I wouldn't use an LLM to cross-check Wikipedia. They're not equals. Sorry if I misunderstood anything in your comment.
- manmal 3y ago
  Using Wikipedia to cross-check can be dangerous since it often is the source of the content you‘re trying to verify.
  - moritzwarhier 3y ago
    This is surely true and can be a danger in many cases. Still, Wikipedia is curated by humans and in most cases I would argue that articles improve over time, errors get corrected etc. I'm not arguing that Wikipedia is a good primary source, I still think it is very suitable as a point of entry if you want to cross-check information from other sources. A user-editable encyclopedia is not perfect, but it is not comparable to automatically generated text that has no regard for correctness, that only tries to fit its training data and prompt. For example, I wouldn't trust GPT when asking for the height of some building in my city. I would consider it likely though that it confidently gives ke a wrong number. Without all the RLHF training to refuse "best guess" answers, situation would be even more bleak.