3 ms·

James Wesley Johnston was Chief Executive Officer (CEO) of R.J. Reynolds c. 1989-95, and was CEO of R.J. Reynolds' domestic division c. 1989. > It's not attrib

James Wesley Johnston was Chief Executive Officer (CEO) of R.J. Reynolds c. 1989-95, and was CEO of R.J. Reynolds' domestic division c. 1989. > It's not attributed to any specific reputable source The LLM model doesn't know true/false, fact or fiction. It's just generating likely text from patterns it's learned. In this case, the qualification is accurately reflecting its training. Alix M. Freedman of The Wall Street Journal won the 1996 Pulitzer Prize in 1996 in National Reporting for her coverage of the tobacco industry. Her December 8, 1995 story headlined, "Phillip Morris Memo Likens Nicotine to Cocaine" includes the quote and states it came from written testimony by Johnston given to an April 1994 congressional hearing. The quote is, "cigarette smoking is no more addictive than coffee, tea or Twinkies." I doubt GPT-4 was trained on documents from the congressional record (and OpenAI has declined to elaborate on their training models), and one instance from a Wall Street Journal article, if was even in the training corpus, doesn't give much weight.

chrisjj 3y ago
> The LLM model doesn't know true/false, fact or fiction. That should be immaterial, given it is asked simply to retrieve a fact. > In this case, the qualification is accurately reflecting its training. What training causes it to evade the question by adopting "reputable"?
- JohnFen 3y ago
  > That should be immaterial, given it is asked simply to retrieve a fact. How is the LLM supposed to recognize what a fact is?
  - chrisjj 3y ago
    It doesn't need to. It needs simply to trawl its data for an answer to the direct question. Which should not be too hard ... given even Google web search can do it: https://www.google.com/search?q=Who+said+cigarettes+were+no+more+addictive+than+coffee%2C+tea+or+Twinkies https://www.google.com/search?q=Who+said+cigarettes+were+no+... My point is, the outcome shouldn't be affected by the truth/falsehood of the statement in question ... barring hamfisted “teach LLMs to stick to the truth”.
    - cratermoon 3y ago
      > the outcome shouldn't be affected by the truth/falsehood of the statement in question Epistemology would beg to differ. See, for example, "Is Justified True Belief Knowledge?" https://academic.oup.com/analysis/article-lookup/doi/10.1093/analys/23.6.121 https://academic.oup.com/analysis/article-lookup/doi/10.1093...
      - chrisjj 3y ago
        I don't think it would. Else we'd be able to ask 1==1? but not 1==2? .
        cratermoon 3y ago
        LLMs are notoriously failures at math and symbolic manipulation in general. The model only statistically predicts the token 1==1 is true more often in its corpus and thus has more weight than 1==2. We can say mathematics makes 1==1 true, but LLMs don't know the logic: all they have are signs (tokens) saying 1==1 is true, which happens to be correct, but the existence of the sign (tokens) itself is not proof.

chrisjj 3y ago

> The LLM model doesn't know true/false, fact or fiction. That should be immaterial, given it is asked simply to retrieve a fact. > In this case, the qualification is accurately reflecting its training. What training causes it to evade the question by adopting "reputable"?

JohnFen 3y ago
> That should be immaterial, given it is asked simply to retrieve a fact. How is the LLM supposed to recognize what a fact is?
- chrisjj 3y ago
  It doesn't need to. It needs simply to trawl its data for an answer to the direct question. Which should not be too hard ... given even Google web search can do it: https://www.google.com/search?q=Who+said+cigarettes+were+no+more+addictive+than+coffee%2C+tea+or+Twinkies https://www.google.com/search?q=Who+said+cigarettes+were+no+... My point is, the outcome shouldn't be affected by the truth/falsehood of the statement in question ... barring hamfisted “teach LLMs to stick to the truth”.
  - cratermoon 3y ago
    > the outcome shouldn't be affected by the truth/falsehood of the statement in question Epistemology would beg to differ. See, for example, "Is Justified True Belief Knowledge?" https://academic.oup.com/analysis/article-lookup/doi/10.1093/analys/23.6.121 https://academic.oup.com/analysis/article-lookup/doi/10.1093...
    - chrisjj 3y ago
      I don't think it would. Else we'd be able to ask 1==1? but not 1==2? .
      - cratermoon 3y ago
        LLMs are notoriously failures at math and symbolic manipulation in general. The model only statistically predicts the token 1==1 is true more often in its corpus and thus has more weight than 1==2. We can say mathematics makes 1==1 true, but LLMs don't know the logic: all they have are signs (tokens) saying 1==1 is true, which happens to be correct, but the existence of the sign (tokens) itself is not proof.