5 ms·

Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd bee

Not just scientists, but everyone! My partner recently went a bit nuts writing an article with the help of GPT4. She was very proud of how productive she'd been until I asked if she'd actually searched for the papers GPT4 had referred to. Of course, many of the referred to papers didn't exist...

JeremyBarbosa 2y ago
Sadly it's not even just references, LLMs still hallucinate or at least misrepresent even the most basic of facts. That and the stereotypical GPT-verbage makes it impossible to use for writing anything significant.
- SrslyJosh 2y ago
  > LLMs still hallucinate Keep in mind that there's no difference between what happens inside a model when it "hallucinates" vs. when it generates "correct" output. It's the exact same process.
  - alpinisme 2y ago
    That’s true, but it’s also true of anything else that makes mistakes, including buggy software. When a buggy sorting algorithm produces a bad ordering it’s doing so “with the exact same process” the good ordering is coming from. Ditto for humans and their slips (although tbh I get a little tired of the analogizing of humans and llms…not that the analogies are wrong, but just that we always analogize human minds with the latest technology: wax writing pads through computers)
  - raincole 2y ago
    Uh... yes? I'm not sure why it's some significant insight. Surely when google gives bad results, it's "the same process" as when it gives good results. And when a book gives wrong information, it's the exact same kind of ink as correct information.
    - Terr_ 2y ago
      I think the point is that it's not some kind of bug to find and fix, it's a fundamental risk with the entire approach.
- switchbak 2y ago
  We were already swimming in a world of bullshit prior to the wide availability of these. I'm not sure what the future holds, but I think intelligent people are going to become very skeptical of virtually all information sources. I would imagine there's also a raft of people who will use it as a reason to give up on any search for truth. I still do hold a lot of hope for their eventual capabilities, but I'm also pretty pessimistic on what the direct and Nth order social effects will be.
flatline 2y ago
That is not writing with the help of GPT 4, that is letting it write for you! I can’t imagine doing anything creative and letting a computer source material for me without having reviewed the material first hand, even if it was accurate. Clearly, this is not where everyone’s head is at, and I suspect your wife’s workflow is more the common case. I’ve said from the outset that in academic settings you should be able to cite an AI as a writing assistant, it would clear up a lot of the confusion about its use. If you used it poorly it’s still on you, but at least there’s some transparency by which to judge the work.
- Mathnerd314 2y ago
  I've sort of worked out a workflow. Like say I had to write an essay and take a side for/against something. Then I would ask GPT to write the strongest argument for, and the strongest argument against, telling it to make up whatever sources it wants. Then after reading those, I would have some idea of my own opinions. I would write from scratch but with the GPT for/against pulled up alongside as reference for how to structure the arguments. Then I would put it through GPT again for proofreading and grammar (or just spelling, if there is AI detection software). It is a bit tricky though, there are definitely points that come up with GPT that people would not think of normally. So in that sense it is still distinguishable from writing solely by oneself, but I would argue the GPT-assisted essays are just better writing and more well-rounded.
  - bsenftner 2y ago
    There is a subtle aspect of LLM AIs that is lost to most people: they are trained on the entirety of the Internet. That means whatever topic you ask these LLM AIs, there are multiple instances of that same information with different levels of seriousness and accuracy in their treatment of the subject. For example: if one asks a question using street slang, the answer generated will be generated from training data about your subject, but from online sources that used street slang in their conversation about that issue. Likewise, if you use ordinary language for your question, the generated response will be from ordinary language conversations of your topic. However, if your question concerns any type of formalized knowledge, by asking your question using the formal language of experts in that topic, then the generated AI answer will come from training data that used this same formal expert terms, and are most likely to be correct, because they come from discussions of that subject’s matter experts. Plus, don't use LLMs for fact retrieval, use them as strategy guides. They really excel as strategy advisors.
EGreg 2y ago
Hmm. In the future the AI in nefarious hands can retroactively make the papers first, and get them past the censors. Just make up a lot of bullshit and then it’s turtles all the way down lmao
- Terr_ 2y ago
  I'm imagining how much easier it would have made work for the Ministry of Truth in 1984.
deleted 2y ago
[deleted]
colechristensen 2y ago
GPT-whatever can’t do sources. I was trying to use it as a research tool and it hallucinated 95% of the references I asked for (not a made up percentage, I counted) Ironically the one real source turned out to be quite useful.

JeremyBarbosa 2y ago

Sadly it's not even just references, LLMs still hallucinate or at least misrepresent even the most basic of facts. That and the stereotypical GPT-verbage makes it impossible to use for writing anything significant.

SrslyJosh 2y ago
> LLMs still hallucinate Keep in mind that there's no difference between what happens inside a model when it "hallucinates" vs. when it generates "correct" output. It's the exact same process.
- alpinisme 2y ago
  That’s true, but it’s also true of anything else that makes mistakes, including buggy software. When a buggy sorting algorithm produces a bad ordering it’s doing so “with the exact same process” the good ordering is coming from. Ditto for humans and their slips (although tbh I get a little tired of the analogizing of humans and llms…not that the analogies are wrong, but just that we always analogize human minds with the latest technology: wax writing pads through computers)
- raincole 2y ago
  Uh... yes? I'm not sure why it's some significant insight. Surely when google gives bad results, it's "the same process" as when it gives good results. And when a book gives wrong information, it's the exact same kind of ink as correct information.
  - Terr_ 2y ago
    I think the point is that it's not some kind of bug to find and fix, it's a fundamental risk with the entire approach.
switchbak 2y ago
We were already swimming in a world of bullshit prior to the wide availability of these. I'm not sure what the future holds, but I think intelligent people are going to become very skeptical of virtually all information sources. I would imagine there's also a raft of people who will use it as a reason to give up on any search for truth. I still do hold a lot of hope for their eventual capabilities, but I'm also pretty pessimistic on what the direct and Nth order social effects will be.

flatline 2y ago

That is not writing with the help of GPT 4, that is letting it write for you! I can’t imagine doing anything creative and letting a computer source material for me without having reviewed the material first hand, even if it was accurate. Clearly, this is not where everyone’s head is at, and I suspect your wife’s workflow is more the common case. I’ve said from the outset that in academic settings you should be able to cite an AI as a writing assistant, it would clear up a lot of the confusion about its use. If you used it poorly it’s still on you, but at least there’s some transparency by which to judge the work.

Mathnerd314 2y ago
I've sort of worked out a workflow. Like say I had to write an essay and take a side for/against something. Then I would ask GPT to write the strongest argument for, and the strongest argument against, telling it to make up whatever sources it wants. Then after reading those, I would have some idea of my own opinions. I would write from scratch but with the GPT for/against pulled up alongside as reference for how to structure the arguments. Then I would put it through GPT again for proofreading and grammar (or just spelling, if there is AI detection software). It is a bit tricky though, there are definitely points that come up with GPT that people would not think of normally. So in that sense it is still distinguishable from writing solely by oneself, but I would argue the GPT-assisted essays are just better writing and more well-rounded.
- bsenftner 2y ago
  There is a subtle aspect of LLM AIs that is lost to most people: they are trained on the entirety of the Internet. That means whatever topic you ask these LLM AIs, there are multiple instances of that same information with different levels of seriousness and accuracy in their treatment of the subject. For example: if one asks a question using street slang, the answer generated will be generated from training data about your subject, but from online sources that used street slang in their conversation about that issue. Likewise, if you use ordinary language for your question, the generated response will be from ordinary language conversations of your topic. However, if your question concerns any type of formalized knowledge, by asking your question using the formal language of experts in that topic, then the generated AI answer will come from training data that used this same formal expert terms, and are most likely to be correct, because they come from discussions of that subject’s matter experts. Plus, don't use LLMs for fact retrieval, use them as strategy guides. They really excel as strategy advisors.

EGreg 2y ago

Hmm. In the future the AI in nefarious hands can retroactively make the papers first, and get them past the censors. Just make up a lot of bullshit and then it’s turtles all the way down lmao

Terr_ 2y ago
I'm imagining how much easier it would have made work for the Ministry of Truth in 1984.

deleted 2y ago

[deleted]

colechristensen 2y ago

GPT-whatever can’t do sources. I was trying to use it as a research tool and it hallucinated 95% of the references I asked for (not a made up percentage, I counted) Ironically the one real source turned out to be quite useful.