4 ms·

Early wikipedia was flawed but c'mon, hoping that a probabilistic text generator happens to string together a series of correct statements is a fundamentally un

Early wikipedia was flawed but c'mon, hoping that a probabilistic text generator happens to string together a series of correct statements is a fundamentally unserious way to gather information

danielbln 3y ago
Did you know that those text generators can invoke tools like web search, retrieval, and more to pull in external truth?
- Capricorn2481 3y ago
  Using chatGPT's web search is the worst of both worlds. You get all the inefficiency of google with the potential of your text generation fucking up what you searched.
  - hack_edu 3y ago
    ChatGPT's search is powered by Bing.
    - Capricorn2481 3y ago
      Even worse
- ifyoubuildit 3y ago
  This actually has made me less happy with chatgpt. If I wanted to google something (or bing, whatever), I would have done that. A major draw for me has been that chatgpt was providing a much better experience than search engines. Now it sometimes feels like a fancy lmgtfy.
  - RugnirViking 3y ago
    use chatgpt classic. It's faster, too. It's my default.
    - namaria 3y ago
      > use chatgpt classic We're really speed running development cycles nowadays aren't we?
    - ifyoubuildit 3y ago
      Awesome suggestion. I just asked a question that chatgpt failed to answer because it tried to bing it. Finally dug through the menu and found chatgpt classic and it answered it just fine (verified myself).
- wolverine876 3y ago
  > tools like web search, retrieval, and more to pull in external truth When did those tools start outputting truth?
olddustytrail 3y ago
And yet it works. I asked Google Bard five multiple choice questions, each with four options and it got them all correct. I'm sure you can figure out the odds of that happening by chance. It makes no sense to say something can't work when it clearly and obviously does. Most humans would do worse.
- jstarfish 3y ago
  Nothing about the LLM experience is deterministic, so these anecdotal experiments mean nothing. Your experiment led the witness by feeding it possible answers. Small wonder it got them all right. Try this: give it four wrong answers for each question and see how it fares. In my experience it will pick one and convincingly rationalize why it's right, unless you question it, at which point you're leading it again. Anecdotes are so worthless, in fact, here's mine-- I asked Azure's GPT for Powershell help. After seven regens in which it tried to include a different fictional library, I gave up. So which of us had the "real" LLM experience? These things are storytellers, not teachers. Sometimes it gets it right. Maybe most of the time. It's convincing enough that unless you're an expert, you'll never guess when it's wrong, and the lies are bespoke for every user so there's never going to be an errata page to document its failures. It will always appear reliable.
  - dinvlad 3y ago
    > These things are storytellers, not teachers. I really liked how a recent paper from DeepMind put it - LLMs are just role-playing: https://arxiv.org/abs/2305.16367 https://arxiv.org/abs/2305.16367. This explains so much.
  - olddustytrail 3y ago
    > Your experiment led the witness by feeding it possible answers. Small wonder it got them all right. Are you claiming you've always scored 100% on every multiple choice test because you've been "fed the answers"? What kind of dumb response is that?
    - thfuran 3y ago
airstrike 3y ago
This is such a tiring comment. Might I suggest you actually try GPT-4? It will do amazingly well at most tasks you throw at it, especially if you're decent enough at the task to course-correct it.

Early wikipedia was flawed but c'mon, hoping that a probabilistic text generator happens to string together a series of correct statements is a fundamentally unserious way to gather information

danielbln 3y ago

Did you know that those text generators can invoke tools like web search, retrieval, and more to pull in external truth?

Capricorn2481 3y ago
Using chatGPT's web search is the worst of both worlds. You get all the inefficiency of google with the potential of your text generation fucking up what you searched.
- hack_edu 3y ago
  ChatGPT's search is powered by Bing.
  - Capricorn2481 3y ago
    Even worse
ifyoubuildit 3y ago
This actually has made me less happy with chatgpt. If I wanted to google something (or bing, whatever), I would have done that. A major draw for me has been that chatgpt was providing a much better experience than search engines. Now it sometimes feels like a fancy lmgtfy.
- RugnirViking 3y ago
  use chatgpt classic. It's faster, too. It's my default.
  - namaria 3y ago
    > use chatgpt classic We're really speed running development cycles nowadays aren't we?
  - ifyoubuildit 3y ago
    Awesome suggestion. I just asked a question that chatgpt failed to answer because it tried to bing it. Finally dug through the menu and found chatgpt classic and it answered it just fine (verified myself).
wolverine876 3y ago
> tools like web search, retrieval, and more to pull in external truth When did those tools start outputting truth?

olddustytrail 3y ago

And yet it works. I asked Google Bard five multiple choice questions, each with four options and it got them all correct. I'm sure you can figure out the odds of that happening by chance. It makes no sense to say something can't work when it clearly and obviously does. Most humans would do worse.

jstarfish 3y ago
Nothing about the LLM experience is deterministic, so these anecdotal experiments mean nothing. Your experiment led the witness by feeding it possible answers. Small wonder it got them all right. Try this: give it four wrong answers for each question and see how it fares. In my experience it will pick one and convincingly rationalize why it's right, unless you question it, at which point you're leading it again. Anecdotes are so worthless, in fact, here's mine-- I asked Azure's GPT for Powershell help. After seven regens in which it tried to include a different fictional library, I gave up. So which of us had the "real" LLM experience? These things are storytellers, not teachers. Sometimes it gets it right. Maybe most of the time. It's convincing enough that unless you're an expert, you'll never guess when it's wrong, and the lies are bespoke for every user so there's never going to be an errata page to document its failures. It will always appear reliable.
- dinvlad 3y ago
  > These things are storytellers, not teachers. I really liked how a recent paper from DeepMind put it - LLMs are just role-playing: https://arxiv.org/abs/2305.16367 https://arxiv.org/abs/2305.16367. This explains so much.
- olddustytrail 3y ago
  > Your experiment led the witness by feeding it possible answers. Small wonder it got them all right. Are you claiming you've always scored 100% on every multiple choice test because you've been "fed the answers"? What kind of dumb response is that?
  - thfuran 3y ago

airstrike 3y ago

This is such a tiring comment. Might I suggest you actually try GPT-4? It will do amazingly well at most tasks you throw at it, especially if you're decent enough at the task to course-correct it.