4 ms·

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here an

by nf3 3y ago

Is it possible that even us developers and hackers, who should know better, have fallen for the hugely exaggerated promise of AI? I read the comments on here and it's as if people really expect to be having an intelligent conversation with a rational being. A kind reminder people: it's just a machine, the only thing that might be intelligent about it is the designs of its makers, and even then I'm not so sure... People are talking about ChatGPT hallucinating. I think it's rather us humans who are.

makin 3y ago
Aren't we also machines? If we make theories that discount our own intelligence that's just throwing our hands up and giving up. Maybe I would agree with you if LLMs weren't already embedded in a lot of useful products. I myself measurably save a lot of time using ChatGPT in such "unintelligent conversations". It's intelligent in the ways that matter for a tool.
- nottorp 3y ago
  IMO you're using the wrong term or have a low bar for 'intelligence'. It's reasonably good at reproducing text and mixing it. Like a lazy high school student that has to write an essay but won't just download one. Instead they'll mix and match stuff off several online sources so it seems original although it isn't. That may be intelligence but it doesn't justify the religious like tone some people use when talking about LLMs.
  - joeythedolphin 3y ago
    I don't know, it's more then this. I ask ChatGPT to teach me all about Hinduism, the siva purana, ancient indian customs, etc., and it is an incredible tutor. I can ask it to analyze things from a Jungian perspective. I can ask it to consider ideas from a non-typical perspective, etc. It is pretty incredible, it is more then a word gargler.
    - nottorp 3y ago
      "teach you all". So you don't know enough to evaluate the answers. Or where to find the original sources it quoted or mixed.
      - throwuwu 3y ago
        Original sources aren’t necessarily infallible either, but that’s beside the point, asking ChatGPT needs to be viewed as asking a person to recount something from memory. It hasn’t been reviewed or edited so it’s like a first draft rather than a published document. That’s the error that people tend to make since they treat it like a search engine instead of a conversation.
        nottorp 3y ago
        > they treat it like a search engine instead of a conversation. That's because web searches are so polluted that for a lot of purposes even ChatGPT 3.5 is better.
      - joeythedolphin 3y ago
        Um yes I do? Lol why do you just assume this?
    - EchoChamberMan 3y ago
      It is, unquestionably, from the math, a "word gargler" as you put it. https://xkcd.com/1838/ https://xkcd.com/1838/
      - joeythedolphin 3y ago
        No it's a digital neural network expressing it's inputs and outputs in language.
        deleted 3y ago
        [deleted]
        3y ago
rsynnott 3y ago
> People are talking about ChatGPT hallucinating It's really unfortunate that some much of the terminology around LLMs is so anthropomorphic; it's extremely misleading. (People sometimes suggest replacing this one with confabulating, but I'm not really sure that's much better)
ggambetta 3y ago
Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow. I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation". It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion. I know I'm part of a minority of people who think this way :( The last few months have felt like I'm taking crazy pills :(
- EchoChamberMan 3y ago
  Sales people, and company owners, have a vested interest in pushing the narrative that the illusion is actually magic.
- crote 3y ago
  I believe that for a large number of applications the distinction between "knowing something" and "containing knowledge" doesn't really matter. In 2010s Google I could type in a query, and it would redirect me to a human-written answer on something like StackOverflow with hopefully the right answer; in 2024 ChatGPT you can type in a query and it'll mash together some relevant human-written snippets into something which is hopefully coherent, relevant, and correct. Sure, it's "just" a Markov chain generator on steroids, but it's still giving you useful output. I do agree that it is a bit worrying to what extent people are anthropomorphizing it, and treating it like something with an actual opinion or an unquestionable source-of-truth. The "I asked ChatGPT" comments should be treated the same as "I put it into Google and this was the first result": it's just line noise and doesn't add anything to the conversation. If you don't have anything to add yourself, why comment at all?
- __loam 3y ago
  You're not alone
- logicprog 3y ago
  I feel the same way. It's frustrating feeling like I'm one of the few people that can see that the emperor has no clothes, and nice to see the occasional person like you that pushes back on this cargo cult fanaticism.
__loam 3y ago
I've been ranting about this for months. The underlying shape of these things is inherently an approximation. We're just guessing but some are acting like we've built the godhead.

by nf3 3y ago

makin 3y ago

Aren't we also machines? If we make theories that discount our own intelligence that's just throwing our hands up and giving up. Maybe I would agree with you if LLMs weren't already embedded in a lot of useful products. I myself measurably save a lot of time using ChatGPT in such "unintelligent conversations". It's intelligent in the ways that matter for a tool.

nottorp 3y ago
IMO you're using the wrong term or have a low bar for 'intelligence'. It's reasonably good at reproducing text and mixing it. Like a lazy high school student that has to write an essay but won't just download one. Instead they'll mix and match stuff off several online sources so it seems original although it isn't. That may be intelligence but it doesn't justify the religious like tone some people use when talking about LLMs.
- joeythedolphin 3y ago
  I don't know, it's more then this. I ask ChatGPT to teach me all about Hinduism, the siva purana, ancient indian customs, etc., and it is an incredible tutor. I can ask it to analyze things from a Jungian perspective. I can ask it to consider ideas from a non-typical perspective, etc. It is pretty incredible, it is more then a word gargler.
  - nottorp 3y ago
    "teach you all". So you don't know enough to evaluate the answers. Or where to find the original sources it quoted or mixed.
    - throwuwu 3y ago
      Original sources aren’t necessarily infallible either, but that’s beside the point, asking ChatGPT needs to be viewed as asking a person to recount something from memory. It hasn’t been reviewed or edited so it’s like a first draft rather than a published document. That’s the error that people tend to make since they treat it like a search engine instead of a conversation.
      - nottorp 3y ago
        > they treat it like a search engine instead of a conversation. That's because web searches are so polluted that for a lot of purposes even ChatGPT 3.5 is better.
    - joeythedolphin 3y ago
      Um yes I do? Lol why do you just assume this?
  - EchoChamberMan 3y ago
    It is, unquestionably, from the math, a "word gargler" as you put it. https://xkcd.com/1838/ https://xkcd.com/1838/
    - joeythedolphin 3y ago
      No it's a digital neural network expressing it's inputs and outputs in language.
      - deleted 3y ago
        [deleted]
      - 3y ago

rsynnott 3y ago

> People are talking about ChatGPT hallucinating It's really unfortunate that some much of the terminology around LLMs is so anthropomorphic; it's extremely misleading. (People sometimes suggest replacing this one with confabulating, but I'm not really sure that's much better)

ggambetta 3y ago

Absolutely. I feel like they're essentially Markov chain generators on steroids. Generally entertaining, sometimes useful, ultimately shallow. I'm really surprised when people, especially technical people, say they "asked" something to ChatGPT, or that they had a "conversation". It doesn't know anything. It doesn't understand anything. The illusion is very convincing, but it's just an illusion. I know I'm part of a minority of people who think this way :( The last few months have felt like I'm taking crazy pills :(

EchoChamberMan 3y ago
Sales people, and company owners, have a vested interest in pushing the narrative that the illusion is actually magic.
crote 3y ago
I believe that for a large number of applications the distinction between "knowing something" and "containing knowledge" doesn't really matter. In 2010s Google I could type in a query, and it would redirect me to a human-written answer on something like StackOverflow with hopefully the right answer; in 2024 ChatGPT you can type in a query and it'll mash together some relevant human-written snippets into something which is hopefully coherent, relevant, and correct. Sure, it's "just" a Markov chain generator on steroids, but it's still giving you useful output. I do agree that it is a bit worrying to what extent people are anthropomorphizing it, and treating it like something with an actual opinion or an unquestionable source-of-truth. The "I asked ChatGPT" comments should be treated the same as "I put it into Google and this was the first result": it's just line noise and doesn't add anything to the conversation. If you don't have anything to add yourself, why comment at all?
__loam 3y ago
You're not alone
logicprog 3y ago
I feel the same way. It's frustrating feeling like I'm one of the few people that can see that the emperor has no clothes, and nice to see the occasional person like you that pushes back on this cargo cult fanaticism.

__loam 3y ago

I've been ranting about this for months. The underlying shape of these things is inherently an approximation. We're just guessing but some are acting like we've built the godhead.