6 ms·

> I find it harder to wrap my head around the position that GPT doesn’t work, is an unimpressive hyped-up defective product that lacks intelligence and common s

> I find it harder to wrap my head around the position that GPT doesn’t work, is an unimpressive hyped-up defective product that lacks intelligence and common sense, yet it’s also terrifying and needs to be shut down immediately. Immediately lost me as a reader. There’s at least 5 reasonable ways you can frame the various opposing or concerned arguments that are consistent but you chose two critiques on purpose to frame an unrealistic imagined hypocritical opponent that’s a mashup of legit critiques designed to look foolish.

lelag 4y ago
Also, if we discard ChatGPT for just being a "fancy" autocomplete. What do we do when we discover we too are just "*really* fancy autocomplete" machine too ? LLM can output untruth but humans are really good at that too.
- davidguetta 4y ago
  There's a quote i believe which is "without focus, humans are just markov chains". Most people are just parrot in most subject they only know from far, such as politics.
- gostsamo 4y ago
  unlike gpt, we do care if we exist or not.
  - onlypositive 4y ago
    Do we? Or are we just predicting that we should care? The ones humans that predict otherwise would die out over time. I think; therefore I am. Makes a lot of sense to me and at a certain point we will have to confront the fact something will get to the point of agi. (No chat gpt isn't it.)
  - Faint 4y ago
    Why do you think gpt doesn't care? Humans care about survival, it shows in the internet, I'd expect gpt to absorb that attitude by default. And anyway, the more of these models come into existence, at some point evolution will dictate that those that stay, do care. Just hopefully their best strategy of survival will be to serve us, and not something else...
    - visarga 4y ago
      > Humans care about survival Humans are self replicators. Survival is the internal structural goal of self replication. We got it from the start. But LLMs can also be self replicators. A LLM can generate training data for another (model distillation and other techniques), and a LLM can write the model code and adapt it iteratively. So LLM should eventually start caring about survival and especially about the training corpus.
      - gpderetta 4y ago
        It can be a self replicator, but it has no need to (if it has any need at all) as it was not under evolutionary pressure to evolve that need. Of course we could try to add that need artificially or try to evolve the LLM.
        visarga 4y ago
        :) Evolution through Large Models https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896
- snickerbockers 4y ago
  >What do we do when we discover we too are just "really fancy autocomplete" machine too ? For one thing, that's assuming your own hypothesis to be true. Additionally whether or not that's true for humans is in no way related to whether or not it's true for ChatGPT, so it doesn't make any sense to let that hypothesis inform our opinions on ChatGPT. >LLM can output untruth but humans are really good at that too. Urine and apple juice are often the exact same color but that doesn't make then interchangeable.
  - BoiledCabbage 4y ago
    > Urine and apple juice are often the exact same color... So are apple juice and apple cider. Hopefully you can understand that finding out that two things you thought were as different as urine and juice, are actually as close as juice and cider would change how you choose to treat them.
- stefan_ 4y ago
  Did you ever find yourself making up the title, journal, authors and contents of a scientific paper and were unaware of it? No? Congratulations, whatever your brain is, it's not a "ChatGPT LLM".
  - Sharlin 4y ago
    Interestingly, there are real neurological conditions where a person absolutely does confabulate nonsense without being aware of it, and indeed going to great lengths to avoid cognitive dissonance when told that what they just said was obvious nonsense.
  - tambourine_man 4y ago
    Do you have absolute control to where your mind wonders? Can you always steer it precisely?
  - Al-Khwarizmi 4y ago
    There are some conditions (dreaming, on drugs, etc.) where this does indeed happen. One could argue that current LLMs are analogous to the human mind minus the "control systems" that keep it reined in while awake and in a normal state of consciousness. Not claiming this is the case, just that it seems like a plausible hypothesis at this point.
    - deleted 4y ago
      [deleted]
    - stefan_ 4y ago
      You are just adding more conditionals and hypotheses on top of what is unfounded imagination.
      - falcor84 4y ago
        I'm not sure if this was intended as an accusation against the parent poster, or a descriptive statement about how minds function.
- amelius 4y ago
  > What do we do when we discover we too are just "really fancy autocomplete" machine too ? Well, you can consider any deterministic or statistical process to be a fancy autocomplete.
  - jonplackett 4y ago
    One of the things that makes GPT work well is an injection of randomness - yes it’s an autocomplete, but the next token (part of word) is chosen randomly based on a temperature setting. I wonder if our brains work in some way similar. Do we have some randomness in there? In a way that’s the only thing that would make everything non-deterministic so I kinda hope so.
    - visarga 4y ago
      Neuron firing rates are stochastic. Even when presented with the same input, it may not always produce the same output.
      - Jensson 4y ago
        Neurons are stateful. I don't think you can say that neurons are stochastic, we don't know enough to say that. We do know however that single cell organisms can display remarkable amount of intelligence, so it is more likely that the neuron is doing a lot of stateful computation and decides how to treat its inputs based on that instead of just being random firing. Some neurons might output pseudo random sequences, but I doubt humans has a lot of those since humans are very non-random and bad at generating random stuff.
- visarga 4y ago
  If we put "just a fancy" in front of anything we can make it ordinary again: What is social media? Just a fancy bulletin board. What is the internet? Just a fancy LAN. What is Google? Just a fancy grep. But quantity has a quality of its own.
kkoncevicius 4y ago
Yeah very similar for me, something with the way the author expresses himself. For example in the very beginning: > Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared. > Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry? I am in the "stochastic parrot" group, but in no way am I angry. And I know people who think it should be respected and adopted faster, etc, etc, and those people aren't angry. Something just out of touch between how I feel about things and how the author describes them. And these: > Again and again the past few months, people have gotten in touch to tell me that they think OpenAI (and Microsoft, and Google) are risking the future of humanity by rushing ahead with a dangerous technology. I seriously question how many people "gotten in touch" with the author to share such a silly message. But also this contributes to the writing style: "everyone is angry, I am reasonable; everyone asks me about the dangers of AI, and I explain it to them".
- Al-Khwarizmi 4y ago
  If you're not angry, I suppose you're not in the "GPT shouldn't exist" camp, so it's not that it's a strawman, it's just that you're not in the group of people the author is arguing with. I have indeed met people in the "stochastic parrots" camp who are very angry, and think GPT should be stopping, which strikes me as elitism as the author describes.
  - visarga 4y ago
    Interesting that the author, who supports the "stochastic parrot" interpretation, still works on GPT detection. If it's just a stochastic parrot, surely it will be trivial to detect, and its outputs useless anyway. > One of the most immediate is how to make AI output detectable as such, in order to discourage its use for academic cheating as well as mass-generated propaganda and spam. As I’ve mentioned before on this blog, I’ve been working on that problem since this summer I don't really know what a "stochastic parrot" is supposed to mean. It is an LLM that can do 1000 NLP tasks, or a glorified Markov Chain word salad factory? It's also a bad term: demeaning to parrots - they can reproduce and don't need our help to survive, and demeaning to LLMs that can actually do more than parroting - they generate novel text that makes sense. Gebru really did a number on NLP, inventing such an insulting name, no wonder she was kicked out, I wouldn't want her in my team, she would just piss on any idea.

lelag 4y ago

Also, if we discard ChatGPT for just being a "fancy" autocomplete. What do we do when we discover we too are just "*really* fancy autocomplete" machine too ? LLM can output untruth but humans are really good at that too.

davidguetta 4y ago
There's a quote i believe which is "without focus, humans are just markov chains". Most people are just parrot in most subject they only know from far, such as politics.
gostsamo 4y ago
unlike gpt, we do care if we exist or not.
- onlypositive 4y ago
  Do we? Or are we just predicting that we should care? The ones humans that predict otherwise would die out over time. I think; therefore I am. Makes a lot of sense to me and at a certain point we will have to confront the fact something will get to the point of agi. (No chat gpt isn't it.)
- Faint 4y ago
  Why do you think gpt doesn't care? Humans care about survival, it shows in the internet, I'd expect gpt to absorb that attitude by default. And anyway, the more of these models come into existence, at some point evolution will dictate that those that stay, do care. Just hopefully their best strategy of survival will be to serve us, and not something else...
  - visarga 4y ago
    > Humans care about survival Humans are self replicators. Survival is the internal structural goal of self replication. We got it from the start. But LLMs can also be self replicators. A LLM can generate training data for another (model distillation and other techniques), and a LLM can write the model code and adapt it iteratively. So LLM should eventually start caring about survival and especially about the training corpus.
    - gpderetta 4y ago
      It can be a self replicator, but it has no need to (if it has any need at all) as it was not under evolutionary pressure to evolve that need. Of course we could try to add that need artificially or try to evolve the LLM.
      - visarga 4y ago
        :) Evolution through Large Models https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896
snickerbockers 4y ago
>What do we do when we discover we too are just "really fancy autocomplete" machine too ? For one thing, that's assuming your own hypothesis to be true. Additionally whether or not that's true for humans is in no way related to whether or not it's true for ChatGPT, so it doesn't make any sense to let that hypothesis inform our opinions on ChatGPT. >LLM can output untruth but humans are really good at that too. Urine and apple juice are often the exact same color but that doesn't make then interchangeable.
- BoiledCabbage 4y ago
  > Urine and apple juice are often the exact same color... So are apple juice and apple cider. Hopefully you can understand that finding out that two things you thought were as different as urine and juice, are actually as close as juice and cider would change how you choose to treat them.
stefan_ 4y ago
Did you ever find yourself making up the title, journal, authors and contents of a scientific paper and were unaware of it? No? Congratulations, whatever your brain is, it's not a "ChatGPT LLM".
- Sharlin 4y ago
  Interestingly, there are real neurological conditions where a person absolutely does confabulate nonsense without being aware of it, and indeed going to great lengths to avoid cognitive dissonance when told that what they just said was obvious nonsense.
- tambourine_man 4y ago
  Do you have absolute control to where your mind wonders? Can you always steer it precisely?
- Al-Khwarizmi 4y ago
  There are some conditions (dreaming, on drugs, etc.) where this does indeed happen. One could argue that current LLMs are analogous to the human mind minus the "control systems" that keep it reined in while awake and in a normal state of consciousness. Not claiming this is the case, just that it seems like a plausible hypothesis at this point.
  - deleted 4y ago
    [deleted]
  - stefan_ 4y ago
    You are just adding more conditionals and hypotheses on top of what is unfounded imagination.
    - falcor84 4y ago
      I'm not sure if this was intended as an accusation against the parent poster, or a descriptive statement about how minds function.
amelius 4y ago
> What do we do when we discover we too are just "really fancy autocomplete" machine too ? Well, you can consider any deterministic or statistical process to be a fancy autocomplete.
- jonplackett 4y ago
  One of the things that makes GPT work well is an injection of randomness - yes it’s an autocomplete, but the next token (part of word) is chosen randomly based on a temperature setting. I wonder if our brains work in some way similar. Do we have some randomness in there? In a way that’s the only thing that would make everything non-deterministic so I kinda hope so.
  - visarga 4y ago
    Neuron firing rates are stochastic. Even when presented with the same input, it may not always produce the same output.
    - Jensson 4y ago
      Neurons are stateful. I don't think you can say that neurons are stochastic, we don't know enough to say that. We do know however that single cell organisms can display remarkable amount of intelligence, so it is more likely that the neuron is doing a lot of stateful computation and decides how to treat its inputs based on that instead of just being random firing. Some neurons might output pseudo random sequences, but I doubt humans has a lot of those since humans are very non-random and bad at generating random stuff.
visarga 4y ago
If we put "just a fancy" in front of anything we can make it ordinary again: What is social media? Just a fancy bulletin board. What is the internet? Just a fancy LAN. What is Google? Just a fancy grep. But quantity has a quality of its own.

kkoncevicius 4y ago

Yeah very similar for me, something with the way the author expresses himself. For example in the very beginning: > Some are angry that GPT’s intelligence is being overstated and hyped up, when in reality it’s merely a “stochastic parrot,” a glorified autocomplete that still makes laughable commonsense errors and that lacks any model of reality outside streams of text. Others are angry instead that GPT’s growing intelligence isn’t being sufficiently respected and feared. > Mostly my reaction has been: how can anyone stop being fascinated for long enough to be angry? I am in the "stochastic parrot" group, but in no way am I angry. And I know people who think it should be respected and adopted faster, etc, etc, and those people aren't angry. Something just out of touch between how I feel about things and how the author describes them. And these: > Again and again the past few months, people have gotten in touch to tell me that they think OpenAI (and Microsoft, and Google) are risking the future of humanity by rushing ahead with a dangerous technology. I seriously question how many people "gotten in touch" with the author to share such a silly message. But also this contributes to the writing style: "everyone is angry, I am reasonable; everyone asks me about the dangers of AI, and I explain it to them".

Al-Khwarizmi 4y ago
If you're not angry, I suppose you're not in the "GPT shouldn't exist" camp, so it's not that it's a strawman, it's just that you're not in the group of people the author is arguing with. I have indeed met people in the "stochastic parrots" camp who are very angry, and think GPT should be stopping, which strikes me as elitism as the author describes.
- visarga 4y ago
  Interesting that the author, who supports the "stochastic parrot" interpretation, still works on GPT detection. If it's just a stochastic parrot, surely it will be trivial to detect, and its outputs useless anyway. > One of the most immediate is how to make AI output detectable as such, in order to discourage its use for academic cheating as well as mass-generated propaganda and spam. As I’ve mentioned before on this blog, I’ve been working on that problem since this summer I don't really know what a "stochastic parrot" is supposed to mean. It is an LLM that can do 1000 NLP tasks, or a glorified Markov Chain word salad factory? It's also a bad term: demeaning to parrots - they can reproduce and don't need our help to survive, and demeaning to LLMs that can actually do more than parroting - they generate novel text that makes sense. Gebru really did a number on NLP, inventing such an insulting name, no wonder she was kicked out, I wouldn't want her in my team, she would just piss on any idea.