5 ms·

I categorically reject the notion that LMMs (Large Markov Models) can ever be aware or intelligent. Comparing weighted next-word-engines to feeling, thinking, a

I categorically reject the notion that LMMs (Large Markov Models) can ever be aware or intelligent. Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting.

burnished 3y ago
Are you intentionally echoing that scene from starship troopers where the pundit makes the same claim about 'smart bugs'?
- hospitalhusband 3y ago
  I don't base my ideas on what constitutes humanity, intelligence, or sentience on hundred million dollar fiction.
  - burnished 3y ago
    Alright then you should also consider not basing it off of whether you find the concept insulting. It doesn't seem to be the strongest rebuttal available.
    - hospitalhusband 3y ago
      That it's insulting has nothing to do with my reasoning. You have it backwards.
sebzim4500 3y ago
What's an LMM? I've never seen the term. > Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting Why is it reasonable to be so reductionist about e.g. GPT-4 but not be so reductionist about a biological brain? E.g., why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? It's equally true, and equally missing the point.
- hawski 3y ago
  I think that machine learning probably can produce something akin to a brain, but LLMs are not really it even if they use the digital equivalent of a neuron. As much as I understand what I read about LLMs they really seem to be descendants of Markov chains. I think they are valuable and can go a long way, but LLMs themselves will not be "it". I think that we will get to a ceiling with them within 10 years if we will not think about something else. I think the ceiling can be made pretty high though. However most probably in 10 years we will all laugh how all of our predictions missed by a long shot.
- hospitalhusband 3y ago
  LMM = Large Markov Model. I use that term because models like GPT-4 and friends are for all intents and purposes Markov chains with more data, more compute, some lossy compression, and a bit of nearest neighbor search. Next-word-engines. > why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? Because we don't think one word at a time, and we don't restart from scratch for every subsequent word.
  - esafak 3y ago
    GPT is not Markovian; it has state.
    - hospitalhusband 3y ago
      Then it's a markov-like with state. Or as I've taken to calling them lately Markov+state. (I couldn't resist, sorry.) A truck towing a trailer isn't just a car because it pivots in the middle and has more wheels. It's fundamentals of operation are still closer to a car or truck without trailer than a bicycle. Humans can form thoughts and get to mostly correct answers even as a gut feeling, and the language to explain why/how need not even be present. We don't form thoughts one word at a time.
      - xcv123 3y ago
deleted 3y ago
[deleted]

I categorically reject the notion that LMMs (Large Markov Models) can ever be aware or intelligent. Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting.

burnished 3y ago

Are you intentionally echoing that scene from starship troopers where the pundit makes the same claim about 'smart bugs'?

hospitalhusband 3y ago
I don't base my ideas on what constitutes humanity, intelligence, or sentience on hundred million dollar fiction.
- burnished 3y ago
  Alright then you should also consider not basing it off of whether you find the concept insulting. It doesn't seem to be the strongest rebuttal available.
  - hospitalhusband 3y ago
    That it's insulting has nothing to do with my reasoning. You have it backwards.

sebzim4500 3y ago

What's an LMM? I've never seen the term. > Comparing weighted next-word-engines to feeling, thinking, aware beings is insulting Why is it reasonable to be so reductionist about e.g. GPT-4 but not be so reductionist about a biological brain? E.g., why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? It's equally true, and equally missing the point.

hawski 3y ago
I think that machine learning probably can produce something akin to a brain, but LLMs are not really it even if they use the digital equivalent of a neuron. As much as I understand what I read about LLMs they really seem to be descendants of Markov chains. I think they are valuable and can go a long way, but LLMs themselves will not be "it". I think that we will get to a ceiling with them within 10 years if we will not think about something else. I think the ceiling can be made pretty high though. However most probably in 10 years we will all laugh how all of our predictions missed by a long shot.
hospitalhusband 3y ago
LMM = Large Markov Model. I use that term because models like GPT-4 and friends are for all intents and purposes Markov chains with more data, more compute, some lossy compression, and a bit of nearest neighbor search. Next-word-engines. > why can't I say that your brain is nothing but a bunch of biological neurons trained using its input and intialized based on your genetics? Because we don't think one word at a time, and we don't restart from scratch for every subsequent word.
- esafak 3y ago
  GPT is not Markovian; it has state.
  - hospitalhusband 3y ago
    Then it's a markov-like with state. Or as I've taken to calling them lately Markov+state. (I couldn't resist, sorry.) A truck towing a trailer isn't just a car because it pivots in the middle and has more wheels. It's fundamentals of operation are still closer to a car or truck without trailer than a bicycle. Humans can form thoughts and get to mostly correct answers even as a gut feeling, and the language to explain why/how need not even be present. We don't form thoughts one word at a time.
    - xcv123 3y ago

deleted 3y ago

[deleted]