7 ms·

No this is not yet the next industrial evolution. ChatGPT is still at the beginning of its hype curve. It is amazing, however, it is just a language model. It c

No this is not yet the next industrial evolution. ChatGPT is still at the beginning of its hype curve. It is amazing, however, it is just a language model. It can't reason, just regurgitate.

echelon 4y ago
Shortbets 24 months. Looking at LLMs is also not seeing the forest of Tacotron, neural vocoders, acoustic models, Stable Diffusion, NeRFs... This is snowballing so fast.
- Xelynega 4y ago
  All of those examples have the exact same limitation, they can't replace processes that create novel ideas since they're statistical recombinations of existing ones.
  - dwaltrip 4y ago
    That's not a fleshed out argument. Why do you believe that "statistical combinations" can't create novel ideas?
    - MagicMoonlight 4y ago
      Because it has no mechanism for creating novel ideas... It's like saying a calculator could spontaneously derive its own proof. No it can't, because it has no ability to do that. It physically does not have a way to create new ideas. It cannot reshape its own circuitry like a brain and it does not have spontaneous thoughts.
      - mashygpig 4y ago
        How do humans come up with novel ideas?
        sanroot98 4y ago
        First we need to define ,what creativity and intelligence means ,so we can compare that with human results
      - dwaltrip 4y ago
        I hear what you are saying, but I'm not convinced. Can you dig a bit deeper? What makes an idea "novel"? What types of physical processes can create new ideas or have spontaneous thoughts, and which can't? What is "special" about brains, in your view? I'm looking for some meat on the bone here.
      - Sparyjerry 4y ago
        ChatGPT is not like other chat bots which operate like a calculator and that is exactly why it is popular. You can put random inputs like "write a short story with 900 words with a bear and wolf about driving to the casino where they win big but can't spend the money" and it will output something total unique based on those inputs.
- nwienert 4y ago
  I’ve been around this stuff for 7 years now, former co-founder was super into AI hype and actually ended up leaving to go work for OpenAI. Nothing is snowballing fast now… it’s been the exact same since at least 7 years ago. At the time I have to admit I was equally bullishly myopic, the new models seemed insanely cool and the process seemed super-exponential. I predicted AI may replace programming, art and many things within 5 years. Then I checked in 5 years later - not much really had changed and I revised my predictions way down. Since then I’ve revised them down again actually, as I see them running into scaling issues again, amongst many other issues. The attitude you take is infectious, I saw it firsthand. But if I look at what concrete is changing, it’s not much. Humans love watching Chess. It’s exploded in popularity in the last 4 years. No one watches AI chess, though the top players certainly gain in their practice from it. But where’s the revolution? I know it’s different but I predict the same happens across the “new” areas - image and text generation. They seem amazing, likely beating humans in many ways. But in the end we don’t care much about computers, we care about novelty. Novelty and humanity drive fashion, music, film, literature… basically all art. It needs to innovate, needs to be extremely personal. It’s not going to disrupt any of them, not even close. But it will be used in the loop.
  - Sparyjerry 4y ago
    What is revolutionary is that ChatGPT is indistinguishable from a real person's writing and intelligence on a subject - and not just in text, but in coding. Before ChatGPT any similar program's responses were so obvious bots that you would look at and say "oh that's cool" but it was still garbage you would never use. Now, I've heard people using ChatGPT for self (ai) therapy, asking it what to do, and how to do something. It's a planner, a worker, a designer, an idea machine. It's like Google but actually gets you to the answers you want without the incredible about of garbage that came when SEO websites took over Google. Then you've got the symbiosis with humans, it's not "look at this AI art." It's artists using AI tools to 'produce' 1000 versions of art a day and pick the best one they find most personal. AI art was already entered into a human contest by someone cheating in which it won. Before iPhone there was plenty of similar products but none that put all the pieces together the way an iPhone did. There is a tipping point to AI and ChatGPT is the iPhone of AI.
- MagicMoonlight 4y ago
  But none of those products solve the problem of the "AI" being stupid as fuck. Because it's not AI, it's just brute forcing. The current solution of 500b parameters or whatever is literally just a lookup table of existing data. They could replace it with an SQLite database with the same amount of data and get the same solution. There is no thought, there is no reasoning. It can't come up with new ways of doing anything, it just regurgitates based on its pre-programmed ruleset. That's why self driving doesn't work - it doesn't know what a fucking car is. Ask it what a road is or a car and it has no idea, it just has a million miles of road in its database. Ask it how to drive on a country lane and it crashes because that's not in the database. There's nothing there.
  - dwaltrip 4y ago
    > The current solution of 500b parameters or whatever is literally just a lookup table of existing data. They could replace it with an SQLite database with the same amount of data and get the same solution. I'm pretty sure this is completely false?
    - pifm_guy 4y ago
      Correct. ML papers always try to find the most similar examples in the training set, and rarely find a close match.
    - ClueslessTech99 4y ago
      How so? What would be the difference? Other than being unviable.
      - dwaltrip 4y ago
        I don’t have a deep understanding of how these language models work, but what do I know makes think that saying “It is basically a lookup table” is sort of like saying “A car is basically a bike”. Sure, there are some very vague similarities, but they are also incredibly different. A bike and a car both take you places. But the kinds of places they can take you and how effectively they do so differs vastly. Likewise, both lookup tables and LLMs take an input and then spit something out. But the similarities basically stop there.
kirso 4y ago
Pretty much this. LLMs are still limited, perhaps by design but it is so. I recently summarised an article by Maggie Appleton on some of the shortcomings of the model and the implication here: https://twitter.com/kirso_/status/1610849402557722625 https://twitter.com/kirso_/status/1610849402557722625 disclaimer: I am not the author of the original article. Several examples: - Language models cannot reason, consult external sources, or sense the world as we can. They also lack access to a shared reality, embodied experiences, and the ability to tell stories grounded in culture, such as referencing obscure concepts or interests. - Current LLMs fall short due to recycled ideas, unoriginal explanations and not having an opinion. Excellence in writing (and thus thinking) requires a greater sense of critical reasoning, scepticism and going beyond the status quo and comfort. EDIT: a few examples added.

No this is not yet the next industrial evolution. ChatGPT is still at the beginning of its hype curve. It is amazing, however, it is just a language model. It can't reason, just regurgitate.

echelon 4y ago

Shortbets 24 months. Looking at LLMs is also not seeing the forest of Tacotron, neural vocoders, acoustic models, Stable Diffusion, NeRFs... This is snowballing so fast.

Xelynega 4y ago
All of those examples have the exact same limitation, they can't replace processes that create novel ideas since they're statistical recombinations of existing ones.
- dwaltrip 4y ago
  That's not a fleshed out argument. Why do you believe that "statistical combinations" can't create novel ideas?
  - MagicMoonlight 4y ago
    Because it has no mechanism for creating novel ideas... It's like saying a calculator could spontaneously derive its own proof. No it can't, because it has no ability to do that. It physically does not have a way to create new ideas. It cannot reshape its own circuitry like a brain and it does not have spontaneous thoughts.
    - mashygpig 4y ago
      How do humans come up with novel ideas?
      - sanroot98 4y ago
        First we need to define ,what creativity and intelligence means ,so we can compare that with human results
    - dwaltrip 4y ago
      I hear what you are saying, but I'm not convinced. Can you dig a bit deeper? What makes an idea "novel"? What types of physical processes can create new ideas or have spontaneous thoughts, and which can't? What is "special" about brains, in your view? I'm looking for some meat on the bone here.
    - Sparyjerry 4y ago
      ChatGPT is not like other chat bots which operate like a calculator and that is exactly why it is popular. You can put random inputs like "write a short story with 900 words with a bear and wolf about driving to the casino where they win big but can't spend the money" and it will output something total unique based on those inputs.
nwienert 4y ago
I’ve been around this stuff for 7 years now, former co-founder was super into AI hype and actually ended up leaving to go work for OpenAI. Nothing is snowballing fast now… it’s been the exact same since at least 7 years ago. At the time I have to admit I was equally bullishly myopic, the new models seemed insanely cool and the process seemed super-exponential. I predicted AI may replace programming, art and many things within 5 years. Then I checked in 5 years later - not much really had changed and I revised my predictions way down. Since then I’ve revised them down again actually, as I see them running into scaling issues again, amongst many other issues. The attitude you take is infectious, I saw it firsthand. But if I look at what concrete is changing, it’s not much. Humans love watching Chess. It’s exploded in popularity in the last 4 years. No one watches AI chess, though the top players certainly gain in their practice from it. But where’s the revolution? I know it’s different but I predict the same happens across the “new” areas - image and text generation. They seem amazing, likely beating humans in many ways. But in the end we don’t care much about computers, we care about novelty. Novelty and humanity drive fashion, music, film, literature… basically all art. It needs to innovate, needs to be extremely personal. It’s not going to disrupt any of them, not even close. But it will be used in the loop.
- Sparyjerry 4y ago
  What is revolutionary is that ChatGPT is indistinguishable from a real person's writing and intelligence on a subject - and not just in text, but in coding. Before ChatGPT any similar program's responses were so obvious bots that you would look at and say "oh that's cool" but it was still garbage you would never use. Now, I've heard people using ChatGPT for self (ai) therapy, asking it what to do, and how to do something. It's a planner, a worker, a designer, an idea machine. It's like Google but actually gets you to the answers you want without the incredible about of garbage that came when SEO websites took over Google. Then you've got the symbiosis with humans, it's not "look at this AI art." It's artists using AI tools to 'produce' 1000 versions of art a day and pick the best one they find most personal. AI art was already entered into a human contest by someone cheating in which it won. Before iPhone there was plenty of similar products but none that put all the pieces together the way an iPhone did. There is a tipping point to AI and ChatGPT is the iPhone of AI.
MagicMoonlight 4y ago
But none of those products solve the problem of the "AI" being stupid as fuck. Because it's not AI, it's just brute forcing. The current solution of 500b parameters or whatever is literally just a lookup table of existing data. They could replace it with an SQLite database with the same amount of data and get the same solution. There is no thought, there is no reasoning. It can't come up with new ways of doing anything, it just regurgitates based on its pre-programmed ruleset. That's why self driving doesn't work - it doesn't know what a fucking car is. Ask it what a road is or a car and it has no idea, it just has a million miles of road in its database. Ask it how to drive on a country lane and it crashes because that's not in the database. There's nothing there.
- dwaltrip 4y ago
  > The current solution of 500b parameters or whatever is literally just a lookup table of existing data. They could replace it with an SQLite database with the same amount of data and get the same solution. I'm pretty sure this is completely false?
  - pifm_guy 4y ago
    Correct. ML papers always try to find the most similar examples in the training set, and rarely find a close match.
  - ClueslessTech99 4y ago
    How so? What would be the difference? Other than being unviable.
    - dwaltrip 4y ago
      I don’t have a deep understanding of how these language models work, but what do I know makes think that saying “It is basically a lookup table” is sort of like saying “A car is basically a bike”. Sure, there are some very vague similarities, but they are also incredibly different. A bike and a car both take you places. But the kinds of places they can take you and how effectively they do so differs vastly. Likewise, both lookup tables and LLMs take an input and then spit something out. But the similarities basically stop there.

kirso 4y ago

Pretty much this. LLMs are still limited, perhaps by design but it is so. I recently summarised an article by Maggie Appleton on some of the shortcomings of the model and the implication here: https://twitter.com/kirso_/status/1610849402557722625 https://twitter.com/kirso_/status/1610849402557722625 disclaimer: I am not the author of the original article. Several examples: - Language models cannot reason, consult external sources, or sense the world as we can. They also lack access to a shared reality, embodied experiences, and the ability to tell stories grounded in culture, such as referencing obscure concepts or interests. - Current LLMs fall short due to recycled ideas, unoriginal explanations and not having an opinion. Excellence in writing (and thus thinking) requires a greater sense of critical reasoning, scepticism and going beyond the status quo and comfort. EDIT: a few examples added.