5 ms·

You (and everyone else) seem to be making the classic "mistake" of looking at an early version and not appreciating that things improve. Ten years ago, AI-gener

by edanm 4y ago

You (and everyone else) seem to be making the classic "mistake" of looking at an early version and not appreciating that things improve. Ten years ago, AI-generated art was at 50%. 2 years ago, 80%. Now it's at 95% and winning competitions. I have no idea if the AI that's getting code 80% right today will get it 95% right in two years, but given current progress, I wouldn't bet against it. I don't think there's any fundamental reason it can't produce better code than I can, at least not at the "write a function that does X" level. Whole systems are a way harder problem that I wouldn't even think of making guesses about.

yamtaddle 4y ago
To be fair to those assumptions, there've been a lot of cases of machine-learning (among other tech) looking very promising, and advancing so quickly that a huge revolution seems imminent—then stalling out at a local maximum for a really long time.
ben_w 4y ago
It might improve like Go AI and shock everyone by beating the world expert at everything, or it might improve like Tesla FSD which is annoyingly harder than "make creative artwork". There's no fundamental reason it can't be the world expert at everything, but that's not a reason to assume we know how to get there from here.
- namelessoracle 4y ago
  What scares me is a death of progress situation. Maybe it cant be an expert, but it can be good enough, and now the supply pipeline of people who could be experts basically gets shut off, because to become an expert you needed to do the work and gain the experiences that are now completely owned by AI.
  - tintor 4y ago
    But it could also make it easier to train experts, by acting as a coach and teacher.
  - nonrandomstring 4y ago
    Exactly this. The problem of a vengeful god who demands the slaughter of infidels lies not in his existence or nonexistence, but peoples' belief in such a god. Similarly, it does not matter whether AI works or it doesn't. It's irrelevant how good it actually is. What matters is whether people "believe" in it. AI is not a technology, it's an ideology. Given time it will fulfil it's own prophecy as "we who believe" steer the world toward that. That's what's changing now. It's in the air. The ruling classes (those who own capital and industry) are looking at this. The workers are looking too. Both of them see a new world approaching, and actually everyone is worried. What is under attack is not the jobs of the current generation, but the value of human skill itself, for all generations to come. And, yes, it's the tail of a trajectory we have been on for a long time. It isn't the only way computers can be. There is IA instead of AI. But intelligence amplification goes against the principles of capital at this stage. Our trajectory has been to make people dumber in service of profit.
    - CadmiumYellow 4y ago
      > What is under attack is not the jobs of the current generation, but the value of human skill itself, for all generations to come. And, yes, it's the tail of a trajectory we have been on for a long time. Wow, yes. This is exactly what I've been thinking but you summed it up more eloquently.
    - melagonster 4y ago
      can't agree more! if anyone start to believe it, it will work in some terrible way, even there is only one algorithm in black box.
    - int_19h 4y ago
      What's under attack is the notion that humans are special - that there's some kind of magic to them that is fundamentally impossible to replicate. No wonder there's a full-blown moral panic about this.
      - nonrandomstring 4y ago
        Agreed, but that train left the station in the late 1800s, driven by Darwin and Nietzsche. The intervening one and a half centuries haven't dislodged the "human spirit" in its secular form. We thought we'd overcome "gods". Now, out of discontent and self-loathing we're going to do what Freud warned against, and find a new external something to subjugate ourselves to. We simply refuse to shoulder the burden of being free.
  - int_19h 4y ago
    https://en.wikipedia.org/wiki/Profession_(novella) https://en.wikipedia.org/wiki/Profession_(novella)
- rafaelero 4y ago
  Tesla is limited by the processing power contained in the chip of each car. That's not the case for language models; they can get arbitrarily large without much problem with latency. If Tesla could train just one huge model in a data center and deliver it by API to every car I bet self driving cars would have already been a reality.
idontpost 4y ago
marcosdumay 4y ago
The architecture behind the chatGPT and the other AIs that are making the news won't ever improve so it can correctly write non-trivial code. There is a fundamental reason for that. Other architectures exist, but you can notice from the lack of people talking about them that they don't produce any output nearly as developed as the chatGPT kind. They will get there eventually, but that's not what we are seeing here.
- edanm 4y ago
  > The architecture behind the chatGPT and the other AIs that are making the news won't ever improve so it can correctly write non-trivial code. There is a fundamental reason for that. What is that?
  - CuriouslyC 4y ago
    Probably because it doesn't maintain long term cohesion. Transformer models are great at producing things that look right over short distances, but as the output length increases it often becomes contradictory or nonsensical. To get good output on larger scales we're going to need a model that is hierarchical with longer term self attention.
muttled 4y ago
Whole systems from a single prompt are probably a ways away, but I was able to get further than I expected by asking it what classes would makeup the task I was trying to do and then having it write those classes.

by edanm 4y ago

yamtaddle 4y ago

To be fair to those assumptions, there've been a lot of cases of machine-learning (among other tech) looking very promising, and advancing so quickly that a huge revolution seems imminent—then stalling out at a local maximum for a really long time.

ben_w 4y ago

It might improve like Go AI and shock everyone by beating the world expert at everything, or it might improve like Tesla FSD which is annoyingly harder than "make creative artwork". There's no fundamental reason it can't be the world expert at everything, but that's not a reason to assume we know how to get there from here.

namelessoracle 4y ago
What scares me is a death of progress situation. Maybe it cant be an expert, but it can be good enough, and now the supply pipeline of people who could be experts basically gets shut off, because to become an expert you needed to do the work and gain the experiences that are now completely owned by AI.
- tintor 4y ago
  But it could also make it easier to train experts, by acting as a coach and teacher.
- nonrandomstring 4y ago
  Exactly this. The problem of a vengeful god who demands the slaughter of infidels lies not in his existence or nonexistence, but peoples' belief in such a god. Similarly, it does not matter whether AI works or it doesn't. It's irrelevant how good it actually is. What matters is whether people "believe" in it. AI is not a technology, it's an ideology. Given time it will fulfil it's own prophecy as "we who believe" steer the world toward that. That's what's changing now. It's in the air. The ruling classes (those who own capital and industry) are looking at this. The workers are looking too. Both of them see a new world approaching, and actually everyone is worried. What is under attack is not the jobs of the current generation, but the value of human skill itself, for all generations to come. And, yes, it's the tail of a trajectory we have been on for a long time. It isn't the only way computers can be. There is IA instead of AI. But intelligence amplification goes against the principles of capital at this stage. Our trajectory has been to make people dumber in service of profit.
  - CadmiumYellow 4y ago
    > What is under attack is not the jobs of the current generation, but the value of human skill itself, for all generations to come. And, yes, it's the tail of a trajectory we have been on for a long time. Wow, yes. This is exactly what I've been thinking but you summed it up more eloquently.
  - melagonster 4y ago
    can't agree more! if anyone start to believe it, it will work in some terrible way, even there is only one algorithm in black box.
  - int_19h 4y ago
    What's under attack is the notion that humans are special - that there's some kind of magic to them that is fundamentally impossible to replicate. No wonder there's a full-blown moral panic about this.
    - nonrandomstring 4y ago
      Agreed, but that train left the station in the late 1800s, driven by Darwin and Nietzsche. The intervening one and a half centuries haven't dislodged the "human spirit" in its secular form. We thought we'd overcome "gods". Now, out of discontent and self-loathing we're going to do what Freud warned against, and find a new external something to subjugate ourselves to. We simply refuse to shoulder the burden of being free.
- int_19h 4y ago
  https://en.wikipedia.org/wiki/Profession_(novella) https://en.wikipedia.org/wiki/Profession_(novella)
rafaelero 4y ago
Tesla is limited by the processing power contained in the chip of each car. That's not the case for language models; they can get arbitrarily large without much problem with latency. If Tesla could train just one huge model in a data center and deliver it by API to every car I bet self driving cars would have already been a reality.

idontpost 4y ago

marcosdumay 4y ago

The architecture behind the chatGPT and the other AIs that are making the news won't ever improve so it can correctly write non-trivial code. There is a fundamental reason for that. Other architectures exist, but you can notice from the lack of people talking about them that they don't produce any output nearly as developed as the chatGPT kind. They will get there eventually, but that's not what we are seeing here.

edanm 4y ago
> The architecture behind the chatGPT and the other AIs that are making the news won't ever improve so it can correctly write non-trivial code. There is a fundamental reason for that. What is that?
- CuriouslyC 4y ago
  Probably because it doesn't maintain long term cohesion. Transformer models are great at producing things that look right over short distances, but as the output length increases it often becomes contradictory or nonsensical. To get good output on larger scales we're going to need a model that is hierarchical with longer term self attention.

muttled 4y ago

Whole systems from a single prompt are probably a ways away, but I was able to get further than I expected by asking it what classes would makeup the task I was trying to do and then having it write those classes.