3 ms·

I’ll dare make a prediction: Transformers will be declared a dead end in AI in few years and GPT-4 might be the last in line. The method is able to produce high

I’ll dare make a prediction: Transformers will be declared a dead end in AI in few years and GPT-4 might be the last in line. The method is able to produce highly convincing gimmicks (which is what it comes to once initial excitement splashes) but that is not how art or intelligence works.

ebiester 4y ago
Everything is eventually called a dead end in AI, no? Once it has moved from the bleeding edge to something the public uses, the limitations become known and everyone says, "of course a computer can do it, but that's not intelligence."
- feet 4y ago
  That's because we aren't building systems capable of self-analysis and awareness. As far as I'm aware, there are only AI/ML systems capable of analyzing some data and giving some output with little to no nuance
- Rerarom 4y ago
  no, the OP says it in a different sense, namely that even if you are duly impressed by these advancements, they will stop in a few years and a new line of research will be needed
deleted 4y ago
[deleted]
lbotos 4y ago
> The method is able to produce highly convincing gimmicks (which is what it comes to once initial excitement splashes) but that is not how art or intelligence works. Uh, art is often about reproducing highly convincing gimmicks. See https://en.wikipedia.org/wiki/Realism_(arts) https://en.wikipedia.org/wiki/Realism_(arts) I think you mean that Transformers will become kitsch, which probably.
- etrautmann 4y ago
  We've already seen this to some extent with GANs - which now look somewhat outdated in comparison with Dall-e and others
Jack000 4y ago
Language model != Transformers. The scaling power of the transformer architecture allows LLMs to exist, but LMs are just one application of transformers. The architecture itself is extremely generic and is applicable to nearly every problem. I'm pretty certain whatever comes after GPT-4 will still be using layered self-attention.

ebiester 4y ago

Everything is eventually called a dead end in AI, no? Once it has moved from the bleeding edge to something the public uses, the limitations become known and everyone says, "of course a computer can do it, but that's not intelligence."

feet 4y ago
That's because we aren't building systems capable of self-analysis and awareness. As far as I'm aware, there are only AI/ML systems capable of analyzing some data and giving some output with little to no nuance
Rerarom 4y ago
no, the OP says it in a different sense, namely that even if you are duly impressed by these advancements, they will stop in a few years and a new line of research will be needed

deleted 4y ago

[deleted]

lbotos 4y ago

> The method is able to produce highly convincing gimmicks (which is what it comes to once initial excitement splashes) but that is not how art or intelligence works. Uh, art is often about reproducing highly convincing gimmicks. See https://en.wikipedia.org/wiki/Realism_(arts) https://en.wikipedia.org/wiki/Realism_(arts) I think you mean that Transformers will become kitsch, which probably.

etrautmann 4y ago
We've already seen this to some extent with GANs - which now look somewhat outdated in comparison with Dall-e and others

Jack000 4y ago

Language model != Transformers. The scaling power of the transformer architecture allows LLMs to exist, but LMs are just one application of transformers. The architecture itself is extremely generic and is applicable to nearly every problem. I'm pretty certain whatever comes after GPT-4 will still be using layered self-attention.