3 ms·

GANs are ruining AI research the way cryptocurrency ruins cryptography research. These are the empty calories of research. Focus on them and you will end up fa

GANs are ruining AI research the way cryptocurrency ruins cryptography research. These are the empty calories of research. Focus on them and you will end up fat (rich) and useless (no publications worth a damn) Prove me wrong, but from the pavement. You don’t need to be on my lawn.

etrautmann 3y ago
Do you mean LLMs or GANs specifically?
- bemusedthrow75 3y ago
  To be fair I mean LLMs, GANs, GPTs, all of this new stuff. It’s a good idea taken way beyond usefulness. Like processed food in plastic packaging. Toxic and empty. Full of promises and delusions about saving time and effort.
  - digging 3y ago
    I think that's possibly true but not certain. We're only just getting through the door on LLM capabilities, and we're seeing surprising emergent behavior such as internally modeling the board state of Othello games[1], so it's very possible this research brings us entirely new realms of capabilities by continuing to scale up and improve. [1] https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-o...
antegamisou 3y ago
This is the problem that almost no one talks about. Shitty papers gaining traction just because some are better bullshitters at grant writing.
simonw 3y ago
Pedantic note, but GANs (Generative Adversarial Networks) aren't particularly relevant these days. DALL-E, Stable Diffusion, Midjourney and the new Sora video model are diffusion models, which work differently from GANs. Maybe you meant "generative AI" rather than GANs? That's an umbrella term that covers ChatGPT-style language models and image/video/audio generation as well.
- bemusedthrow75 3y ago
  Generative AI covers it fine, yes. (I am typing on my phone on a train in between connections, both rail and internet, and I am grateful for your correction) Everyone knows what I mean by “empty calories” anyway. If they think I am wrong, they are in denial. ;-)
Workaccount2 3y ago
The reason these seem like empty calories now is because the compute cost is still too high for widespread implementation, combined with the simple fact that it just takes time for things to on the large scale. It won't be long (4-5 years) before AI starts appearing everywhere.
- bemusedthrow75 3y ago
  No, that’s not the reason they are empty calories, IMO. It’s the banality of the culture around the technology, as well as the banality of the technologies being built with it. It’s just academics chasing the dream of technology transfer to the ad-tech industry.

etrautmann 3y ago

Do you mean LLMs or GANs specifically?

bemusedthrow75 3y ago
To be fair I mean LLMs, GANs, GPTs, all of this new stuff. It’s a good idea taken way beyond usefulness. Like processed food in plastic packaging. Toxic and empty. Full of promises and delusions about saving time and effort.
- digging 3y ago
  I think that's possibly true but not certain. We're only just getting through the door on LLM capabilities, and we're seeing surprising emergent behavior such as internally modeling the board state of Othello games[1], so it's very possible this research brings us entirely new realms of capabilities by continuing to scale up and improve. [1] https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-o...

antegamisou 3y ago

This is the problem that almost no one talks about. Shitty papers gaining traction just because some are better bullshitters at grant writing.

simonw 3y ago

Pedantic note, but GANs (Generative Adversarial Networks) aren't particularly relevant these days. DALL-E, Stable Diffusion, Midjourney and the new Sora video model are diffusion models, which work differently from GANs. Maybe you meant "generative AI" rather than GANs? That's an umbrella term that covers ChatGPT-style language models and image/video/audio generation as well.

bemusedthrow75 3y ago
Generative AI covers it fine, yes. (I am typing on my phone on a train in between connections, both rail and internet, and I am grateful for your correction) Everyone knows what I mean by “empty calories” anyway. If they think I am wrong, they are in denial. ;-)

Workaccount2 3y ago

The reason these seem like empty calories now is because the compute cost is still too high for widespread implementation, combined with the simple fact that it just takes time for things to on the large scale. It won't be long (4-5 years) before AI starts appearing everywhere.

bemusedthrow75 3y ago
No, that’s not the reason they are empty calories, IMO. It’s the banality of the culture around the technology, as well as the banality of the technologies being built with it. It’s just academics chasing the dream of technology transfer to the ad-tech industry.