5 ms·
Transformer == Modern AI? Wow, I dont know where should I start My mind literally stuck at the absurdity
by bigcat12345678 3y ago
Transformer == Modern AI?
Wow, I dont know where should I start
My mind literally stuck at the absurdity
- ozymandias1337 3y agoAI is in the eye of the beholder.
- deleted 3y ago[deleted]
- supportengineer 3y agoAnd in this case, the transformers are more than meets the eye.
- selimthegrim 3y agoI’m sure there’s a beam search joke in here somewhere but it’s too late in the day for me to think about it.
- itsoktocry 3y agoIt's 2024; do we really have to point out the fact that publications take liberties with headlines? It's been that way forever. Besides that, are you denying that transformers are the fundamental piece behind all the AI hype today? That's the point of the article. I think the fact that a mainstream publication is writing about the Transformers paper is awesome.
- YeGoblynQueenne 3y agoOK, so the title should be changed to "8 Google Employees Invented Modern AI Hype"?
- bigcat12345678 3y ago8 Google Employees authored the Transformer paper, which is the common architecture of all LLMs
- tsunamifury 3y agoAI hipsterism
- bigcat12345678 3y agoModern AI is such a big term that its way off the scale of normal click-baiting... Modern AI should anything after https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop Not transformer AI has been people's dream since written history. i.e., everyone in their own sense would want to invent something that can do things and think for themselves. That's literally the meaning of AI.
- calf 3y agoWhat makes transformers the fundamental piece over say, big datasets, or Moore's law, or deep nets in general, etc?
- J_Shelby_J 3y agoDo you think this current AI hype machine came from something other than transformers?
- golol 3y agoNo, but it is weird to use "modern" here. Modern suggest a longer timeframe. I would say machine learning deep NNs is modern AI. It's just not true that everything not transformer is outdated, but it is "kinda" true that everything not deep NN is outdated.
- cjbprime 3y ago> It's just not true that everything not transformer is outdated. Am not expert, do you have some links about this? i.e. a neural net construction that outperforms a transformer model of the same size.
- deleted 3y ago[deleted]
- pedrosorio 3y agoConvNets Match Vision Transformers at Scale (Oct 2023) https://arxiv.org/abs/2310.16764 https://arxiv.org/abs/2310.16764
- golol 3y agoWell transformers still have some plausible competition in NLP but besides that, there are other fields of AI where convnets or RNNs still make a lot of sense.
- deleted 3y ago[deleted]
- HarHarVeryFunny 3y agoThe phrasing kinda makes sense to me. Consider "modern" to mean NN/connectionist vs GOFAI AI attempts like CYC or SOAR. I guess it depends on how you define "AI", and whether you accept the media's labelling of anything ML-related as AI. To me, LLMs are the first thing deserving to be called AI, and other NNs like CNNs better just called ML since there is no intelligence there.
- startupsfail 3y agoTransformer is an evolution of large scale training of seq2seq. Diffusion models is an evolution of even older approach - training denoising autoencoders. It had been a relatively gradual and accelerating progress, with number of people working in the field increasing exponentially, since 2010 or so, when Deep Learning on GPUs was popularized at NIPS by Theano. Tens of thousands people working together on Deep Learning. Many more on GPUs.
- serjester 3y agoModern AI != Machine Learning. Artificial intelligence has always been about models being able to accomplish a broad swath of tasks that they weren't explicitly trained on. It seems hard to argue we had accomplished this before massive transformers came along.