4 ms·

> Echoing many who now find themselves blindsided by the emergent abilities and rapid adoption of LLMs, the OP I've said this over and over again: there are no

> Echoing many who now find themselves blindsided by the emergent abilities and rapid adoption of LLMs, the OP I've said this over and over again: there are no emergent abilities. Before you leap to link me this paper, I'll link it myself: https://arxiv.org/abs/2206.07682 https://arxiv.org/abs/2206.07682 I read that paper. Did you? Did you understand it? Because if you had, you'd have seen that early on they define what they mean when they say "emergent abilities", and it's not what almost anyone else means when they say "emergent abilities". They're not claiming that the abilities of LLMs are anything more than the sum of their parts. "Emergent abilities" in that paper is an extraordinarily poor communication of the idea that larger models can do more than smaller models, which should be a surprise to no one. Stop spreading this nonsense. I'm not saying that LLMs aren't impressive. I'm just saying this breathless fantasy where they're doing totally unexpected and unexplained things that are beyond human understanding, is totally false. Since this is controversial and seemingly most HN folks can't hold a conversation with any nuance, if you don't use the word "shape" in your response to this post, I'm simply going to point out that you didn't read the post you're responding to, and therefore shouldn't be responding. If you stop reading at the first chance you see to correct something, go away--you're dragging down the level of the conversation.

aquariusDue 3y ago
I agree with you and hopefully once the hype dies down a year or two from now we will see how LLMs can actually shape the tech landscape (if at all significantly).
albertzeyer 3y ago
> the idea that larger models can do more than smaller models, which should be a surprise to no one. Actually this was quite a surprise to a lot of people, since the whole race to scaling up began, with GPT2 or so. It was totally not obvious that you can scale up the model (and also training) and it would improve the performance. Many (most?) people thought there would be some limit, and we were close to that limit with 100M-500M params or so. Then GPT2 came. And it was a surprise to a lot of people, that scaling up works so well. But then the question remained, is the limit reached now, or not, or is there any? The scaling laws appeared, and seemed to indicate that there really is no limit. Still, GPT3 and then GPT4 were still surprising to people, that it really got better and better. But the question still remains, is there a limit? If there is no limit, it means we can easily surpass human intelligence by just scaling up further. Maybe the limit is just always current technical hardware limitations and cost.
- kerkeslager 3y ago
  You didn't read the post you're responding to, and shouldn't be responding. In fact, you didn't read the part of the post you quoted, where I said it should be a surprise to no one. But, unsurprisingly, the sort of people who stop reading at the first chance they see to correct something, are easily surprised, since actually understanding LLMs would require actually doing some nuanced reading.
  - pmoriarty 3y ago
    Please consider HN's Guidelines[1] when replying. In particular: - Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". - Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. - Please don't fulminate. Please don't sneer, including at the rest of the community. [1] - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - kerkeslager 3y ago
      > Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". This guideline is likely one of the main reasons Hacker News comments are so simultaneously overconfident and undereducated. If you want HN to be a safe place for people interrupting informed conversation with whatever nonsense pops into their head, fine, but I don't want that, and until that guideline becomes a rule, I'm not going to be following it. People should read what they're responding to before responding. Note, in this case, it's objectively clear that the person did not read my post--I put something in my post to prove that fact. This isn't just snark. > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. Is it kind to jump in at the first opportunity to correct someone without reading what they've said? Is it kind to spread misinformation that causes societal harm? This is a very shallow idea of kindness.
      - mustacheemperor 3y ago
byby 3y ago
>"Emergent abilities" in that paper is an extraordinarily poor communication of the idea that larger models can do more than smaller models, which should be a surprise to no one. Nothing on the face of the earth can be a sum of more than their parts. It's called conservation of mass and conservation of energy. So based off of your logic how can the term "emergent ability" even exist? Why do we even use the term? You're getting hung up on a language quirk. Abilities aren't being built out of thin air, the term "emergent ability" is just an expression for "unexpected abilities". Even you as a human are a sum of it's parts. There's no nonsense being spread here. Experts and eminent researchers including the father of modern AI (Hinton) ALL use the term "emergent abilities". Are you an expert? Why is your opinion on this "spreading of nonsense" and why is it directly contradictory to expert opinions? Clearly you have logic that these experts and researchers haven't thought about. Can you spell it out step by step exactly what makes you more utterly clear about the "nonsense" then the experts who in actuality claim they don't fully understand what's going on? I'm not being sarcastic here, there is a huge contingent of people on HN who are totally dismissing LLMs and it would be a disservice not for them and you to not allow them to spell out their logic clearly. But what I'm seeing in your paragraph is just an investigation on vocabulary on what is meant by "emergent abilities"
deleted 3y ago
[deleted]
xcv123 3y ago
[flagged]

aquariusDue 3y ago

I agree with you and hopefully once the hype dies down a year or two from now we will see how LLMs can actually shape the tech landscape (if at all significantly).

albertzeyer 3y ago

> the idea that larger models can do more than smaller models, which should be a surprise to no one. Actually this was quite a surprise to a lot of people, since the whole race to scaling up began, with GPT2 or so. It was totally not obvious that you can scale up the model (and also training) and it would improve the performance. Many (most?) people thought there would be some limit, and we were close to that limit with 100M-500M params or so. Then GPT2 came. And it was a surprise to a lot of people, that scaling up works so well. But then the question remained, is the limit reached now, or not, or is there any? The scaling laws appeared, and seemed to indicate that there really is no limit. Still, GPT3 and then GPT4 were still surprising to people, that it really got better and better. But the question still remains, is there a limit? If there is no limit, it means we can easily surpass human intelligence by just scaling up further. Maybe the limit is just always current technical hardware limitations and cost.

kerkeslager 3y ago
You didn't read the post you're responding to, and shouldn't be responding. In fact, you didn't read the part of the post you quoted, where I said it should be a surprise to no one. But, unsurprisingly, the sort of people who stop reading at the first chance they see to correct something, are easily surprised, since actually understanding LLMs would require actually doing some nuanced reading.
- pmoriarty 3y ago
  Please consider HN's Guidelines[1] when replying. In particular: - Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". - Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. - Please don't fulminate. Please don't sneer, including at the rest of the community. [1] - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - kerkeslager 3y ago
    > Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". This guideline is likely one of the main reasons Hacker News comments are so simultaneously overconfident and undereducated. If you want HN to be a safe place for people interrupting informed conversation with whatever nonsense pops into their head, fine, but I don't want that, and until that guideline becomes a rule, I'm not going to be following it. People should read what they're responding to before responding. Note, in this case, it's objectively clear that the person did not read my post--I put something in my post to prove that fact. This isn't just snark. > Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. Is it kind to jump in at the first opportunity to correct someone without reading what they've said? Is it kind to spread misinformation that causes societal harm? This is a very shallow idea of kindness.
    - mustacheemperor 3y ago

byby 3y ago

>"Emergent abilities" in that paper is an extraordinarily poor communication of the idea that larger models can do more than smaller models, which should be a surprise to no one. Nothing on the face of the earth can be a sum of more than their parts. It's called conservation of mass and conservation of energy. So based off of your logic how can the term "emergent ability" even exist? Why do we even use the term? You're getting hung up on a language quirk. Abilities aren't being built out of thin air, the term "emergent ability" is just an expression for "unexpected abilities". Even you as a human are a sum of it's parts. There's no nonsense being spread here. Experts and eminent researchers including the father of modern AI (Hinton) ALL use the term "emergent abilities". Are you an expert? Why is your opinion on this "spreading of nonsense" and why is it directly contradictory to expert opinions? Clearly you have logic that these experts and researchers haven't thought about. Can you spell it out step by step exactly what makes you more utterly clear about the "nonsense" then the experts who in actuality claim they don't fully understand what's going on? I'm not being sarcastic here, there is a huge contingent of people on HN who are totally dismissing LLMs and it would be a disservice not for them and you to not allow them to spell out their logic clearly. But what I'm seeing in your paragraph is just an investigation on vocabulary on what is meant by "emergent abilities"

deleted 3y ago

[deleted]

xcv123 3y ago

[flagged]