5 ms·

People are impressed because its black fucking magic my friend.

Jtsummers 3y ago
Pretty sure it's not magic, black fucking or otherwise, just really complex math, large data sets, and very fast computers.
- esafak 3y ago
  Magic is that which can not be explained. Could you explain GPT-4's results if you saw them a few years ago?
  - Jtsummers 3y ago
    Probably, I mean I first studied ANNs over two decades ago and had conversations with people about them prior to starting college in the 90s who had developed solutions with them in the 80s (obviously severely computationally constrained in those days compared to today). So still not magic. To be very blunt: If you believe that GPT and the like are magic, then you're not thinking clearly. You're blinded by the results (which are impressive).
    - esafak 3y ago
      ANNs in the 90s were nothing like this. They were not even like this a few years ago. And neither were HMMs. There is an emergent human quality to them because they have approached our abilities; the comparison is tenable, whereas before it was not.
      - Jtsummers 3y ago
        Scale and topology. That's the difference between ANNs of the 90s and 00s and today. They're still based on the same fundamental principles and doing (modulo scale) the same fundamental things: classification, prediction, generation. It is not magic, it has never been magic.
        esafak 3y ago
        Scale and topology also differentiates all living creatures. In fact, our topologies are more similar than that of various neural networks, due to evolution. "Classification, prediction, generation" encapsulates everything we do too. So I guess we are not magical either.
        Jtsummers 3y ago
        > So I guess we are not magical either. We aren't, and I haven't said otherwise.
        esafak 3y ago
        We have a different understanding of magic. Say if someone pressed a button and a human-like thing emerged out of a machine, I would call that pretty magical. Even if it was DNA-based, which we "understand", or ran an ML model, which we "understand". This is something that never come close to being done. Yet I think you would not find it magical. Einstein found wonder in the simplicity of a circle. What do you find magical?
        Jtsummers 3y ago
        You initially wrote: > Magic is that which can not be explained. You have now redefined what you mean by "magic" as "that which inspires wonder". Changing definitions after a series of comments is a pretty poor way to have a discussion.
        esafak 3y ago
        That which can not be explained inspires wonder. That's basically what makes magic magic.
    - gyrovagueGeist 3y ago
      "It doesn't stop being magic just because you know how it works" - Terry Prachett
      - throwaway675309 3y ago
        Just because we have a pithy quote doesn't detract from the fact that we shouldn't go all starry eyed and bandy about terms which obfuscate rather than explore the mechanics of how things work. Calling something "black magic" is too close to all of the anthropomorphism bullshit that seems to travel like a miasma around LLMs.
        burnished 3y ago
        Magic is a garbage explanation for the function of a system, but the original context was some one expressing dismay at people being impressed and 'the thing resembles magic' is a great explanation for that.
  - kerkeslager 3y ago
    Setting aside the silliness of that definition of magic, there's a huge leap between "I can't explain it" and "It can't be explained". There are plenty of explanations of how LLMs work, by their creators, incidentally.
    - celestialcheese 3y ago
      Yet there are emergent behaviours from these LLMs that are both surprising and not immediately understood. [1][2][3] Everyone has theories, of course, but still pretty "magic" considering these behaviours weren't theorised in papers prior to observation. 1 - https://www.jasonwei.net/blog/emergence https://www.jasonwei.net/blog/emergence 2 - https://arxiv.org/pdf/2206.07682.pdf https://arxiv.org/pdf/2206.07682.pdf 3 - https://www.quantamagazine.org/the-unpredictable-abilities-emerging-from-large-ai-models-20230316/ https://www.quantamagazine.org/the-unpredictable-abilities-e...
      - kerkeslager 3y ago
        Don't cite stuff you didn't read or understand. [1] Is a summary of [2], by one of its authors, not a separate source. [2] Defines "emergent behaviors" in a way that you're clearly misunderstanding (because "emergent behaviors" is an extraordinarily poor way of communicating this--it's partly the fault of the researchers who chose this ambiguous language). All it's saying is that bigger models can do things that smaller models can't, which should be surprising to no one. It's NOT saying that the capabilities are anything more than the sum of the input data. [3] Is written by a journalist, not an AI researcher, and so it's limited by the things the journalist is excited about. The journalist, for example, downplays sections like, "The other, less sensational possibility, she said, is that what appears to be emergent may instead be the culmination of an internal, statistics-driven process that works through chain-of-thought-type reasoning. Large LLMs may simply be learning heuristics that are out of reach for those with fewer parameters or lower-quality data." If you're going to try to gather things from journalists rather than subject matter experts, you need to understand how journalists work, and how subject matter experts work, and look for paragraphs like that to understand what's actually happening.
- burnished 3y ago
  It is absolutely magic! Have you played around with it at all? It is potent to the point of invoking wonder and awe. But I can see from your other replies that your real objection appears to be use of the m word, which seems odd given its expressive power, but you do you.
gatkinso 3y ago
it does seem that way, and maybe it's better thought of that way. surely plenty of hn commenters are here to explain why its actually not magic...

Pretty sure it's not magic, black fucking or otherwise, just really complex math, large data sets, and very fast computers.

esafak 3y ago
Magic is that which can not be explained. Could you explain GPT-4's results if you saw them a few years ago?
- Jtsummers 3y ago
  Probably, I mean I first studied ANNs over two decades ago and had conversations with people about them prior to starting college in the 90s who had developed solutions with them in the 80s (obviously severely computationally constrained in those days compared to today). So still not magic. To be very blunt: If you believe that GPT and the like are magic, then you're not thinking clearly. You're blinded by the results (which are impressive).
  - esafak 3y ago
    ANNs in the 90s were nothing like this. They were not even like this a few years ago. And neither were HMMs. There is an emergent human quality to them because they have approached our abilities; the comparison is tenable, whereas before it was not.
    - Jtsummers 3y ago
      Scale and topology. That's the difference between ANNs of the 90s and 00s and today. They're still based on the same fundamental principles and doing (modulo scale) the same fundamental things: classification, prediction, generation. It is not magic, it has never been magic.
      - esafak 3y ago
        Scale and topology also differentiates all living creatures. In fact, our topologies are more similar than that of various neural networks, due to evolution. "Classification, prediction, generation" encapsulates everything we do too. So I guess we are not magical either.
        Jtsummers 3y ago
        > So I guess we are not magical either. We aren't, and I haven't said otherwise.
        esafak 3y ago
        We have a different understanding of magic. Say if someone pressed a button and a human-like thing emerged out of a machine, I would call that pretty magical. Even if it was DNA-based, which we "understand", or ran an ML model, which we "understand". This is something that never come close to being done. Yet I think you would not find it magical. Einstein found wonder in the simplicity of a circle. What do you find magical?
        Jtsummers 3y ago
        You initially wrote: > Magic is that which can not be explained. You have now redefined what you mean by "magic" as "that which inspires wonder". Changing definitions after a series of comments is a pretty poor way to have a discussion.
        esafak 3y ago
        That which can not be explained inspires wonder. That's basically what makes magic magic.
  - gyrovagueGeist 3y ago
    "It doesn't stop being magic just because you know how it works" - Terry Prachett
    - throwaway675309 3y ago
      Just because we have a pithy quote doesn't detract from the fact that we shouldn't go all starry eyed and bandy about terms which obfuscate rather than explore the mechanics of how things work. Calling something "black magic" is too close to all of the anthropomorphism bullshit that seems to travel like a miasma around LLMs.
      - burnished 3y ago
        Magic is a garbage explanation for the function of a system, but the original context was some one expressing dismay at people being impressed and 'the thing resembles magic' is a great explanation for that.
- kerkeslager 3y ago
  Setting aside the silliness of that definition of magic, there's a huge leap between "I can't explain it" and "It can't be explained". There are plenty of explanations of how LLMs work, by their creators, incidentally.
  - celestialcheese 3y ago
    Yet there are emergent behaviours from these LLMs that are both surprising and not immediately understood. [1][2][3] Everyone has theories, of course, but still pretty "magic" considering these behaviours weren't theorised in papers prior to observation. 1 - https://www.jasonwei.net/blog/emergence https://www.jasonwei.net/blog/emergence 2 - https://arxiv.org/pdf/2206.07682.pdf https://arxiv.org/pdf/2206.07682.pdf 3 - https://www.quantamagazine.org/the-unpredictable-abilities-emerging-from-large-ai-models-20230316/ https://www.quantamagazine.org/the-unpredictable-abilities-e...
    - kerkeslager 3y ago
      Don't cite stuff you didn't read or understand. [1] Is a summary of [2], by one of its authors, not a separate source. [2] Defines "emergent behaviors" in a way that you're clearly misunderstanding (because "emergent behaviors" is an extraordinarily poor way of communicating this--it's partly the fault of the researchers who chose this ambiguous language). All it's saying is that bigger models can do things that smaller models can't, which should be surprising to no one. It's NOT saying that the capabilities are anything more than the sum of the input data. [3] Is written by a journalist, not an AI researcher, and so it's limited by the things the journalist is excited about. The journalist, for example, downplays sections like, "The other, less sensational possibility, she said, is that what appears to be emergent may instead be the culmination of an internal, statistics-driven process that works through chain-of-thought-type reasoning. Large LLMs may simply be learning heuristics that are out of reach for those with fewer parameters or lower-quality data." If you're going to try to gather things from journalists rather than subject matter experts, you need to understand how journalists work, and how subject matter experts work, and look for paragraphs like that to understand what's actually happening.
burnished 3y ago
It is absolutely magic! Have you played around with it at all? It is potent to the point of invoking wonder and awe. But I can see from your other replies that your real objection appears to be use of the m word, which seems odd given its expressive power, but you do you.

gatkinso 3y ago

it does seem that way, and maybe it's better thought of that way. surely plenty of hn commenters are here to explain why its actually not magic...