5 ms·

> Firstly, no need to get all prickly. In the absence of hard evidence everyone is entitled to their own opinion. There is not an absence of hard evidence. We

> Firstly, no need to get all prickly. In the absence of hard evidence everyone is entitled to their own opinion. There is not an absence of hard evidence. We know exactly how LLMs work, and it's clear that you don't, although there's nothing stopping you from learning since the information is widely available. Not a single line of code in these systems is devoted to preference or originality, and there is no vector by which preference or originality could emerge from these systems. "Everyone is entitled to their own opinion" applies to subjective things like preferences, not to objective facts. Your belief that LLMs are even a step in the direction of "potentially conscious entities" isn't an opinion, it's an objectively incorrect belief. You can't reasonably expect people to treat your uninformed and incorrect speculation with the same respect they would give to an opinion or, god forbid, an actual fact. You're just wrong, and there's nothing "prickly" about telling you that.

spion 4y ago
No we don't
- kerkeslager 4y ago
  > No we don't Then maybe you should listen to people who do instead of speculating.
  - baq 4y ago
    How do you know how something with 7, 14 or 135 billion parameters work?
    - kerkeslager 4y ago
      The same way you know how to compute the average of 135 billion numbers. You can't look at all 135 billion numbers in the input data, but you can easily understand that preference and originality aren't going to emerge from computing averages. Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student.
      - mynameisvlad 4y ago
        > Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student. Then by all means, please share with the class. If it’s so easy a third year could understand it then an expert such as yourself should find it mind-numbingly easy to explain it to everyone else.
        deleted 4y ago
        [deleted]
        kerkeslager 4y ago
        I'm not an expert, and I'm not going to write you a college course curriculum. However, I'll be happy to point you to some resources such as this: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY
        spion 4y ago
        This is how NNs are trained, not how they work. This is a good analogy that illustrates the problem with not making the distinction https://twitter.com/nearcyan/status/1632661647226462211 https://twitter.com/nearcyan/status/1632661647226462211
        kerkeslager 4y ago
        Nothing in that tweet is in any way related to anything I've said. It's certainly not about the distinction between how NNs are trained and how they work. The way NNs are trained is part (and obviously only part) of how they work. I'm well aware that LLMs are not simple. Remember when I said you'd need around two years of a college CS degree to understand it, and elsewhere where I said that it could be a college course? The area between "simple" and "beyond human understanding" is pretty large, though. Your description of "emergent capabilities" so far is pointing to a paper which says that putting larger data into a model produces different results, which is, you know, obvious. Calling those differences "emergent capabilities" is extraordinarily poor communication. What I mean when I say "emergent capabilities" is capabilities which aren't explained by the input data and program code, and you have yet to present a single one. Certainly nothing that could be called originality or preference, which were my two original examples of what ChatGPT doesn't have.
    - frutiger 4y ago
      How do you know how a box of gas (~10^23 or ~100 000 billion billion billion parameters) will react when you squeeze it? https://en.wikipedia.org/wiki/Statistical_mechanics https://en.wikipedia.org/wiki/Statistical_mechanics.
      - baq 4y ago
        You’re making my point for me, thanks.
    - danaris 4y ago
      How do you know how something with 5.3 trillion transistors works? If it's Micron's 2TB 3D-stacked NAND, we know exactly how it works, because we understand how it was made. Just putting a very high number on something doesn't mean it's automatically sentient, or even incomprehensible. Just because we don't know the precise means by which ChatGPT arrives at one particular answer to a prompt doesn't mean we don't understand the underlying computations and data structures that make it up, and that they don't add up to sentience any more than Eliza does.
      - spion 4y ago
        We don't understand them. There is a key difference between building something from parts and running gradient descent to automatically find the network connectivity. That difference is that we don't understand the final result at all.
        throwaway290 4y ago
        We understand exactly how it works. It just works in such a way that we cannot predict the outcome, which makes it pretty bad for many applications. If you can't explain how it works doesn't mean it's not understood how it works.
        spion 4y ago
        We really don't know why LLMs have the most of their emergent capabilities. The entire GPT-3 paper was about being surprised about it.
        throwaway290 4y ago
        Maybe we don't disagree then. We know how LLM work. Parameters. Training data. Random number generator. That stuff. We don't know why it outputs what it outputs because rngs are notoriously unpredictable and we know it. So we are surprised but that in itself is unsurprising.
  - spion 4y ago
    I meant that nobody knows. There is a big difference between knowing how the models are trained and knowing how they actually work. This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all.
    - kerkeslager 4y ago
      > I meant that nobody knows. I know what you mean, and I'm saying you're wrong. > This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Sorry, which emergent capabilities are you talking about? You're trying to slip in that there even are emergent capabilities, when in fact there's literally nothing unexpected which has emerged. Given how LLMs work, literally everything they do is exactly what we'd expect them to do. The problem of explainability is twofold: 1) Connecting the input data with the output data, so a human can see what input data produced the output data. This is actually pretty easy to do naively, but it's hard to do performantly. 2) Presenting that data in a way that is meaningful to a non-technical user. Note that both of these are just about being able to audit which input data produced which output data. Nothing here is about how the output data was produced from the input data: we understand that pretty clearly, it's just that the process isn't reversible. > Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all. There are some things we can't understand about the final program, but there are still pretty understandable bounds on what the final program can do. For example, it cannot produce something from nothing: any output the program produces must strictly be a combination of its inputs. From this we can tell that no program produced is capable of originality. Likewise, I'm not even sure what argument you could possibly make that it would be capable of preference: any preferences you might perceive are obviously representative of the preferences of the humans who created the input data. As I've said elsewhere, just because you use a program to take the average of billions of integers which you can't possibly understand all of, doesn't mean you don't understand what the "average()" function does. Obviously LLMs are a lot more complex than an average, but they aren't beyond human understanding.

spion 4y ago

No we don't

kerkeslager 4y ago
> No we don't Then maybe you should listen to people who do instead of speculating.
- baq 4y ago
  How do you know how something with 7, 14 or 135 billion parameters work?
  - kerkeslager 4y ago
    The same way you know how to compute the average of 135 billion numbers. You can't look at all 135 billion numbers in the input data, but you can easily understand that preference and originality aren't going to emerge from computing averages. Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student.
    - mynameisvlad 4y ago
      > Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student. Then by all means, please share with the class. If it’s so easy a third year could understand it then an expert such as yourself should find it mind-numbingly easy to explain it to everyone else.
      - deleted 4y ago
        [deleted]
      - kerkeslager 4y ago
        I'm not an expert, and I'm not going to write you a college course curriculum. However, I'll be happy to point you to some resources such as this: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY
        spion 4y ago
        This is how NNs are trained, not how they work. This is a good analogy that illustrates the problem with not making the distinction https://twitter.com/nearcyan/status/1632661647226462211 https://twitter.com/nearcyan/status/1632661647226462211
        kerkeslager 4y ago
        Nothing in that tweet is in any way related to anything I've said. It's certainly not about the distinction between how NNs are trained and how they work. The way NNs are trained is part (and obviously only part) of how they work. I'm well aware that LLMs are not simple. Remember when I said you'd need around two years of a college CS degree to understand it, and elsewhere where I said that it could be a college course? The area between "simple" and "beyond human understanding" is pretty large, though. Your description of "emergent capabilities" so far is pointing to a paper which says that putting larger data into a model produces different results, which is, you know, obvious. Calling those differences "emergent capabilities" is extraordinarily poor communication. What I mean when I say "emergent capabilities" is capabilities which aren't explained by the input data and program code, and you have yet to present a single one. Certainly nothing that could be called originality or preference, which were my two original examples of what ChatGPT doesn't have.
  - frutiger 4y ago
    How do you know how a box of gas (~10^23 or ~100 000 billion billion billion parameters) will react when you squeeze it? https://en.wikipedia.org/wiki/Statistical_mechanics https://en.wikipedia.org/wiki/Statistical_mechanics.
    - baq 4y ago
      You’re making my point for me, thanks.
  - danaris 4y ago
    How do you know how something with 5.3 trillion transistors works? If it's Micron's 2TB 3D-stacked NAND, we know exactly how it works, because we understand how it was made. Just putting a very high number on something doesn't mean it's automatically sentient, or even incomprehensible. Just because we don't know the precise means by which ChatGPT arrives at one particular answer to a prompt doesn't mean we don't understand the underlying computations and data structures that make it up, and that they don't add up to sentience any more than Eliza does.
    - spion 4y ago
      We don't understand them. There is a key difference between building something from parts and running gradient descent to automatically find the network connectivity. That difference is that we don't understand the final result at all.
      - throwaway290 4y ago
        We understand exactly how it works. It just works in such a way that we cannot predict the outcome, which makes it pretty bad for many applications. If you can't explain how it works doesn't mean it's not understood how it works.
        spion 4y ago
        We really don't know why LLMs have the most of their emergent capabilities. The entire GPT-3 paper was about being surprised about it.
        throwaway290 4y ago
        Maybe we don't disagree then. We know how LLM work. Parameters. Training data. Random number generator. That stuff. We don't know why it outputs what it outputs because rngs are notoriously unpredictable and we know it. So we are surprised but that in itself is unsurprising.
- spion 4y ago
  I meant that nobody knows. There is a big difference between knowing how the models are trained and knowing how they actually work. This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all.
  - kerkeslager 4y ago
    > I meant that nobody knows. I know what you mean, and I'm saying you're wrong. > This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Sorry, which emergent capabilities are you talking about? You're trying to slip in that there even are emergent capabilities, when in fact there's literally nothing unexpected which has emerged. Given how LLMs work, literally everything they do is exactly what we'd expect them to do. The problem of explainability is twofold: 1) Connecting the input data with the output data, so a human can see what input data produced the output data. This is actually pretty easy to do naively, but it's hard to do performantly. 2) Presenting that data in a way that is meaningful to a non-technical user. Note that both of these are just about being able to audit which input data produced which output data. Nothing here is about how the output data was produced from the input data: we understand that pretty clearly, it's just that the process isn't reversible. > Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all. There are some things we can't understand about the final program, but there are still pretty understandable bounds on what the final program can do. For example, it cannot produce something from nothing: any output the program produces must strictly be a combination of its inputs. From this we can tell that no program produced is capable of originality. Likewise, I'm not even sure what argument you could possibly make that it would be capable of preference: any preferences you might perceive are obviously representative of the preferences of the humans who created the input data. As I've said elsewhere, just because you use a program to take the average of billions of integers which you can't possibly understand all of, doesn't mean you don't understand what the "average()" function does. Obviously LLMs are a lot more complex than an average, but they aren't beyond human understanding.