5 ms·

No we don't

by spion 4y ago

No we don't

kerkeslager 4y ago
> No we don't Then maybe you should listen to people who do instead of speculating.
- baq 4y ago
  How do you know how something with 7, 14 or 135 billion parameters work?
  - kerkeslager 4y ago
    The same way you know how to compute the average of 135 billion numbers. You can't look at all 135 billion numbers in the input data, but you can easily understand that preference and originality aren't going to emerge from computing averages. Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student.
    - mynameisvlad 4y ago
      > Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student. Then by all means, please share with the class. If it’s so easy a third year could understand it then an expert such as yourself should find it mind-numbingly easy to explain it to everyone else.
      - deleted 4y ago
        [deleted]
      - kerkeslager 4y ago
        I'm not an expert, and I'm not going to write you a college course curriculum. However, I'll be happy to point you to some resources such as this: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY
        spion 4y ago
        This is how NNs are trained, not how they work. This is a good analogy that illustrates the problem with not making the distinction https://twitter.com/nearcyan/status/1632661647226462211 https://twitter.com/nearcyan/status/1632661647226462211
        kerkeslager 4y ago
        Nothing in that tweet is in any way related to anything I've said. It's certainly not about the distinction between how NNs are trained and how they work. The way NNs are trained is part (and obviously only part) of how they work. I'm well aware that LLMs are not simple. Remember when I said you'd need around two years of a college CS degree to understand it, and elsewhere where I said that it could be a college course? The area between "simple" and "beyond human understanding" is pretty large, though. Your description of "emergent capabilities" so far is pointing to a paper which says that putting larger data into a model produces different results, which is, you know, obvious. Calling those differences "emergent capabilities" is extraordinarily poor communication. What I mean when I say "emergent capabilities" is capabilities which aren't explained by the input data and program code, and you have yet to present a single one. Certainly nothing that could be called originality or preference, which were my two original examples of what ChatGPT doesn't have.
        wrycoder 4y ago
        We don’t really know how the emergent behavior arises, because the CNN weights are not programmed by us.
        kerkeslager 4y ago
        Again, what "emergent behavior"? By which I mean, what behaviors do these models have which are not easily explained by the program design and inputs? The CNN weights are not programmed by us, they're programmed by a program which is programmed by us. This level of indirection doesn't suddenly mean we don't understand anything. Another way of thinking of it: all we have is a program written by us, which takes the training data and a prompt as inputs, and spits out an output. The CNN weights are just a compressed cache of the training data part of the inputs so that you don't have to train the model for every prompt.
    - squeaky-clean 4y ago
      This is like saying a human brain put into a blender is still a human brain. Also you can have 2 different models with the exact same number of parameters and the exact same architecture with different weights and get 2 completely different AI's. Look at all the Stable Diffusion variants. There's one that will try to make any prompt you give it into porn.
      - kerkeslager 4y ago
        > This is like saying a human brain put into a blender is still a human brain. I'm not saying the inputs and outputs are the same, I'm saying that the outputs cannot contain anything that isn't in the inputs. > Also you can have 2 different models with the exact same number of parameters and the exact same architecture with different weights and get 2 completely different AI's. It sounds like you understand why that's happening (different weights), so this isn't an effective argument that we don't understand the programs.
  - frutiger 4y ago
    How do you know how a box of gas (~10^23 or ~100 000 billion billion billion parameters) will react when you squeeze it? https://en.wikipedia.org/wiki/Statistical_mechanics https://en.wikipedia.org/wiki/Statistical_mechanics.
    - baq 4y ago
      You’re making my point for me, thanks.
  - danaris 4y ago
    How do you know how something with 5.3 trillion transistors works? If it's Micron's 2TB 3D-stacked NAND, we know exactly how it works, because we understand how it was made. Just putting a very high number on something doesn't mean it's automatically sentient, or even incomprehensible. Just because we don't know the precise means by which ChatGPT arrives at one particular answer to a prompt doesn't mean we don't understand the underlying computations and data structures that make it up, and that they don't add up to sentience any more than Eliza does.
    - spion 4y ago
      We don't understand them. There is a key difference between building something from parts and running gradient descent to automatically find the network connectivity. That difference is that we don't understand the final result at all.
      - throwaway290 4y ago
        We understand exactly how it works. It just works in such a way that we cannot predict the outcome, which makes it pretty bad for many applications. If you can't explain how it works doesn't mean it's not understood how it works.
        spion 4y ago
        We really don't know why LLMs have the most of their emergent capabilities. The entire GPT-3 paper was about being surprised about it.
        throwaway290 4y ago
        Maybe we don't disagree then. We know how LLM work. Parameters. Training data. Random number generator. That stuff. We don't know why it outputs what it outputs because rngs are notoriously unpredictable and we know it. So we are surprised but that in itself is unsurprising.
- spion 4y ago
  I meant that nobody knows. There is a big difference between knowing how the models are trained and knowing how they actually work. This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all.
  - kerkeslager 4y ago
    > I meant that nobody knows. I know what you mean, and I'm saying you're wrong. > This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Sorry, which emergent capabilities are you talking about? You're trying to slip in that there even are emergent capabilities, when in fact there's literally nothing unexpected which has emerged. Given how LLMs work, literally everything they do is exactly what we'd expect them to do. The problem of explainability is twofold: 1) Connecting the input data with the output data, so a human can see what input data produced the output data. This is actually pretty easy to do naively, but it's hard to do performantly. 2) Presenting that data in a way that is meaningful to a non-technical user. Note that both of these are just about being able to audit which input data produced which output data. Nothing here is about how the output data was produced from the input data: we understand that pretty clearly, it's just that the process isn't reversible. > Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all. There are some things we can't understand about the final program, but there are still pretty understandable bounds on what the final program can do. For example, it cannot produce something from nothing: any output the program produces must strictly be a combination of its inputs. From this we can tell that no program produced is capable of originality. Likewise, I'm not even sure what argument you could possibly make that it would be capable of preference: any preferences you might perceive are obviously representative of the preferences of the humans who created the input data. As I've said elsewhere, just because you use a program to take the average of billions of integers which you can't possibly understand all of, doesn't mean you don't understand what the "average()" function does. Obviously LLMs are a lot more complex than an average, but they aren't beyond human understanding.

No we don't

by spion 4y ago

No we don't

kerkeslager 4y ago

> No we don't Then maybe you should listen to people who do instead of speculating.

baq 4y ago
How do you know how something with 7, 14 or 135 billion parameters work?
- kerkeslager 4y ago
  The same way you know how to compute the average of 135 billion numbers. You can't look at all 135 billion numbers in the input data, but you can easily understand that preference and originality aren't going to emerge from computing averages. Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student.
  - mynameisvlad 4y ago
    > Obviously the function of an LLM is a lot more complicated than the "average()" function, but it's not beyond human understanding. I'd venture it can be understood by an average 3rd-year undergraduate CS student. Then by all means, please share with the class. If it’s so easy a third year could understand it then an expert such as yourself should find it mind-numbingly easy to explain it to everyone else.
    - deleted 4y ago
      [deleted]
    - kerkeslager 4y ago
      I'm not an expert, and I'm not going to write you a college course curriculum. However, I'll be happy to point you to some resources such as this: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY
      - spion 4y ago
        This is how NNs are trained, not how they work. This is a good analogy that illustrates the problem with not making the distinction https://twitter.com/nearcyan/status/1632661647226462211 https://twitter.com/nearcyan/status/1632661647226462211
        kerkeslager 4y ago
        Nothing in that tweet is in any way related to anything I've said. It's certainly not about the distinction between how NNs are trained and how they work. The way NNs are trained is part (and obviously only part) of how they work. I'm well aware that LLMs are not simple. Remember when I said you'd need around two years of a college CS degree to understand it, and elsewhere where I said that it could be a college course? The area between "simple" and "beyond human understanding" is pretty large, though. Your description of "emergent capabilities" so far is pointing to a paper which says that putting larger data into a model produces different results, which is, you know, obvious. Calling those differences "emergent capabilities" is extraordinarily poor communication. What I mean when I say "emergent capabilities" is capabilities which aren't explained by the input data and program code, and you have yet to present a single one. Certainly nothing that could be called originality or preference, which were my two original examples of what ChatGPT doesn't have.
        wrycoder 4y ago
        We don’t really know how the emergent behavior arises, because the CNN weights are not programmed by us.
        kerkeslager 4y ago
        Again, what "emergent behavior"? By which I mean, what behaviors do these models have which are not easily explained by the program design and inputs? The CNN weights are not programmed by us, they're programmed by a program which is programmed by us. This level of indirection doesn't suddenly mean we don't understand anything. Another way of thinking of it: all we have is a program written by us, which takes the training data and a prompt as inputs, and spits out an output. The CNN weights are just a compressed cache of the training data part of the inputs so that you don't have to train the model for every prompt.
  - squeaky-clean 4y ago
    This is like saying a human brain put into a blender is still a human brain. Also you can have 2 different models with the exact same number of parameters and the exact same architecture with different weights and get 2 completely different AI's. Look at all the Stable Diffusion variants. There's one that will try to make any prompt you give it into porn.
    - kerkeslager 4y ago
      > This is like saying a human brain put into a blender is still a human brain. I'm not saying the inputs and outputs are the same, I'm saying that the outputs cannot contain anything that isn't in the inputs. > Also you can have 2 different models with the exact same number of parameters and the exact same architecture with different weights and get 2 completely different AI's. It sounds like you understand why that's happening (different weights), so this isn't an effective argument that we don't understand the programs.
- frutiger 4y ago
  How do you know how a box of gas (~10^23 or ~100 000 billion billion billion parameters) will react when you squeeze it? https://en.wikipedia.org/wiki/Statistical_mechanics https://en.wikipedia.org/wiki/Statistical_mechanics.
  - baq 4y ago
    You’re making my point for me, thanks.
- danaris 4y ago
  How do you know how something with 5.3 trillion transistors works? If it's Micron's 2TB 3D-stacked NAND, we know exactly how it works, because we understand how it was made. Just putting a very high number on something doesn't mean it's automatically sentient, or even incomprehensible. Just because we don't know the precise means by which ChatGPT arrives at one particular answer to a prompt doesn't mean we don't understand the underlying computations and data structures that make it up, and that they don't add up to sentience any more than Eliza does.
  - spion 4y ago
    We don't understand them. There is a key difference between building something from parts and running gradient descent to automatically find the network connectivity. That difference is that we don't understand the final result at all.
    - throwaway290 4y ago
      We understand exactly how it works. It just works in such a way that we cannot predict the outcome, which makes it pretty bad for many applications. If you can't explain how it works doesn't mean it's not understood how it works.
      - spion 4y ago
        We really don't know why LLMs have the most of their emergent capabilities. The entire GPT-3 paper was about being surprised about it.
        throwaway290 4y ago
        Maybe we don't disagree then. We know how LLM work. Parameters. Training data. Random number generator. That stuff. We don't know why it outputs what it outputs because rngs are notoriously unpredictable and we know it. So we are surprised but that in itself is unsurprising.
spion 4y ago
I meant that nobody knows. There is a big difference between knowing how the models are trained and knowing how they actually work. This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all.
- kerkeslager 4y ago
  > I meant that nobody knows. I know what you mean, and I'm saying you're wrong. > This is a basic problem of machine learning (explainability) and we're nowhere understanding how or why LLMs have the emergent capabilities that they do (we're just getting started with that research) Sorry, which emergent capabilities are you talking about? You're trying to slip in that there even are emergent capabilities, when in fact there's literally nothing unexpected which has emerged. Given how LLMs work, literally everything they do is exactly what we'd expect them to do. The problem of explainability is twofold: 1) Connecting the input data with the output data, so a human can see what input data produced the output data. This is actually pretty easy to do naively, but it's hard to do performantly. 2) Presenting that data in a way that is meaningful to a non-technical user. Note that both of these are just about being able to audit which input data produced which output data. Nothing here is about how the output data was produced from the input data: we understand that pretty clearly, it's just that the process isn't reversible. > Gradient descent training is just a little more efficient method of essentially permuting program code at random until you get something that passes all cases. Doesn't mean you understand the final program at all. There are some things we can't understand about the final program, but there are still pretty understandable bounds on what the final program can do. For example, it cannot produce something from nothing: any output the program produces must strictly be a combination of its inputs. From this we can tell that no program produced is capable of originality. Likewise, I'm not even sure what argument you could possibly make that it would be capable of preference: any preferences you might perceive are obviously representative of the preferences of the humans who created the input data. As I've said elsewhere, just because you use a program to take the average of billions of integers which you can't possibly understand all of, doesn't mean you don't understand what the "average()" function does. Obviously LLMs are a lot more complex than an average, but they aren't beyond human understanding.