4 ms·

Just because we know the definition of something, doesn't mean we know all there is to know about it. Every mathematician knows that. Yes, we know the definitio

Just because we know the definition of something, doesn't mean we know all there is to know about it. Every mathematician knows that. Yes, we know the definition of GPT. But that it is so good at doing what it does, with such a freakingly simple method, one word after another, is emergent behaviour, and wasn't known to us before seeing it with our own eyes. We still don't know WHY it is so good at it. If we did, maybe we could construct one a hundred times smaller.

juve1996 4y ago
I don't think that matters as much as we think it does, though. There's a lot of internal things we don't know about how the brain works - but we can still describe the macro.
YeGoblynQueenne 4y ago
>> Just because we know the definition of something, doesn't mean we know all there is to know about it. Every mathematician knows that. Well, I don't know about mathematicians. I'm a computer scientist. I study computers. And I know that the behaviour of a computer is entirely determined by its design, and its inputs. If a computer is doing something that is not in its design to do, then that's because there is an error in the implementation of its design. And if it's difficult, or impossible, to anticipate the behaviour of a computer, that's because it's programmed to do something unpredictable- like in my RNG example above. Computers are machines. Programs are other machines, running on computers. There are no spirits in any level of a computer's functionality, no magick, and no metaphysical mysteries to solve. Maybe that's not how it works in other sciences, that study the natural world, where there are too many unknowns. But in computers there are no unknowns. We make them, and we know how they work. Regarding LLLMs, we know why they're so good at what they do, because we know what they do, because we are the ones who made them to do it, and we know that what they do gets better the larger the number of parameters and the larger the data they're trained on. We've known that for decades. And as to "emergent behaviour", that is just asking for a mystery to be there, when no mystery is anywhere near the whole thing. "Emergent behaviour" just means what I say above, that there's a complex system that displays behaviour that is difficult to predict, even if its design principles are very well known. There's a time to wonder, and wonder is the seed of science. But to wonder about a machine that is doing what it's designed to do, is not the wonder that leads to science. That kind of wonder, well, it reminds me of the old joke: They ask a mystic what is the greatest mystery in nature. "The thermos!" he says. "It keeps my tea hot in the winter, but cold in the summer. That little bottle! How does it know?"
- auggierose 4y ago
  > Well, I don't know about mathematicians. I'm a computer scientist. Does an LLM gain "consciousness", like ours? I doubt it. There is a doubt here though, because while I know how an LLM works, I don't know how we work. Can an LLM, through training, gain an "understanding" that rivals our notion of consciousness and can in fact simulate it, purely through mathematical models it forms? Well, you are not a mathematician, so how would you know? You say we have known for decades that LLMs would work? Liar.
  - YeGoblynQueenne 4y ago
    >> You say we have known for decades that LLMs would work? Liar. I beg your pardon? Would you not prefer to ask me to clarify what I meant, instead of jumping to conclusions so you can call me names?
    - auggierose 4y ago
      It is pretty clear what you meant. Did you not say you know why LLMs work, and that we have known so for decades? But feel free to clarify, in case I misunderstood!
      - YeGoblynQueenne 4y ago
        You don't have it in you to have an exchange in good faith, do you? No need for me to bother then.
        auggierose 4y ago
        I very much appreciate an exchange in good faith. Is that what you are doing?
- diziet 4y ago
  We are not close to determining the answer to Busy Beaver(5,5). Not every problem is decidable.

juve1996 4y ago

I don't think that matters as much as we think it does, though. There's a lot of internal things we don't know about how the brain works - but we can still describe the macro.

YeGoblynQueenne 4y ago

>> Just because we know the definition of something, doesn't mean we know all there is to know about it. Every mathematician knows that. Well, I don't know about mathematicians. I'm a computer scientist. I study computers. And I know that the behaviour of a computer is entirely determined by its design, and its inputs. If a computer is doing something that is not in its design to do, then that's because there is an error in the implementation of its design. And if it's difficult, or impossible, to anticipate the behaviour of a computer, that's because it's programmed to do something unpredictable- like in my RNG example above. Computers are machines. Programs are other machines, running on computers. There are no spirits in any level of a computer's functionality, no magick, and no metaphysical mysteries to solve. Maybe that's not how it works in other sciences, that study the natural world, where there are too many unknowns. But in computers there are no unknowns. We make them, and we know how they work. Regarding LLLMs, we know why they're so good at what they do, because we know what they do, because we are the ones who made them to do it, and we know that what they do gets better the larger the number of parameters and the larger the data they're trained on. We've known that for decades. And as to "emergent behaviour", that is just asking for a mystery to be there, when no mystery is anywhere near the whole thing. "Emergent behaviour" just means what I say above, that there's a complex system that displays behaviour that is difficult to predict, even if its design principles are very well known. There's a time to wonder, and wonder is the seed of science. But to wonder about a machine that is doing what it's designed to do, is not the wonder that leads to science. That kind of wonder, well, it reminds me of the old joke: They ask a mystic what is the greatest mystery in nature. "The thermos!" he says. "It keeps my tea hot in the winter, but cold in the summer. That little bottle! How does it know?"

auggierose 4y ago
> Well, I don't know about mathematicians. I'm a computer scientist. Does an LLM gain "consciousness", like ours? I doubt it. There is a doubt here though, because while I know how an LLM works, I don't know how we work. Can an LLM, through training, gain an "understanding" that rivals our notion of consciousness and can in fact simulate it, purely through mathematical models it forms? Well, you are not a mathematician, so how would you know? You say we have known for decades that LLMs would work? Liar.
- YeGoblynQueenne 4y ago
  >> You say we have known for decades that LLMs would work? Liar. I beg your pardon? Would you not prefer to ask me to clarify what I meant, instead of jumping to conclusions so you can call me names?
  - auggierose 4y ago
    It is pretty clear what you meant. Did you not say you know why LLMs work, and that we have known so for decades? But feel free to clarify, in case I misunderstood!
    - YeGoblynQueenne 4y ago
      You don't have it in you to have an exchange in good faith, do you? No need for me to bother then.
      - auggierose 4y ago
        I very much appreciate an exchange in good faith. Is that what you are doing?
diziet 4y ago
We are not close to determining the answer to Busy Beaver(5,5). Not every problem is decidable.