3 ms·
>> Just because we know the definition of something, doesn't mean we know all there is to know about it. Every mathematician knows that. Well, I don't know abo
by YeGoblynQueenne 4y ago
>> Just because we know the definition of something, doesn't mean we know all there is to know about it. Every mathematician knows that.
Well, I don't know about mathematicians. I'm a computer scientist. I study computers. And I know that the behaviour of a computer is entirely determined by its design, and its inputs. If a computer is doing something that is not in its design to do, then that's because there is an error in the implementation of its design. And if it's difficult, or impossible, to anticipate the behaviour of a computer, that's because it's programmed to do something unpredictable- like in my RNG example above.
Computers are machines. Programs are other machines, running on computers. There are no spirits in any level of a computer's functionality, no magick, and no metaphysical mysteries to solve.
Maybe that's not how it works in other sciences, that study the natural world, where there are too many unknowns. But in computers there are no unknowns. We make them, and we know how they work.
Regarding LLLMs, we know why they're so good at what they do, because we know what they do, because we are the ones who made them to do it, and we know that what they do gets better the larger the number of parameters and the larger the data they're trained on. We've known that for decades.
And as to "emergent behaviour", that is just asking for a mystery to be there, when no mystery is anywhere near the whole thing. "Emergent behaviour" just means what I say above, that there's a complex system that displays behaviour that is difficult to predict, even if its design principles are very well known.
There's a time to wonder, and wonder is the seed of science. But to wonder about a machine that is doing what it's designed to do, is not the wonder that leads to science. That kind of wonder, well, it reminds me of the old joke:
They ask a mystic what is the greatest mystery in nature.
"The thermos!" he says. "It keeps my tea hot in the winter, but cold in the summer. That little bottle! How does it know?"
- auggierose 4y ago> Well, I don't know about mathematicians. I'm a computer scientist. Does an LLM gain "consciousness", like ours? I doubt it. There is a doubt here though, because while I know how an LLM works, I don't know how we work. Can an LLM, through training, gain an "understanding" that rivals our notion of consciousness and can in fact simulate it, purely through mathematical models it forms? Well, you are not a mathematician, so how would you know? You say we have known for decades that LLMs would work? Liar.
- YeGoblynQueenne 4y ago>> You say we have known for decades that LLMs would work? Liar. I beg your pardon? Would you not prefer to ask me to clarify what I meant, instead of jumping to conclusions so you can call me names?
- auggierose 4y agoIt is pretty clear what you meant. Did you not say you know why LLMs work, and that we have known so for decades? But feel free to clarify, in case I misunderstood!
- YeGoblynQueenne 4y agoYou don't have it in you to have an exchange in good faith, do you? No need for me to bother then.
- auggierose 4y agoI very much appreciate an exchange in good faith. Is that what you are doing?
- diziet 4y agoWe are not close to determining the answer to Busy Beaver(5,5). Not every problem is decidable.
- YeGoblynQueenne 4y agoFamously so. But what does undecidability have to do with language modelling? There's an obvious misunderstanding I could make of your comment but I hate it when people do that to me, so can you please clarify? Also: is your username an allusion to Diziet Sma, from the Iain M. Banks stories?
- diziet 4y agoI referenced a class of problems easily defined via mathematical tools and notation that are seemingly not possible to solve in our universe as an adjacent example for difficult problems that border on our definition of understanding. A problem doesn't have to be undecidable to be realistically unsolvable. For example, I don't think that we can compute TREE(3). Or Graham's number, which is quite a bit smaller. TREE(3) is computable and defined. However, we can't fit it into our universe even if converted every atom into computronium. So do we understand TREE(3)? Surely yes, though at the same time no? How about Rayo's number? As such, I am merely making a statement that at certain scales, to our brains, quantity has a quality of its own. Yes, it is. I've used it for about 15 years now.
- diziet 4y agoI also took a bit of time to look up a more well formulated statement on this -- specifically, Rice's theorem https://en.wikipedia.org/wiki/Rice%27s_theorem https://en.wikipedia.org/wiki/Rice%27s_theorem