3 ms·

I agree LLMs are useful tools, but I am dismayed by a lot of cargo-culting around them, which happens because we don't understand them. I think we will have muc

by js8 18d ago

I agree LLMs are useful tools, but I am dismayed by a lot of cargo-culting around them, which happens because we don't understand them. I think we will have much better tools when we understand what is the expectation and what is the algorithm they run. Somebody else said that the magician analogy was poor. I like magic tricks, but it took many years of cultural change (influenced by people like Houdini, Randi, Penn & Teller) to stop illusionists (and mentalists) make claims they have supernatural abilities, or people believing it on their own (a magician pretending to be able to catch a bullet was shot by an audience member who didn't understand the distinction). It is detrimental, I think, to treat LLMs as if they have magical abilities ("superintelligence") rather than understanding they just run some clever algorithm. The fear for (programming) jobs comes from that framing; nobody fears of their job because of compilers, since compilers are understood. (And it actually runs against kind of "socialist" framing of the problem, which I agree with, that is why should people be worried about the jobs in the first place, when society is getting richer as a result of better tools?)

sanderjd 17d ago
I see what you're saying, but I don't really think understanding how they work is the problem. We do understand how they work. You can go read the many papers that have been published as the technology has been developed. It is not mysterious. We don't need to understand the "clever algorithm", we already understand that. The sense of mystery is because the results of the algorithm are non deterministic, and because the number of parameters that determine the outcome is so large that it appears to exhibit emergent behavior. I guess I agree with your general premise that tools we use should not be treated as magic. But I don't think this is a problem to be solved, it's possible to learn how these things work, and people should definitely do that.
- js8 17d ago
  I disagree we understand LLMs, your view is quite reductionist. It's like saying we understand human brain because we understand physics and evolution, or that we understand some program because it's just a binary code in a known instruction set. I think Dennett's theory of intentionality (see https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance) applies here. We do understand LLMs from physical stance (ML algorithm and inference), but we don't fully understand them from design stance (it's internal workings have been evolved so it's hard to tell the functional units) and from the intentional stance they are a complete mystery. And I talk about obstacles to this understanding elsewhere in this thread.
  - sanderjd 17d ago
    [dead]

by js8 18d ago

sanderjd 17d ago

I see what you're saying, but I don't really think understanding how they work is the problem. We do understand how they work. You can go read the many papers that have been published as the technology has been developed. It is not mysterious. We don't need to understand the "clever algorithm", we already understand that. The sense of mystery is because the results of the algorithm are non deterministic, and because the number of parameters that determine the outcome is so large that it appears to exhibit emergent behavior. I guess I agree with your general premise that tools we use should not be treated as magic. But I don't think this is a problem to be solved, it's possible to learn how these things work, and people should definitely do that.

js8 17d ago
I disagree we understand LLMs, your view is quite reductionist. It's like saying we understand human brain because we understand physics and evolution, or that we understand some program because it's just a binary code in a known instruction set. I think Dennett's theory of intentionality (see https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance) applies here. We do understand LLMs from physical stance (ML algorithm and inference), but we don't fully understand them from design stance (it's internal workings have been evolved so it's hard to tell the functional units) and from the intentional stance they are a complete mystery. And I talk about obstacles to this understanding elsewhere in this thread.
- sanderjd 17d ago
  [dead]