3 ms·

I totally agree with the skepticism that we'll ever get to a point where natural language becomes the "formalism" and stop needing people who understand the act

I totally agree with the skepticism that we'll ever get to a point where natural language becomes the "formalism" and stop needing people who understand the actual formalism underneath. And I agree that you don't need LLM based tools. But things that you don't need can still be (and often are) incredibly useful. Nobody needs an IDE, nobody needs vim or emacs or bash or even compilers or assemblers. But we have all those tools and they are useful. That is, their utility is net positive. Using LLMs to generate code currently also has (wildly) net positive utility. Maybe that will change because some part of the calculation changes. But this is the situation right now.

jopsen 18d ago
Also not everything needs to be formal. I might want the REST API for a webshop to be solid, payment and checkout process, sure. But the UI. So long as the LLM doesn't falsify product information, why customize the layout, theme, look and feel for each and every single customer. Okay, maybe don't, but point is: you could take bigger risks, you maybe don't need to review UI changes as much.
- apsurd 18d ago
  The bit about "but the UI" sounds like a developer thinking "UI is a solved problem" in the same way MBAs are told "coding is a solved problem". Product people do seem to think it worth exploring "personalized software for everyone" like literally each person gets their own custom UI. This does sound like a support mess but it's not obviously wrong when you think about the Microsoft Word alternative.
js8 18d ago
I agree LLMs are useful tools, but I am dismayed by a lot of cargo-culting around them, which happens because we don't understand them. I think we will have much better tools when we understand what is the expectation and what is the algorithm they run. Somebody else said that the magician analogy was poor. I like magic tricks, but it took many years of cultural change (influenced by people like Houdini, Randi, Penn & Teller) to stop illusionists (and mentalists) make claims they have supernatural abilities, or people believing it on their own (a magician pretending to be able to catch a bullet was shot by an audience member who didn't understand the distinction). It is detrimental, I think, to treat LLMs as if they have magical abilities ("superintelligence") rather than understanding they just run some clever algorithm. The fear for (programming) jobs comes from that framing; nobody fears of their job because of compilers, since compilers are understood. (And it actually runs against kind of "socialist" framing of the problem, which I agree with, that is why should people be worried about the jobs in the first place, when society is getting richer as a result of better tools?)
- sanderjd 18d ago
  I see what you're saying, but I don't really think understanding how they work is the problem. We do understand how they work. You can go read the many papers that have been published as the technology has been developed. It is not mysterious. We don't need to understand the "clever algorithm", we already understand that. The sense of mystery is because the results of the algorithm are non deterministic, and because the number of parameters that determine the outcome is so large that it appears to exhibit emergent behavior. I guess I agree with your general premise that tools we use should not be treated as magic. But I don't think this is a problem to be solved, it's possible to learn how these things work, and people should definitely do that.
  - js8 18d ago
    I disagree we understand LLMs, your view is quite reductionist. It's like saying we understand human brain because we understand physics and evolution, or that we understand some program because it's just a binary code in a known instruction set. I think Dennett's theory of intentionality (see https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance) applies here. We do understand LLMs from physical stance (ML algorithm and inference), but we don't fully understand them from design stance (it's internal workings have been evolved so it's hard to tell the functional units) and from the intentional stance they are a complete mystery. And I talk about obstacles to this understanding elsewhere in this thread.
    - sanderjd 17d ago
      [dead]

jopsen 18d ago

Also not everything needs to be formal. I might want the REST API for a webshop to be solid, payment and checkout process, sure. But the UI. So long as the LLM doesn't falsify product information, why customize the layout, theme, look and feel for each and every single customer. Okay, maybe don't, but point is: you could take bigger risks, you maybe don't need to review UI changes as much.

apsurd 18d ago
The bit about "but the UI" sounds like a developer thinking "UI is a solved problem" in the same way MBAs are told "coding is a solved problem". Product people do seem to think it worth exploring "personalized software for everyone" like literally each person gets their own custom UI. This does sound like a support mess but it's not obviously wrong when you think about the Microsoft Word alternative.

js8 18d ago

I agree LLMs are useful tools, but I am dismayed by a lot of cargo-culting around them, which happens because we don't understand them. I think we will have much better tools when we understand what is the expectation and what is the algorithm they run. Somebody else said that the magician analogy was poor. I like magic tricks, but it took many years of cultural change (influenced by people like Houdini, Randi, Penn & Teller) to stop illusionists (and mentalists) make claims they have supernatural abilities, or people believing it on their own (a magician pretending to be able to catch a bullet was shot by an audience member who didn't understand the distinction). It is detrimental, I think, to treat LLMs as if they have magical abilities ("superintelligence") rather than understanding they just run some clever algorithm. The fear for (programming) jobs comes from that framing; nobody fears of their job because of compilers, since compilers are understood. (And it actually runs against kind of "socialist" framing of the problem, which I agree with, that is why should people be worried about the jobs in the first place, when society is getting richer as a result of better tools?)

sanderjd 18d ago
I see what you're saying, but I don't really think understanding how they work is the problem. We do understand how they work. You can go read the many papers that have been published as the technology has been developed. It is not mysterious. We don't need to understand the "clever algorithm", we already understand that. The sense of mystery is because the results of the algorithm are non deterministic, and because the number of parameters that determine the outcome is so large that it appears to exhibit emergent behavior. I guess I agree with your general premise that tools we use should not be treated as magic. But I don't think this is a problem to be solved, it's possible to learn how these things work, and people should definitely do that.
- js8 18d ago
  I disagree we understand LLMs, your view is quite reductionist. It's like saying we understand human brain because we understand physics and evolution, or that we understand some program because it's just a binary code in a known instruction set. I think Dennett's theory of intentionality (see https://en.wikipedia.org/wiki/Intentional_stance https://en.wikipedia.org/wiki/Intentional_stance) applies here. We do understand LLMs from physical stance (ML algorithm and inference), but we don't fully understand them from design stance (it's internal workings have been evolved so it's hard to tell the functional units) and from the intentional stance they are a complete mystery. And I talk about obstacles to this understanding elsewhere in this thread.
  - sanderjd 17d ago
    [dead]