3 ms·
Going from programming language to LLM is not the same kind of abstraction as going from assembler to Fortran or Algol. With programming languages, there is a
by bitwize 9mo ago
Going from programming language to LLM is not the same kind of abstraction as going from assembler to Fortran or Algol.
With programming languages, there is a transparent homomorphism between the code you write and what the machine actually executes. Programmers use this property of programming languages to execute considerable control over the computational process they're evolving while still potentially at a high level of abstraction. With LLMs, the mapping between your input and the executable output is opaque and nondeterministic. This drives people like me batty.
A while back I wrote a Lisp koan:
https://www.hackersdictionary.com/html/Some-AI-Koans.html https://www.hackersdictionary.com/html/Some-AI-Koans.html
Mine goes like this:
"A student travelled to the East to hear Sussman's teachings. Sussman was expounding on low-level development in Lisp, when the student interrupted him. 'Master Sussman,' said he, 'is not Lisp a high-level language, indeed perhaps the highest-level of all programming languages?' Sussman replied, 'Once there was a man who was walking along the beach when he spotted an eagle. Brother eagle, said he, how impossibly distant is the sky! The eagle said nothing, and flew away.' Thus the student was enlightened."
The story is in some sense true: I did meet Gerald Sussman and was in some sense enlightened by him. Another hacker was talking about working in a "low-level language" like Lisp, and I corrected him telling him that Lisp was in fact very high level. He said "Uh... I need Jerry to explain this to you. Jerry? Can you come here a minute?" "Jerry" was Gerald Sussman, who proceeded to explain to me that Lisp was a virtual machine, one which he implemented in silico for his Ph.D. thesis:
https://dspace.mit.edu/handle/1721.1/5731 https://dspace.mit.edu/handle/1721.1/5731
Thus I was enlightened. If Lisp is a virtual machine, so are all programming languages. And even a buck JavaScript kiddie fresh out of boot camp, working in React, is working in machine code for the JavaScript+browser+React VM. An abstraction is a point of view, and the programmer working in a "high level" programming language is really just working in the same medium as machine code: computation itself. But with a point of view that offers more convenience.
LLM work is different. LLMs are enormously complicated algorithms that give probabilistic interpretations of loose, informal human languages. So instructing a computer through the filter of an LLM is inherently probabilistic, not to mention damn near inscrutable.
This is why LLMs are being met with more resistance even than compilers were. They're not the same thing. Compilers scaled the work, which remained essentially the same. LLMs are changing it.