4 ms·

>API calls from one server-side API endpoint to another public API endpoint on localhost:3000 (instead of just importing a function and calling it directly). >

by kgeist 2y ago

>API calls from one server-side API endpoint to another public API endpoint on localhost:3000 (instead of just importing a function and calling it directly). >LLMs will obediently provide the solutions you ask for. If you’re missing fundamental understanding, you won’t be able to spot when your questions have gone off the rails. This made me think: most of the time, when we write code, we have no idea (and don't really care) what kind of assembly the compiler will generate. If a compiler expert looked at the generated assembly, they’d probably say something similar: "They have no idea what they’re doing. The generated assembly shows signs of a fundamental misunderstanding of the underlying hardware," etc. I'm sure most compiled code could be restructured or optimized in a much better, more "professional" way and looks like a total mess to an assembly expert—but no one has really cared for at least two decades now. At the end of the day, as long as your code does what you intend and performs well, does it really matter what it compiles to under the hood? Maybe this is just another paradigm shift (forgive me for using that word) where we start seeing high-level languages as just another compiler backend—except this time, the LLM is the compiler, and natural human language is the programming language.

jrflowers 2y ago
> does it really matter what it compiles to under the hood? The example you quoted could trigger a DDoS if a page using that code got popular.
- kgeist 2y ago
  I'm not claiming the code is perfect; early compilers that generated assembly often produced inefficient code as well. I hope LLMs' coding abilities will improve over time. For now, I'm not ready myself to use LLMs beyond basic prototyping.
jcgrillo 2y ago
The problem with this analogy is that compilers behave deterministically, in that they'll reliably generate the same lower level output for a given source input. Therefore, the program binaries (or bytecode) they generate are debuggable. You can't treat LLM prompt inputs like source code input to a compiler, because the output has no deterministic causal relationship to the input. Also, you may not care now what assembly instructions your function gets compiled down to, but someday you might care a great deal, if for example you need to optimize some inner loop with SIMD parallelism. To do these things you need to be able to reliably control the output using source code. LLMs cannot do this.
- kgeist 2y ago
  LLMs can be made more deterministic if you decrease the temperature parameter and have a fixed seed. Outputs can be controlled with test suites (i.e. that they do not change behavior or have performance regressions). For me as a team lead, a human programmer is already a very non-deterministic agent :) Give a non-trivial task to 10 human programmers and they will all solve it differently. Lack of debuggability is a good argument. Maybe it's only a problem if you want a human to debug the generated code? How about let an LLM iteratively run the code and figure out where it goes wrong by itself (o1 style).
  - jcgrillo 2y ago
    > Maybe it's only a problem if you want a human to debug the generated code? What will you tell your customers when you're suffering some performance regression and e.g. your kafka lag is growing without bound? "I'm sorry, the LLM seems to be unable to figure out how to fix the latest performance regression"? You can't just absolve yourself of responsibility like that. You, the human, are responsible for every single thing the computer does in production, and if you absolve yourself of ownership by leaning on an LLM you end up risking catastrophic helplessness. So you'd better be confident the LLM can debug every issue that will ever come up otherwise your decision to use the LLM could come back at you really hard.
    - kgeist 2y ago
      >You can't just absolve yourself of responsibility like that. You, the human, are responsible for every single thing the computer does in production, and if you absolve yourself of ownership by leaning on an LLM you end up risking catastrophic helplessness >So you'd better be confident the LLM can debug every issue that will ever come up otherwise your decision to use the LLM could come back at you really hard. Most of our programmers are PHP devs. They don't know any C. Once, we hit a bug in the PHP runtime which sporadically crashed our entire application. None of the PHP devs were able to fix the bug because they had no experience debugging C code, let alone the PHP runtime specifically. Fortunately, I had experience with C so I was able to research PHP's source code, and trace the crashes to a memory corruption bug in PHP which only surfaced when a very specific set of options was enabled and only under a high production load (so we did not see it during testing). We reverted the changes and the bug disappeared. What would happen if there was no one to investigate and find the root cause of the bug? Without knowing the cause, they'd probably first try to revert the changes ASAP and that would already solve the problem for the customers. The situation is pretty similar to what you're describing: there's a class of problems which requires knowing what happens "under the hood" at a lower level, and many shops, especially, say, in webdev, don't have the luxury of having engineers which know all ins and outs of the entire system. So this situation can happen any time without any LLMs involved: hardware failures, a kernel bug, a runtime bug -- they all can catch you unprepared. My point is, the risk is definitely there ("I have no idea what's happening and how to fix it") but it's not something novel and can happen without LLMs, and people usually find workarounds. As for debuggability, although LLMs can produce pretty bad code that is harder to debug, I think it's still debuggable by a human, in case of a rare event when even a sufficiently smart LLM cannot debug the problem. The code which, say, ChatGPT generates, is pretty readable and understandable.

by kgeist 2y ago

jrflowers 2y ago

> does it really matter what it compiles to under the hood? The example you quoted could trigger a DDoS if a page using that code got popular.

kgeist 2y ago
I'm not claiming the code is perfect; early compilers that generated assembly often produced inefficient code as well. I hope LLMs' coding abilities will improve over time. For now, I'm not ready myself to use LLMs beyond basic prototyping.

jcgrillo 2y ago

The problem with this analogy is that compilers behave deterministically, in that they'll reliably generate the same lower level output for a given source input. Therefore, the program binaries (or bytecode) they generate are debuggable. You can't treat LLM prompt inputs like source code input to a compiler, because the output has no deterministic causal relationship to the input. Also, you may not care now what assembly instructions your function gets compiled down to, but someday you might care a great deal, if for example you need to optimize some inner loop with SIMD parallelism. To do these things you need to be able to reliably control the output using source code. LLMs cannot do this.

kgeist 2y ago
LLMs can be made more deterministic if you decrease the temperature parameter and have a fixed seed. Outputs can be controlled with test suites (i.e. that they do not change behavior or have performance regressions). For me as a team lead, a human programmer is already a very non-deterministic agent :) Give a non-trivial task to 10 human programmers and they will all solve it differently. Lack of debuggability is a good argument. Maybe it's only a problem if you want a human to debug the generated code? How about let an LLM iteratively run the code and figure out where it goes wrong by itself (o1 style).
- jcgrillo 2y ago
  > Maybe it's only a problem if you want a human to debug the generated code? What will you tell your customers when you're suffering some performance regression and e.g. your kafka lag is growing without bound? "I'm sorry, the LLM seems to be unable to figure out how to fix the latest performance regression"? You can't just absolve yourself of responsibility like that. You, the human, are responsible for every single thing the computer does in production, and if you absolve yourself of ownership by leaning on an LLM you end up risking catastrophic helplessness. So you'd better be confident the LLM can debug every issue that will ever come up otherwise your decision to use the LLM could come back at you really hard.
  - kgeist 2y ago
    >You can't just absolve yourself of responsibility like that. You, the human, are responsible for every single thing the computer does in production, and if you absolve yourself of ownership by leaning on an LLM you end up risking catastrophic helplessness >So you'd better be confident the LLM can debug every issue that will ever come up otherwise your decision to use the LLM could come back at you really hard. Most of our programmers are PHP devs. They don't know any C. Once, we hit a bug in the PHP runtime which sporadically crashed our entire application. None of the PHP devs were able to fix the bug because they had no experience debugging C code, let alone the PHP runtime specifically. Fortunately, I had experience with C so I was able to research PHP's source code, and trace the crashes to a memory corruption bug in PHP which only surfaced when a very specific set of options was enabled and only under a high production load (so we did not see it during testing). We reverted the changes and the bug disappeared. What would happen if there was no one to investigate and find the root cause of the bug? Without knowing the cause, they'd probably first try to revert the changes ASAP and that would already solve the problem for the customers. The situation is pretty similar to what you're describing: there's a class of problems which requires knowing what happens "under the hood" at a lower level, and many shops, especially, say, in webdev, don't have the luxury of having engineers which know all ins and outs of the entire system. So this situation can happen any time without any LLMs involved: hardware failures, a kernel bug, a runtime bug -- they all can catch you unprepared. My point is, the risk is definitely there ("I have no idea what's happening and how to fix it") but it's not something novel and can happen without LLMs, and people usually find workarounds. As for debuggability, although LLMs can produce pretty bad code that is harder to debug, I think it's still debuggable by a human, in case of a rare event when even a sufficiently smart LLM cannot debug the problem. The code which, say, ChatGPT generates, is pretty readable and understandable.