7 ms·

> Having a conversation with a computer to generate code that I don't entirely understand and then have to try to review is just not fun. Same for me. But mayb

by Garlef 1y ago

> Having a conversation with a computer to generate code that I don't entirely understand and then have to try to review is just not fun. Same for me. But maybe that's ultimately an UX issue? And maybe things will straighten out once we figure out how to REALLY do AI assisted software development. As an anology: Most people wouldn't want to dig through machine code/compiler output. At least not without proper tooling. So: Maybe once we have good tools to understand the output it might be fun again. (I guess this would include advances in structuring/architecting the output)

bitwize 1y ago
I think that AI assistance in coding will become enjoyable for me once the technology exists for AI to translate my brainwaves into text. Then I could think my code into computer, greatly speeding the OODA loop of programming. As it is, giving high-level directives to an LLM and debugging the output seems like a waste of my time and a hindrance to my learning process. But that's how professional coding will be done in the near future. 100% human written code will become like hand-writing a business letter in cursive: something people used to be taught in school, but no one actually does in the real world because it's too time-consuming. Ultimately, the business world only cares about productivity and what the stopwatch says is faster, not whether you enjoy or learn from the process.
cratermoon 1y ago
Tim doesn't address this in his essay, so I'm going to harp on it: "AI will soon be able to...". That phrase is far too load-bearing. The part of AI hype that says, "sure, it's kinda janky now, but this is just the beginning" has been repeated for 3 years now, and everything has been just around the corner the entire time. It's the first step fallacy, saying that if we can build a really tall ladder now, surely we'll soon be able to build a ladder tall enough to reach the moon. The reality is that we've seen incremental and diminishing returns, and the promises haven't been met.
- tptacek 1y ago
  Diminishing returns? Am I reading right that you believe the last 6 months has been marked by a decrease in the capability of these systems?
  - cratermoon 1y ago
    That's not what diminishing returns means.
    - tptacek 1y ago
      That's true, but it's nearest bit of evidence at hand to how the "returns" could be "diminishing". I'm fine if someone wants to provide any other coherent claim as to how we're in a "diminishing returns" state with coding LLMs right now.
      - cratermoon 1y ago
        https://techcrunch.com/2024/11/20/ai-scaling-laws-are-showing-diminishing-returns-forcing-ai-labs-to-change-course/ https://techcrunch.com/2024/11/20/ai-scaling-laws-are-showin...
        tptacek 1y ago
        What's the implication of this story to someone who had started writing code with LLMs 6 months ago, and is today as well. How has their experience changed? Have the returns to that activity diminished?
username223 1y ago
> As an anology: Most people wouldn't want to dig through machine code/compiler output. At least not without proper tooling. My analogy is GUI builders from the late 90s that let you drag elements around, then generated a pile of code. They worked sometimes, but God help you if you wanted to do something the builder couldn't do, and had to edit the generated code. Looking at compiler output is actually more pleasant. You profile your code, find the hot spots, and see that something isn't getting inlined, vectorized, etc. At that point you can either convince the compiler to do what you want or rewrite it by hand, and the task is self-contained.
layer8 1y ago
The compiler analogy doesn’t quite fit, because the essential difference is that source code is (mostly) deterministic and thus can be reasoned about (you can largely predict in detail what behavior code will exhibit even before writing it), which isn’t the case for LLM instructions. That’s a major factor why many developers don’t like AI coding, because every prompt becomes a non-reproducible, literally un-reasonable experiment.
- steveklabnik 1y ago
  I think the "largely" in there is interesting and load-bearing: a lot of people find compiler output quite surprising! But that doesn't mean that it's not a gradient, and LLM output may be meaningfully harder to reason about than compiler output, and that may matter.
  - layer8 1y ago
    Assembly output may sometimes be surprising, but maintains the language semantics. The surprise comes from either misunderstanding the language semantics, or from performance aspects. Nevertheless, if you understand the language semantics correctly, the program behavior resulting from the output is deterministic and predictable. This is not true for LLMs.
    - steveklabnik 1y ago
      I don't disagree on a factual level, I am just describing some people's subjective experiences: some language semantics can be very subtle, and miscompilation bugs are real. Determining if it is just an aggressive optimization or a real codegen bug can be difficult sometimes, that's all.
      - layer8 1y ago
        To some extent yes, but I don’t think that changes much about the distinction to AI coding I was making. The thing is that language misconceptions are fixable; the LLM unpredictability isn’t.

by Garlef 1y ago

bitwize 1y ago

I think that AI assistance in coding will become enjoyable for me once the technology exists for AI to translate my brainwaves into text. Then I could think my code into computer, greatly speeding the OODA loop of programming. As it is, giving high-level directives to an LLM and debugging the output seems like a waste of my time and a hindrance to my learning process. But that's how professional coding will be done in the near future. 100% human written code will become like hand-writing a business letter in cursive: something people used to be taught in school, but no one actually does in the real world because it's too time-consuming. Ultimately, the business world only cares about productivity and what the stopwatch says is faster, not whether you enjoy or learn from the process.

cratermoon 1y ago

Tim doesn't address this in his essay, so I'm going to harp on it: "AI will soon be able to...". That phrase is far too load-bearing. The part of AI hype that says, "sure, it's kinda janky now, but this is just the beginning" has been repeated for 3 years now, and everything has been just around the corner the entire time. It's the first step fallacy, saying that if we can build a really tall ladder now, surely we'll soon be able to build a ladder tall enough to reach the moon. The reality is that we've seen incremental and diminishing returns, and the promises haven't been met.

tptacek 1y ago
Diminishing returns? Am I reading right that you believe the last 6 months has been marked by a decrease in the capability of these systems?
- cratermoon 1y ago
  That's not what diminishing returns means.
  - tptacek 1y ago
    That's true, but it's nearest bit of evidence at hand to how the "returns" could be "diminishing". I'm fine if someone wants to provide any other coherent claim as to how we're in a "diminishing returns" state with coding LLMs right now.
    - cratermoon 1y ago
      https://techcrunch.com/2024/11/20/ai-scaling-laws-are-showing-diminishing-returns-forcing-ai-labs-to-change-course/ https://techcrunch.com/2024/11/20/ai-scaling-laws-are-showin...
      - tptacek 1y ago
        What's the implication of this story to someone who had started writing code with LLMs 6 months ago, and is today as well. How has their experience changed? Have the returns to that activity diminished?

username223 1y ago

> As an anology: Most people wouldn't want to dig through machine code/compiler output. At least not without proper tooling. My analogy is GUI builders from the late 90s that let you drag elements around, then generated a pile of code. They worked sometimes, but God help you if you wanted to do something the builder couldn't do, and had to edit the generated code. Looking at compiler output is actually more pleasant. You profile your code, find the hot spots, and see that something isn't getting inlined, vectorized, etc. At that point you can either convince the compiler to do what you want or rewrite it by hand, and the task is self-contained.

layer8 1y ago

The compiler analogy doesn’t quite fit, because the essential difference is that source code is (mostly) deterministic and thus can be reasoned about (you can largely predict in detail what behavior code will exhibit even before writing it), which isn’t the case for LLM instructions. That’s a major factor why many developers don’t like AI coding, because every prompt becomes a non-reproducible, literally un-reasonable experiment.

steveklabnik 1y ago
I think the "largely" in there is interesting and load-bearing: a lot of people find compiler output quite surprising! But that doesn't mean that it's not a gradient, and LLM output may be meaningfully harder to reason about than compiler output, and that may matter.
- layer8 1y ago
  Assembly output may sometimes be surprising, but maintains the language semantics. The surprise comes from either misunderstanding the language semantics, or from performance aspects. Nevertheless, if you understand the language semantics correctly, the program behavior resulting from the output is deterministic and predictable. This is not true for LLMs.
  - steveklabnik 1y ago
    I don't disagree on a factual level, I am just describing some people's subjective experiences: some language semantics can be very subtle, and miscompilation bugs are real. Determining if it is just an aggressive optimization or a real codegen bug can be difficult sometimes, that's all.
    - layer8 1y ago
      To some extent yes, but I don’t think that changes much about the distinction to AI coding I was making. The thing is that language misconceptions are fixable; the LLM unpredictability isn’t.