4 ms·

> I got to do my hobby as a career for the past 15 years, but that’s ending. Frankly I don't think so. The AI using LLMs is the perpetual motion mechanism sca

by qsera 7mo ago

> I got to do my hobby as a career for the past 15 years, but that’s ending. Frankly I don't think so. The AI using LLMs is the perpetual motion mechanism scam of our time. But it is cloaked in unimaginable complexity, and thus it is the perfect scam. But even the most elaborately hidden power source in a perpetual motion machine cannot fool nature and should come to a complete stop as it runs out.

Waterluvian 7mo ago
I love the perpetual motion machine / thermodynamics analogy. It kind of feels like companies are being fooled into outsourcing/offshoring their jr. developer level work. Then the companies depend on it because operational inertia is powerful, and will pay as the price keeps going up to cover the perpetual motion lie. Then they look back and realize they're just paying Microsoft for 20 jr. developers but are getting zero benefit from in-house skill development.
colechristensen 7mo ago
This is silly. I can build products in a weekend that would take me a year by myself. I am still necessary 1% of the time for debug, design, and direction and those of not at all a shallow skill. I have some graduate algebra texts on the way my math friend is guiding me through because I have found a publishable result and need to shore up my background before writing the paper... It's not perpetual motion, it's very real capability, you just have to be able to learn how to use it.
- askafriend 7mo ago
  You can see their ego trying to protect itself.
- tpdly 7mo ago
  You're fooling yourself. People yeating a (shitty) Github clone with Claude in a week apparently can't imagine it, but if you know the shit out of Rails, start with a good a boiler plate, and have a good git library, a solo dev can also build a (shitty) Github clone in a week. And they'll be able to take it somewhere, unlike the llm ratsnest that will require increasingly expensive tokens to (frustratingly) modify.
  - mikkupikku 7mo ago
    You're fooling yourself. It's very easy to get demonstrably working results in an afternoon that would take weeks at least without coding agents. Demonstrably working, as in you can prove the code actually works by then putting it to use. I had a coding agent write an entire declarative GUI library for mpv userscripts, rendering all widgets with ASS subtitles, then proceeded to prove to my satisfaction that it does in fact work by using it to make a node editor for constructing ffmpeg filter graphs and an in-mpv nonlinear video editor. All of this is stuff I already knew how to do in practice, had intended to do one day for years now, but never bit the bullet because I knew it would turn into weeks of me pouring over auto-generated ASS doing things it was never intended to do to figure out why something is rendering subtly wrong. Fairly straightforward but a ton of bitch work. The LLM blasted through it like it was nothing. Fooling myself? The code works, I'm using it, you're fooling yourself.
    - zozbot234 7mo ago
      > Demonstrably working, as in you can prove the code actually works by then putting it to use. That's not how you prove that code works properly and isn't going to fail due to some obscure or unforessen corner case. You need actual proof that's driven by the code's overall structure. Humans do this at least informally when they code, AI's can't do that with any reliability, especially not for non-trivial projects (for reasons that are quite structural and hard to change) so most coding agents simply work their way iteratively to get their test results to pass. That's not a robust methodology.
      - coldtea 7mo ago
        >That's not how you prove that code works properly and isn't going to fail due to some obscure or unforessen corner case. So? We didn't prove human code "isn't going to fail due to some obscure or unforessen corner case" either (aside the tiny niche of formal verification). So from that aspect it's quite similar. >so most coding agents simply work their way iteratively to get their test results to pass. That's not a robust methodology. You seem to imply they do some sort of random iteration until the tests pass, which is not the case. Usually they can see the test failing, and describe the issue exactly in the way a human programmer would, then fix it.
- qsera 7mo ago
  No one is saying that it cannot do what you say now. What I am saying is that once the high quality training data runs out, it will drop in its capabilities pretty fast. That is how I compare it to perpetual motion mechanism scams. In the case of a perpetual motion machine, it appear that it will continue to run indefinitely. That is analogous to the impression that you have now. You feel that this will go on and on for ever, and that is the scam you are falling for.
  - _aavaa_ 7mo ago
    Why would the capabilities drop instead of stagnate?
    - qsera 7mo ago
      Because technologies, programming languages, best practices, won't stay frozen. If LLMs cannot catch up with it, I think it can be considered as a drop in capability. No?
      - coldtea 7mo ago
        Close, but no. What will happen is that "technologies, programming languages, best practices" will stay frozen because human innovation will drop, and the whole field will stagnate.
        californical 7mo ago
        This is the biggest fear! I don’t see an easy fix. Will the developer of a new programming language be able to reach out to model companies to give a huge amount of training data, ensuring that the models are good at that new language? I don’t think a small team can write enough code, the models already struggle in medium-popularity languages that have years of history. They hallucinate lua functionality sometimes, for example, even though I’m sure there is lots of lua code out there. So if most people use coding agents, we’re stuck with the current most popular languages because no new language will get past the barrier of having enough code that models can write it well, meaning nobody adopts the new language, etc. Same thing with libraries and frameworks - technical decisions are already being made based on “is this popular enough that the agents can use it well?” Rather than a newer library that meets our needs perfectly but isn’t in the training data
- coldtea 7mo ago
  >This is silly. I can build products in a weekend that would take me a year by myself Is the world any better for them existing? The decline of coding and sw engineering skills in humans from outsourcing the practice of it to AI is it worth it and sustainable long term?
  - colechristensen 7mo ago
    >Is the world any better for them existing? The decline of coding and sw engineering skills in humans from outsourcing the practice of it to AI is it worth it and sustainable long term? The world is going to be no worse than it was when humans transitioned from writing assembly to writing compilers for high level languages. Assembly is still necessary, but not that often. In the same way writing code is going to become less necessary as tools are going to be written in higher level language in standards and requirements documents instead of code most of the time, with more specific exact coding only occasionally. Programmers were mostly solving the same plumbing problems over and over in secret because of "proprietary" needs to hide your code, but one million separate integrations of your billing backend with Stripe didn't really add to humanity. We're cutting out the boring middle drudgery and human effort is going to be freed up to work on the edges of human knowledge instead of tromping around in the middle.
    - coldtea 7mo ago
      >The world is going to be no worse than it was when humans transitioned from writing assembly to writing compilers for high level languages When I open some Electron apps I wish we stopped right about there.

by qsera 7mo ago

Waterluvian 7mo ago

I love the perpetual motion machine / thermodynamics analogy. It kind of feels like companies are being fooled into outsourcing/offshoring their jr. developer level work. Then the companies depend on it because operational inertia is powerful, and will pay as the price keeps going up to cover the perpetual motion lie. Then they look back and realize they're just paying Microsoft for 20 jr. developers but are getting zero benefit from in-house skill development.

colechristensen 7mo ago

This is silly. I can build products in a weekend that would take me a year by myself. I am still necessary 1% of the time for debug, design, and direction and those of not at all a shallow skill. I have some graduate algebra texts on the way my math friend is guiding me through because I have found a publishable result and need to shore up my background before writing the paper... It's not perpetual motion, it's very real capability, you just have to be able to learn how to use it.

askafriend 7mo ago
You can see their ego trying to protect itself.
tpdly 7mo ago
You're fooling yourself. People yeating a (shitty) Github clone with Claude in a week apparently can't imagine it, but if you know the shit out of Rails, start with a good a boiler plate, and have a good git library, a solo dev can also build a (shitty) Github clone in a week. And they'll be able to take it somewhere, unlike the llm ratsnest that will require increasingly expensive tokens to (frustratingly) modify.
- mikkupikku 7mo ago
  You're fooling yourself. It's very easy to get demonstrably working results in an afternoon that would take weeks at least without coding agents. Demonstrably working, as in you can prove the code actually works by then putting it to use. I had a coding agent write an entire declarative GUI library for mpv userscripts, rendering all widgets with ASS subtitles, then proceeded to prove to my satisfaction that it does in fact work by using it to make a node editor for constructing ffmpeg filter graphs and an in-mpv nonlinear video editor. All of this is stuff I already knew how to do in practice, had intended to do one day for years now, but never bit the bullet because I knew it would turn into weeks of me pouring over auto-generated ASS doing things it was never intended to do to figure out why something is rendering subtly wrong. Fairly straightforward but a ton of bitch work. The LLM blasted through it like it was nothing. Fooling myself? The code works, I'm using it, you're fooling yourself.
  - zozbot234 7mo ago
    > Demonstrably working, as in you can prove the code actually works by then putting it to use. That's not how you prove that code works properly and isn't going to fail due to some obscure or unforessen corner case. You need actual proof that's driven by the code's overall structure. Humans do this at least informally when they code, AI's can't do that with any reliability, especially not for non-trivial projects (for reasons that are quite structural and hard to change) so most coding agents simply work their way iteratively to get their test results to pass. That's not a robust methodology.
    - coldtea 7mo ago
      >That's not how you prove that code works properly and isn't going to fail due to some obscure or unforessen corner case. So? We didn't prove human code "isn't going to fail due to some obscure or unforessen corner case" either (aside the tiny niche of formal verification). So from that aspect it's quite similar. >so most coding agents simply work their way iteratively to get their test results to pass. That's not a robust methodology. You seem to imply they do some sort of random iteration until the tests pass, which is not the case. Usually they can see the test failing, and describe the issue exactly in the way a human programmer would, then fix it.
qsera 7mo ago
No one is saying that it cannot do what you say now. What I am saying is that once the high quality training data runs out, it will drop in its capabilities pretty fast. That is how I compare it to perpetual motion mechanism scams. In the case of a perpetual motion machine, it appear that it will continue to run indefinitely. That is analogous to the impression that you have now. You feel that this will go on and on for ever, and that is the scam you are falling for.
- _aavaa_ 7mo ago
  Why would the capabilities drop instead of stagnate?
  - qsera 7mo ago
    Because technologies, programming languages, best practices, won't stay frozen. If LLMs cannot catch up with it, I think it can be considered as a drop in capability. No?
    - coldtea 7mo ago
      Close, but no. What will happen is that "technologies, programming languages, best practices" will stay frozen because human innovation will drop, and the whole field will stagnate.
      - californical 7mo ago
        This is the biggest fear! I don’t see an easy fix. Will the developer of a new programming language be able to reach out to model companies to give a huge amount of training data, ensuring that the models are good at that new language? I don’t think a small team can write enough code, the models already struggle in medium-popularity languages that have years of history. They hallucinate lua functionality sometimes, for example, even though I’m sure there is lots of lua code out there. So if most people use coding agents, we’re stuck with the current most popular languages because no new language will get past the barrier of having enough code that models can write it well, meaning nobody adopts the new language, etc. Same thing with libraries and frameworks - technical decisions are already being made based on “is this popular enough that the agents can use it well?” Rather than a newer library that meets our needs perfectly but isn’t in the training data
coldtea 7mo ago
>This is silly. I can build products in a weekend that would take me a year by myself Is the world any better for them existing? The decline of coding and sw engineering skills in humans from outsourcing the practice of it to AI is it worth it and sustainable long term?
- colechristensen 7mo ago
  >Is the world any better for them existing? The decline of coding and sw engineering skills in humans from outsourcing the practice of it to AI is it worth it and sustainable long term? The world is going to be no worse than it was when humans transitioned from writing assembly to writing compilers for high level languages. Assembly is still necessary, but not that often. In the same way writing code is going to become less necessary as tools are going to be written in higher level language in standards and requirements documents instead of code most of the time, with more specific exact coding only occasionally. Programmers were mostly solving the same plumbing problems over and over in secret because of "proprietary" needs to hide your code, but one million separate integrations of your billing backend with Stripe didn't really add to humanity. We're cutting out the boring middle drudgery and human effort is going to be freed up to work on the edges of human knowledge instead of tromping around in the middle.
  - coldtea 7mo ago
    >The world is going to be no worse than it was when humans transitioned from writing assembly to writing compilers for high level languages When I open some Electron apps I wish we stopped right about there.