5 ms·

I don't get the obsession some tech people have to push the idea that this stuff accelerate your coding, increase your productivity. It's all about fast and fas

I don't get the obsession some tech people have to push the idea that this stuff accelerate your coding, increase your productivity. It's all about fast and faster output. In my experience LLMs have mostly produced gibberish oververbose code, surely faster than me, but my lower speed usually produce better code. I don't like this present state of things where we need to chat faster to quickly get out results and go fast in production... that is the kind of mentality that pushed subpar products on the web for so many years. Instead of dropping names lile vibe coding/engineering whatever comes next, let's have a serious discussion why we need faster low quality and we can't just improve automation and processes. Eg I can get unit test generated very fast sure, but my question is: why do we need all these unit tests in the first place? Dont get me wrong they're useful, but I feel like we're advancing higher abstraction instead of advancing lower level tools

rhetocj23 1y ago
It’ll be the greatest episode of sunk cost fallacy 5 years from now.
treve 1y ago
I think the general issue with software engineering discourse is that while our tools and languages may be the same, there's a massive gradient in tolerance for incorrectness, security, compliance, maintainability. Some of us are building prototypes, and others are building software with a 10 year horizon or work with sensitive personal data. So on the one side you get people that are super efficient cranking things out, and others that read this and feel appalled anyone would be comfortable with this and see software engineering as theory building. Neither are really wrong here, but the context / risk of what people are working on always seems to be missing when people talk about this.
- hollowturtle 1y ago
  I have yet to see in my life a prototypes that doesn't become production :) Btw my whole point wasn't on security and can't find a compelling reason to talk about it, it rather questions "faster results" as "better productivity" it isn't, and imo we should pause for a moment and focus on better tooling
computerex 1y ago
We live in an objective reality. LLM's help a damn lot in speed of development. As someone who has been coding since 5th grade for over 20 years who is known to be a LIGHTNING FAST implementor by many people I have been scaled ridiculously with LLM's. Genie is out of the bag, you have to go with the flow, adapt or else.... I am just as pissed as anybody else at the lack of certainty in the future.... Thought I had my career made. So it's not that I don't emphatize with engineers in my shoes.
- hollowturtle 1y ago
  Good lord I wish I could have as many certainties as well, one point at a time: * There is no objective reality, there isn't one in physics, it's just a non argument * "LLM's help a damn lot in speed of development" That may be your experience and my whole point was arguing that speed may not matter * "Genie is out of the bag, you have to go with the flow, adapt or else" I choose else if this the apex of the argument
  - holbrad 1y ago
    I'm genuinely baffled by the assertion that there isn't objective reality.
    - gls2ro 1y ago
      If you go down in physics you might discover that the reality seems to be composed a lot more by probabilities. An objective reality built on probabilities does not seem so objective afterall. Here is a fun think to try: demonstrate that you and me are seeing the same color in the same way? You might find some papers trying to prove it but you will se they are all based on subjective answers of people. So while at macro level the reality seems hard and objective at the micro level it is not.
      - computerex 1y ago
        It's an objective reality. Even at quantum level the known laws of nature hold. Even if there is uncertainty, nature is predictable in following the laws of physics. And ofc at macro level we live in a very objective reality. This is the basis of science.
jackcviers3 1y ago
I'll attempt to provide a reasonable argument for why speed of delivery is the most important thing in software development. I'll concede that I don't know if the below is true, and haven't conducted formal experiments, and have no real-world data to back up the claims, nor even define all the terms in the argument beyond generally accepted terminology. The premise of the argument therefore may be incorrect. Trivial software is software for which - the value of which the software solution is widely accepted and widely known in practice and - formal verification exists and is possible to automate or - only has a single satisfying possible implementation. Most software is non-trivial. There will always be: - bugs in implementation - missed requirements - leaky abstractions - incorrect features with no user or business value - problems with integration - problems with performance - security problems - complexity problems - maintenance problems in any non-trivial software no matter how "good" the engineer producing the code is or how "good" the code is. These problems are surfaced and reduced to lie within acceptable operational tolerances via iterative development. It doesn't matter how formal our specifications are or how rigorous our verification procedures are if they are validated against an incorrect model of the problem we are attempting to solve with the software we write. These problems can only be discovered through iterative acceptance testing, experimentation, and active use, maintenance, and constructive feedback on the quality of the software we write. This means that the overall quality of any non-trivial software is dominated by the total number of quality feedback loops executed during its lifetime. The number of feedback loops during the software's lifetime are bound by the time it takes to complete a single synchchronous feedback loop. Multiple feedback loops may be executed in parallel, but Amdahl's law holds for overall delivery. Therefore, time to delivery is the dominant factor to consider in order to produce valuable software products. Your slower to produce, higher quality code puts a boundary on the duration of a single feedback loop iteration. The code you produce can perfectly solve the problem as you understand it within an iteration, but cannot guarantee that your understanding of the problem is not wrong. In that sense, many lower quality iterations produces better software quality as the number of iterations approaches infinity.
- hollowturtle 1y ago
  >> Your slower to produce, higher quality code puts a boundary on the duration of a single feedback loop iteration. The code you produce can perfectly solve the problem as you understand it within an iteration, but cannot guarantee that your understanding of the problem is not wrong. In that sense, many lower quality iterations produces better software quality as the number of iterations approaches infinity. I'll reply just to that as it being the tldr. First of all tech debt is a thing and it's the thing that accumulates mostly thanks to fast feedback iterations. And in my experience the better the comunication, to get the implementation right, and the better the implementation and it happens that you can have solid features that you'll unlikely ever touch again, user base habit is also a thing, continuing on interating on something a user knows how to use and changing it is a bad thing. I'd also argue it's bad product/project management. But my whole original argument was why we'd need to have a greater speed in the first place, better tooling doesn't necessarily means faster output, productivity as well isn't measured as just faster output. Let me make a concrete example, if you ask an LLM X to produce a UI with some features, most of them will default to using React, why? Why can't we question the current state of web instead of continue to pile up abstractions over abstractions? Even if I ask the LLM to create a vanilla web app with HTML, why can't we have better tooling for sharing apps over the internet? The web is stagnant and instead of fixing it we're building castles over castles over it
krschacht 1y ago
You aren’t entertaining the possibility that some experienced engineerings are using these tools to produce incredibly high quality code, while still massively increasing productivity. With good prompting and “vibe engineering” practices, I can assure you: the code I get Claude Code to produce is top notch.
- hollowturtle 1y ago
  I'm experienced, I don't accept the implication that I might not be able to use these tools are their full potential and you won't convince me only because you mention an anecdotical example
  - simonw 1y ago
    You must be very confident in your own ability if you think you can use any tool to its full potential with no scope for getting better. I have tools I've been using for 25 years that I still think I could be using better.
    - hollowturtle 1y ago
      Absolutely but we're talking about structured tools, like a cli, not unstructured non deterministic "agents" that fails to give the same answer twice, ls -la doesn't lie
    - sarchertech 1y ago
      You also must be very confident in your own ability if you don’t think that at least some of the things you’re doing that you’d classify as “skill at using the tool” aren’t just superstitions à la Skinner’s pigeons.
      - simonw 1y ago
        I'm sure a lot of them are superstitions! I've written about that before: https://simonwillison.net/2023/Aug/27/wordcamp-llms/#superstitious https://simonwillison.net/2023/Aug/27/wordcamp-llms/#superst... One of the more "engineering" like skills in using this stuff is methodically figuring out what's a superstition and what actually works.
joenot443 1y ago
> that is the kind of mentality that pushed subpar products on the web for so many years Famously, some of those subpar products are now household names who were able to stake out their place in the market because of their ability to move quickly and iterate. Had they prioritized long-maintainable code quality rather than user journey, it's possible they wouldn't be where they are today. "Move fast and break things" wasn't a joke; Mark really did encourage people to build faster and it helped cement their positioning. Think of the quantity of features FB shoveled out between 2009-2014 or so, that just wouldn't have been possible if their primary objective was flawless code. The code isn't the product, the product is the product. In all my years of engineering I've yet to have an end-user tell me they liked my coding style, they've always been more concerned with what I'd built them.
- hollowturtle 1y ago
  Facebook didn't solved any real problem, "Move fast and break things" is for investors not hackers. Famously gmail was very good quality web app and code(can't say the same today) surely not the product of today's "fast iteration" culture
  - nc 1y ago
    Meta has a 1.79T market cap, they definitely solved some very real problems to get there. There are lots of companies doing well producing high quality products out of the gate today though, look at Linear. Both approaches are valid for building sustainable enduring businesses.
    - hollowturtle 1y ago
      > Meta has a 1.79T market cap, they definitely solved some very real problems to get there. Sure exploiting human attention to sell them ads, it's not technological marvel, I'd say psychological
      - nc 1y ago
        Pretty negative take, TV and Radio also exploit human attention in your book? Technology wise they created React, PyTorch, GraphQL and Llama is open weights.
athrowaway3z 1y ago
You see a large % of failures, but you're drawing an unsupported conclusion. We all agree, the people that _feel_ the most productivity spike are the sub-par engineers. That shouldn't be controversial, and it's even predictable. But their volume can't be taken as an argument one way or the other. The question is, are there _any_ good engineers that don't just feel more productive, but objectively are.
mekoka 1y ago
People are constantly looking for definite tendencies and magic patterns so they can abdicate situational awareness and critical thinking. We observe that fast delivery has often correlated with success in software and we infer that fast delivery is a factor of success. Then it becomes about the mindless pursuit of the measure, speed of delivery, as Goodhart's law predicts. Let's even concede that speed of delivery indeed is an actual factor, there has to be a threshold. There's a point where people just don't care how fast you're putting out features, because your product has found its sweet spot and is perfectly scratching its market's itch. A few will clearly notice when the line of diminishing returns is crossed and if they can reset their outlook to fit the new context, a continuous focus on speed of delivery will look increasingly obsessive and nonsensical. But that's the reality of the majority of the software development world. Few of us work on anything mission critical. We could produce nice sane software at a reasonable pace with decent output, but we're sold the idea that there's always more productivity to squeeze and we're told that we really really want that juice.
- wartywhoa23 1y ago
  That all things develop only so much before they degrade into overdevelopment was a very well understood phenomenon for ancient Taoists, and it will be the death of the modern Blackrock/Vanguard owned world which is absolutely ignorant of this principle.
gwbas1c 1y ago
> why do we need all these unit tests in the first place? The same reason we've always needed them: 1: They prevent regressions. (IE, bugs in features that were shipped and already working.) 2: They are very easy to run at the push of a button in your IDE. (But in this context the LLM runs them.) 3: They run in CI. This is an important line of defense in making sure a pull request doesn't introduce a bug. Now, depending on what you're writing, you might not need unit tests! Perhaps you're trying to get a minimum viable product out the door? Perhaps you're trying to demo a feature to see if it's worth building? Perhaps you're writing a 1-off tool that you'll run a few times and throw away? But, understand that if you're writing an industrial-strength program, your unit tests help you ship bug-free software. They allow you to do some rather major refactors, sometimes touching areas of the codebase that you only lightly understand, without needing to manually test everything. (And, to keep it in context,) your LLM will also have the same benefits from this tired-and-true process.
- hollowturtle 1y ago
  Thanks for the non solicited lesson, I learned basically nothing new. That unit tests ships bug free software is such an overstatement... Btw please go re read my comment and try to understand what I was actually trying to argue about, and I also wrote that I think they can be useful...

rhetocj23 1y ago

It’ll be the greatest episode of sunk cost fallacy 5 years from now.

treve 1y ago

I think the general issue with software engineering discourse is that while our tools and languages may be the same, there's a massive gradient in tolerance for incorrectness, security, compliance, maintainability. Some of us are building prototypes, and others are building software with a 10 year horizon or work with sensitive personal data. So on the one side you get people that are super efficient cranking things out, and others that read this and feel appalled anyone would be comfortable with this and see software engineering as theory building. Neither are really wrong here, but the context / risk of what people are working on always seems to be missing when people talk about this.

hollowturtle 1y ago
I have yet to see in my life a prototypes that doesn't become production :) Btw my whole point wasn't on security and can't find a compelling reason to talk about it, it rather questions "faster results" as "better productivity" it isn't, and imo we should pause for a moment and focus on better tooling

computerex 1y ago

We live in an objective reality. LLM's help a damn lot in speed of development. As someone who has been coding since 5th grade for over 20 years who is known to be a LIGHTNING FAST implementor by many people I have been scaled ridiculously with LLM's. Genie is out of the bag, you have to go with the flow, adapt or else.... I am just as pissed as anybody else at the lack of certainty in the future.... Thought I had my career made. So it's not that I don't emphatize with engineers in my shoes.

hollowturtle 1y ago
Good lord I wish I could have as many certainties as well, one point at a time: * There is no objective reality, there isn't one in physics, it's just a non argument * "LLM's help a damn lot in speed of development" That may be your experience and my whole point was arguing that speed may not matter * "Genie is out of the bag, you have to go with the flow, adapt or else" I choose else if this the apex of the argument
- holbrad 1y ago
  I'm genuinely baffled by the assertion that there isn't objective reality.
  - gls2ro 1y ago
    If you go down in physics you might discover that the reality seems to be composed a lot more by probabilities. An objective reality built on probabilities does not seem so objective afterall. Here is a fun think to try: demonstrate that you and me are seeing the same color in the same way? You might find some papers trying to prove it but you will se they are all based on subjective answers of people. So while at macro level the reality seems hard and objective at the micro level it is not.
    - computerex 1y ago
      It's an objective reality. Even at quantum level the known laws of nature hold. Even if there is uncertainty, nature is predictable in following the laws of physics. And ofc at macro level we live in a very objective reality. This is the basis of science.

jackcviers3 1y ago

I'll attempt to provide a reasonable argument for why speed of delivery is the most important thing in software development. I'll concede that I don't know if the below is true, and haven't conducted formal experiments, and have no real-world data to back up the claims, nor even define all the terms in the argument beyond generally accepted terminology. The premise of the argument therefore may be incorrect. Trivial software is software for which - the value of which the software solution is widely accepted and widely known in practice and - formal verification exists and is possible to automate or - only has a single satisfying possible implementation. Most software is non-trivial. There will always be: - bugs in implementation - missed requirements - leaky abstractions - incorrect features with no user or business value - problems with integration - problems with performance - security problems - complexity problems - maintenance problems in any non-trivial software no matter how "good" the engineer producing the code is or how "good" the code is. These problems are surfaced and reduced to lie within acceptable operational tolerances via iterative development. It doesn't matter how formal our specifications are or how rigorous our verification procedures are if they are validated against an incorrect model of the problem we are attempting to solve with the software we write. These problems can only be discovered through iterative acceptance testing, experimentation, and active use, maintenance, and constructive feedback on the quality of the software we write. This means that the overall quality of any non-trivial software is dominated by the total number of quality feedback loops executed during its lifetime. The number of feedback loops during the software's lifetime are bound by the time it takes to complete a single synchchronous feedback loop. Multiple feedback loops may be executed in parallel, but Amdahl's law holds for overall delivery. Therefore, time to delivery is the dominant factor to consider in order to produce valuable software products. Your slower to produce, higher quality code puts a boundary on the duration of a single feedback loop iteration. The code you produce can perfectly solve the problem as you understand it within an iteration, but cannot guarantee that your understanding of the problem is not wrong. In that sense, many lower quality iterations produces better software quality as the number of iterations approaches infinity.

hollowturtle 1y ago
>> Your slower to produce, higher quality code puts a boundary on the duration of a single feedback loop iteration. The code you produce can perfectly solve the problem as you understand it within an iteration, but cannot guarantee that your understanding of the problem is not wrong. In that sense, many lower quality iterations produces better software quality as the number of iterations approaches infinity. I'll reply just to that as it being the tldr. First of all tech debt is a thing and it's the thing that accumulates mostly thanks to fast feedback iterations. And in my experience the better the comunication, to get the implementation right, and the better the implementation and it happens that you can have solid features that you'll unlikely ever touch again, user base habit is also a thing, continuing on interating on something a user knows how to use and changing it is a bad thing. I'd also argue it's bad product/project management. But my whole original argument was why we'd need to have a greater speed in the first place, better tooling doesn't necessarily means faster output, productivity as well isn't measured as just faster output. Let me make a concrete example, if you ask an LLM X to produce a UI with some features, most of them will default to using React, why? Why can't we question the current state of web instead of continue to pile up abstractions over abstractions? Even if I ask the LLM to create a vanilla web app with HTML, why can't we have better tooling for sharing apps over the internet? The web is stagnant and instead of fixing it we're building castles over castles over it

krschacht 1y ago

You aren’t entertaining the possibility that some experienced engineerings are using these tools to produce incredibly high quality code, while still massively increasing productivity. With good prompting and “vibe engineering” practices, I can assure you: the code I get Claude Code to produce is top notch.

hollowturtle 1y ago
I'm experienced, I don't accept the implication that I might not be able to use these tools are their full potential and you won't convince me only because you mention an anecdotical example
- simonw 1y ago
  You must be very confident in your own ability if you think you can use any tool to its full potential with no scope for getting better. I have tools I've been using for 25 years that I still think I could be using better.
  - hollowturtle 1y ago
    Absolutely but we're talking about structured tools, like a cli, not unstructured non deterministic "agents" that fails to give the same answer twice, ls -la doesn't lie
  - sarchertech 1y ago
    You also must be very confident in your own ability if you don’t think that at least some of the things you’re doing that you’d classify as “skill at using the tool” aren’t just superstitions à la Skinner’s pigeons.
    - simonw 1y ago
      I'm sure a lot of them are superstitions! I've written about that before: https://simonwillison.net/2023/Aug/27/wordcamp-llms/#superstitious https://simonwillison.net/2023/Aug/27/wordcamp-llms/#superst... One of the more "engineering" like skills in using this stuff is methodically figuring out what's a superstition and what actually works.

joenot443 1y ago

> that is the kind of mentality that pushed subpar products on the web for so many years Famously, some of those subpar products are now household names who were able to stake out their place in the market because of their ability to move quickly and iterate. Had they prioritized long-maintainable code quality rather than user journey, it's possible they wouldn't be where they are today. "Move fast and break things" wasn't a joke; Mark really did encourage people to build faster and it helped cement their positioning. Think of the quantity of features FB shoveled out between 2009-2014 or so, that just wouldn't have been possible if their primary objective was flawless code. The code isn't the product, the product is the product. In all my years of engineering I've yet to have an end-user tell me they liked my coding style, they've always been more concerned with what I'd built them.

hollowturtle 1y ago
Facebook didn't solved any real problem, "Move fast and break things" is for investors not hackers. Famously gmail was very good quality web app and code(can't say the same today) surely not the product of today's "fast iteration" culture
- nc 1y ago
  Meta has a 1.79T market cap, they definitely solved some very real problems to get there. There are lots of companies doing well producing high quality products out of the gate today though, look at Linear. Both approaches are valid for building sustainable enduring businesses.
  - hollowturtle 1y ago
    > Meta has a 1.79T market cap, they definitely solved some very real problems to get there. Sure exploiting human attention to sell them ads, it's not technological marvel, I'd say psychological
    - nc 1y ago
      Pretty negative take, TV and Radio also exploit human attention in your book? Technology wise they created React, PyTorch, GraphQL and Llama is open weights.

athrowaway3z 1y ago

You see a large % of failures, but you're drawing an unsupported conclusion. We all agree, the people that _feel_ the most productivity spike are the sub-par engineers. That shouldn't be controversial, and it's even predictable. But their volume can't be taken as an argument one way or the other. The question is, are there _any_ good engineers that don't just feel more productive, but objectively are.

mekoka 1y ago

People are constantly looking for definite tendencies and magic patterns so they can abdicate situational awareness and critical thinking. We observe that fast delivery has often correlated with success in software and we infer that fast delivery is a factor of success. Then it becomes about the mindless pursuit of the measure, speed of delivery, as Goodhart's law predicts. Let's even concede that speed of delivery indeed is an actual factor, there has to be a threshold. There's a point where people just don't care how fast you're putting out features, because your product has found its sweet spot and is perfectly scratching its market's itch. A few will clearly notice when the line of diminishing returns is crossed and if they can reset their outlook to fit the new context, a continuous focus on speed of delivery will look increasingly obsessive and nonsensical. But that's the reality of the majority of the software development world. Few of us work on anything mission critical. We could produce nice sane software at a reasonable pace with decent output, but we're sold the idea that there's always more productivity to squeeze and we're told that we really really want that juice.

wartywhoa23 1y ago
That all things develop only so much before they degrade into overdevelopment was a very well understood phenomenon for ancient Taoists, and it will be the death of the modern Blackrock/Vanguard owned world which is absolutely ignorant of this principle.

gwbas1c 1y ago

> why do we need all these unit tests in the first place? The same reason we've always needed them: 1: They prevent regressions. (IE, bugs in features that were shipped and already working.) 2: They are very easy to run at the push of a button in your IDE. (But in this context the LLM runs them.) 3: They run in CI. This is an important line of defense in making sure a pull request doesn't introduce a bug. Now, depending on what you're writing, you might not need unit tests! Perhaps you're trying to get a minimum viable product out the door? Perhaps you're trying to demo a feature to see if it's worth building? Perhaps you're writing a 1-off tool that you'll run a few times and throw away? But, understand that if you're writing an industrial-strength program, your unit tests help you ship bug-free software. They allow you to do some rather major refactors, sometimes touching areas of the codebase that you only lightly understand, without needing to manually test everything. (And, to keep it in context,) your LLM will also have the same benefits from this tired-and-true process.

hollowturtle 1y ago
Thanks for the non solicited lesson, I learned basically nothing new. That unit tests ships bug free software is such an overstatement... Btw please go re read my comment and try to understand what I was actually trying to argue about, and I also wrote that I think they can be useful...