4 ms·

> I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive

by troupo 1y ago

> I have never heard anybody successfully using LLMs say this before. Most of what I've learned from talking to people about their workflows is counterintuitive and subtle. Because for all our posturing about being skeptical and data driven we all believe in magic. Those "counterintuitive non-trivial workflows"? They work about as well as just prompting "implement X" with no rules, agents.md, careful lists etc. Because 1) literally no one actually measures whether magical incarnations work and 2) it's impossible to make such measurements due to non-determinism

roxolotl 1y ago
On top of this a lot of the “learning to work with LLMs” is breaking down tasks into small pieces with clear instructions and acceptance criteria. That’s just part of working efficiently but maybe don’t want to be bothered to do it.
- georgeburdell 1y ago
  Working efficiently as a team, perhaps, but during solo development this is unnecessary beyond the extent that is necessary to document the code
  - tptacek 1y ago
    Even this opens up a whole field of weird subtle workflow tricks people have, because people run parallel asynchronous agents that step on each other in git. Solo developers run teams now! Really wild to hear someone say out loud "there's no learning curve to using this stuff".
    - troupo 1y ago
      The "learning curve" is reading "experts opinion" on the ever-changing set of magical rituals that may or may not work but trust us it works.
      - azan_ 1y ago
        No, you do not need to trust anyone, you can just verify what works and what doesn't, it's very easy.
        troupo 1y ago
        Indeed. And it's extremely easy to verify my original comment: https://news.ycombinator.com/item?id=44849887 https://news.ycombinator.com/item?id=44849887
simonw 1y ago
The problem with your argument here is that you're effectively saying that developers (like myself) who put effort into figuring out good workflows for coding with LLMs are deceiving themselves, and are effectively wasting their time. Either I've wasted significant chunks of the past ~3 years of my life or you're missing something here. Up to you to decide which you believe. I agree that it's hard to take solid measurements due to non-determinism. The same goes for managing people, and yet somehow many good engineering managers can judge if their team is performing well and figure out what levers they can pull to help them perform better.
- tptacek 1y ago
  Another problem with it is that you could have said the same thing about virtually any advancement in programming over the last 30 years.
  - Tainnor 1y ago
    There have been so many "advances" in software development in the last decades - powerful type systems, null safety, sane error handling, Erlang-style fault tolerance, property testing, model checking, etc. - and yet people continue to write garbage code in unsafe languages with underpowered IDEs. I think many in the industry have absolutely no clue what they're doing and are bad at evaluating productivity, often prioritising short term delivery over longterm maintenance. LLMs can absolutely be useful but I'm very concerned that some people just use them to churn out code instead of thinking more carefully about what and how to build things. I wish we had at least the same amount of discussions about those things I mentioned above as we have about whether Opus, Sonnet, GPT5 or Gemini is the best model.
    - Karrot_Kream 1y ago
      > I wish we had at least the same amount of discussions about those things I mentioned above as we have about whether Opus, Sonnet, GPT5 or Gemini is the best model. I mean we do. I think programmers are more interested in long term maintainable software than its users are. Generally that makes sense, a user doesn't really care how much effort it takes to add features or fix bugs, these are things that programmers care about. Moreover the cost of mistakes of most software is so low that most people don't seem interested in paying extra for more reliable software. The few areas of software that require high reliability are the ones regulated or are sold by companies that offer SLAs or other such reliability agreements. My observation over the years is that maintainability and reliability are much more important to programmers who comment in online forums than they are to users. It usually comes with the pride of work that programmers have but my observation is that this has little market demand.

by troupo 1y ago

roxolotl 1y ago

On top of this a lot of the “learning to work with LLMs” is breaking down tasks into small pieces with clear instructions and acceptance criteria. That’s just part of working efficiently but maybe don’t want to be bothered to do it.

georgeburdell 1y ago
Working efficiently as a team, perhaps, but during solo development this is unnecessary beyond the extent that is necessary to document the code
- tptacek 1y ago
  Even this opens up a whole field of weird subtle workflow tricks people have, because people run parallel asynchronous agents that step on each other in git. Solo developers run teams now! Really wild to hear someone say out loud "there's no learning curve to using this stuff".
  - troupo 1y ago
    The "learning curve" is reading "experts opinion" on the ever-changing set of magical rituals that may or may not work but trust us it works.
    - azan_ 1y ago
      No, you do not need to trust anyone, you can just verify what works and what doesn't, it's very easy.
      - troupo 1y ago
        Indeed. And it's extremely easy to verify my original comment: https://news.ycombinator.com/item?id=44849887 https://news.ycombinator.com/item?id=44849887

simonw 1y ago

The problem with your argument here is that you're effectively saying that developers (like myself) who put effort into figuring out good workflows for coding with LLMs are deceiving themselves, and are effectively wasting their time. Either I've wasted significant chunks of the past ~3 years of my life or you're missing something here. Up to you to decide which you believe. I agree that it's hard to take solid measurements due to non-determinism. The same goes for managing people, and yet somehow many good engineering managers can judge if their team is performing well and figure out what levers they can pull to help them perform better.

tptacek 1y ago
Another problem with it is that you could have said the same thing about virtually any advancement in programming over the last 30 years.
- Tainnor 1y ago
  There have been so many "advances" in software development in the last decades - powerful type systems, null safety, sane error handling, Erlang-style fault tolerance, property testing, model checking, etc. - and yet people continue to write garbage code in unsafe languages with underpowered IDEs. I think many in the industry have absolutely no clue what they're doing and are bad at evaluating productivity, often prioritising short term delivery over longterm maintenance. LLMs can absolutely be useful but I'm very concerned that some people just use them to churn out code instead of thinking more carefully about what and how to build things. I wish we had at least the same amount of discussions about those things I mentioned above as we have about whether Opus, Sonnet, GPT5 or Gemini is the best model.
  - Karrot_Kream 1y ago
    > I wish we had at least the same amount of discussions about those things I mentioned above as we have about whether Opus, Sonnet, GPT5 or Gemini is the best model. I mean we do. I think programmers are more interested in long term maintainable software than its users are. Generally that makes sense, a user doesn't really care how much effort it takes to add features or fix bugs, these are things that programmers care about. Moreover the cost of mistakes of most software is so low that most people don't seem interested in paying extra for more reliable software. The few areas of software that require high reliability are the ones regulated or are sold by companies that offer SLAs or other such reliability agreements. My observation over the years is that maintainability and reliability are much more important to programmers who comment in online forums than they are to users. It usually comes with the pride of work that programmers have but my observation is that this has little market demand.