5 ms·

If AI is supposed to deliver on this magical no-lift ease of use task flexibility that everyone likes to talk about I think it should be able to work with a REA

If AI is supposed to deliver on this magical no-lift ease of use task flexibility that everyone likes to talk about I think it should be able to work with a README instead of clogging up ALL of my directories with yet another fucking config file. Also this isn’t portable to other potential AI tools. Do I need 3+ md files in every directory?

stingraycharles 11mo ago
It’s not delivering on magical stuff. Getting real productivity improvements out of this requires engineering and planning and it needs to be approached as such. One of the big mistakes I think is that all these tools are over-promising on the “magic” part of it. It’s not. You need to really learn how to use all these tools effectively. This is not done in days or weeks even, it takes months in the same way becoming proficient in eMacs or vim or a programming language is. Once you’ve done that, though, it can absolutely enhance productivity. Not 10x, but definitely in the area of 2x. Especially for projects / domains you’re uncomfortable with. And of course the most important thing is that you need to enjoy all this stuff as well, which I happen to do. I can totally understand the resistance as it’s a shitload of stuff you need to learn, and it may not even be relevant anymore next year.
- giancarlostoro 11mo ago
  Yeah I feel like on average I still spend a similar amount of time developing but drastically less time fixing obscure bugs, because once it codes the feature and I describe the bugs it fixed them, the rest of my times spent testing and reviewing code.
- tsimionescu 11mo ago
  While I believe you're probably right that getting any productivity gains from these tools requires an investment, I think calling the process "engineering" is really stretching the meaning of the word. It's really closer to ritual magic than any solid engineering practices at this point. People have guesses and practices that may or may not actually work for them (since measuring productivity increases is difficult if not impossible), and they teach others their magic formulas for controlling the demon.
  - gtaylor 11mo ago
    Most countries don’t have a notion of a formally licensed software engineer, anyway. Arguing what is and is not engineering is not useful.
    - llbeansandrice 11mo ago
      I think it's relevant when people keep using terms like "prompt engineering" to try and beef up this charade of md files that don't even seem to work consistently. This is a far far cry from even writing yaml for Github/Gitlab CICD pipelines. Folks keep trying to say "engineering" when every AI thread like this seems to push me more towards "snake oil" as an appropriate term.
      - stingraycharles 11mo ago
        Prompt engineering is a real thing though, but it’s not related to markdown files etc. If you’re not benchmarking and systematically measuring the impact of your changes, it’s not prompt engineering, it’s just improving stuff.
    - tsimionescu 11mo ago
      Most countries don't have a notion of a formally licenses physicist either. That doesn't make it right to call astrology physics. And all of the practices around using LLM agents for coding are a lot closer to astrology than they are to astronomy. I was replying to someone who claimed that getting real productivity gains from this tool requires engineering and needs to be approached as such. It also compared learning to use LLM agents to learning to code in emacs or vim, or learning a programming language - things which are nothing alike to learning to control an inherently stochastic tool that can't even be understood using any of our regular scientific methods.
  - stingraycharles 11mo ago
    Well it requires a lot of planning, spec’ing, and validation. The whole point is to get a process around it that works and gets the “ritual magic” out of it. I labeled that as engineering.
- fpauser 11mo ago
  >> [..] and it may not even be relevant anymore next year. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- nineteen999 11mo ago
  Learning how to equip a local LLM with tools it can use to interact with to extend its capabilities has been a lot of fun for me and is a great educational experience for anyone who is interested. Just another tool for the toolchest.
- llbeansandrice 11mo ago
  My issue is not with learning. This "tool" has an incredibly shallow learning curve. My issue is that I'm having to make way for these "tools" that everyone says vastly increases productivity but seems to just churn out tech-debt as quickly as it can write it. It a large leap to "requires engineering and planning" when no one even in this thread can seem to agree on the behavior of any of these "tools". Some comments tell anecdotes of not getting the agents to listen until the context of the whole world is laid out in these md files. Others say the only way is to keep the context tight and focused, going so far as to have written _yet more tools_ to remove and re-add code comments so they don't "poison" the context. I am slightly straw-manning, but the tone in this thread has already shifted from a few months ago where these "tools" were going to immediately give huge productivity gains but now you're telling me they need 1) their own special files everywhere (again, this isn't even agreed on) and 2) "engineering and planning...not done in days or weeks even The entire economy is propped up on this tech right now and no one can even agree on whether it's effective or how to use it properly? Not to mention the untold damage it is doing to learning outcomes.
- deleted 11mo ago
  [deleted]
whywhywhywhy 11mo ago
> Do I need 3+ md files in every directory? Don’t worry, as of about 6 weeks ago when they changed the system prompt Claude will make sure every folder has way more than 3 .md files seen as it often writes 2 or more per task so if you don’t clean them up…
- solumunus 11mo ago
  Strange. I haven’t experienced this a single time and I use it almost all day everyday.
  - beardedwizard 11mo ago
    That is strange because it's been going on since sonnet 4.5 release.
    - adobesubmarine 11mo ago
      I also have seen this happen since then, but I actually like it. The .md files never make it into a commit, but they are quite handy for PR drafts.
    - solumunus 11mo ago
      I wonder if it's because I have instructions for Claude to add comment blocks with explanations of behavior, etc that it can self reference in future. I guess that is filling the role that these .md files would.
mewpmewp2 11mo ago
Is your logic that unless something is perfect it should not be used even though it is delivering massive productivity gains?
- swiftcoder 11mo ago
  > it is delivering massive productivity gains [citation needed] Every article I can find about this is citing the valuation of the S&P500 as evidence of the productivity gains, and that feels very circular
  - mewpmewp2 11mo ago
    What kind of citation or evidence would you expect? How would you measure productivity gains? From my personal life and activities however it is very clear. I have simply been able to do so many things I would have never been able to do without AI.
    - adobesubmarine 11mo ago
      I feel similarly, but the other guy's point is still good; there is no empirical evidence that AI is really as helpful as we think, and there is empirical evidence that among those who perceive productivity gains, most are wrong and the rest are overestimating the effect. Our experiences don't agree with the established facts, but the plural of "anecdote" is not "data."
      - mewpmewp2 10mo ago
        I don't think it is something you can scientifically measure that easily and the studies that have tried are flawed. You can measure factory work, but you can't measure AI and software eng productivity gains or loss objectively.
    - swiftcoder 11mo ago
      > How would you measure productivity gains? I'm not entirely sure - certainly anecdotes abound (as do anecdotes to the contrary). I am however sure that when someone claims that a technology is world-changing, and that it has already led to significant productivity gains... the onus is on them to provide evidence for such claims

stingraycharles 11mo ago

It’s not delivering on magical stuff. Getting real productivity improvements out of this requires engineering and planning and it needs to be approached as such. One of the big mistakes I think is that all these tools are over-promising on the “magic” part of it. It’s not. You need to really learn how to use all these tools effectively. This is not done in days or weeks even, it takes months in the same way becoming proficient in eMacs or vim or a programming language is. Once you’ve done that, though, it can absolutely enhance productivity. Not 10x, but definitely in the area of 2x. Especially for projects / domains you’re uncomfortable with. And of course the most important thing is that you need to enjoy all this stuff as well, which I happen to do. I can totally understand the resistance as it’s a shitload of stuff you need to learn, and it may not even be relevant anymore next year.

giancarlostoro 11mo ago
Yeah I feel like on average I still spend a similar amount of time developing but drastically less time fixing obscure bugs, because once it codes the feature and I describe the bugs it fixed them, the rest of my times spent testing and reviewing code.
tsimionescu 11mo ago
While I believe you're probably right that getting any productivity gains from these tools requires an investment, I think calling the process "engineering" is really stretching the meaning of the word. It's really closer to ritual magic than any solid engineering practices at this point. People have guesses and practices that may or may not actually work for them (since measuring productivity increases is difficult if not impossible), and they teach others their magic formulas for controlling the demon.
- gtaylor 11mo ago
  Most countries don’t have a notion of a formally licensed software engineer, anyway. Arguing what is and is not engineering is not useful.
  - llbeansandrice 11mo ago
    I think it's relevant when people keep using terms like "prompt engineering" to try and beef up this charade of md files that don't even seem to work consistently. This is a far far cry from even writing yaml for Github/Gitlab CICD pipelines. Folks keep trying to say "engineering" when every AI thread like this seems to push me more towards "snake oil" as an appropriate term.
    - stingraycharles 11mo ago
      Prompt engineering is a real thing though, but it’s not related to markdown files etc. If you’re not benchmarking and systematically measuring the impact of your changes, it’s not prompt engineering, it’s just improving stuff.
  - tsimionescu 11mo ago
    Most countries don't have a notion of a formally licenses physicist either. That doesn't make it right to call astrology physics. And all of the practices around using LLM agents for coding are a lot closer to astrology than they are to astronomy. I was replying to someone who claimed that getting real productivity gains from this tool requires engineering and needs to be approached as such. It also compared learning to use LLM agents to learning to code in emacs or vim, or learning a programming language - things which are nothing alike to learning to control an inherently stochastic tool that can't even be understood using any of our regular scientific methods.
- stingraycharles 11mo ago
  Well it requires a lot of planning, spec’ing, and validation. The whole point is to get a process around it that works and gets the “ritual magic” out of it. I labeled that as engineering.
fpauser 11mo ago
>> [..] and it may not even be relevant anymore next year. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
nineteen999 11mo ago
Learning how to equip a local LLM with tools it can use to interact with to extend its capabilities has been a lot of fun for me and is a great educational experience for anyone who is interested. Just another tool for the toolchest.
llbeansandrice 11mo ago
My issue is not with learning. This "tool" has an incredibly shallow learning curve. My issue is that I'm having to make way for these "tools" that everyone says vastly increases productivity but seems to just churn out tech-debt as quickly as it can write it. It a large leap to "requires engineering and planning" when no one even in this thread can seem to agree on the behavior of any of these "tools". Some comments tell anecdotes of not getting the agents to listen until the context of the whole world is laid out in these md files. Others say the only way is to keep the context tight and focused, going so far as to have written _yet more tools_ to remove and re-add code comments so they don't "poison" the context. I am slightly straw-manning, but the tone in this thread has already shifted from a few months ago where these "tools" were going to immediately give huge productivity gains but now you're telling me they need 1) their own special files everywhere (again, this isn't even agreed on) and 2) "engineering and planning...not done in days or weeks even The entire economy is propped up on this tech right now and no one can even agree on whether it's effective or how to use it properly? Not to mention the untold damage it is doing to learning outcomes.
deleted 11mo ago
[deleted]

whywhywhywhy 11mo ago

> Do I need 3+ md files in every directory? Don’t worry, as of about 6 weeks ago when they changed the system prompt Claude will make sure every folder has way more than 3 .md files seen as it often writes 2 or more per task so if you don’t clean them up…

solumunus 11mo ago
Strange. I haven’t experienced this a single time and I use it almost all day everyday.
- beardedwizard 11mo ago
  That is strange because it's been going on since sonnet 4.5 release.
  - adobesubmarine 11mo ago
    I also have seen this happen since then, but I actually like it. The .md files never make it into a commit, but they are quite handy for PR drafts.
  - solumunus 11mo ago
    I wonder if it's because I have instructions for Claude to add comment blocks with explanations of behavior, etc that it can self reference in future. I guess that is filling the role that these .md files would.

mewpmewp2 11mo ago

Is your logic that unless something is perfect it should not be used even though it is delivering massive productivity gains?

swiftcoder 11mo ago
> it is delivering massive productivity gains [citation needed] Every article I can find about this is citing the valuation of the S&P500 as evidence of the productivity gains, and that feels very circular
- mewpmewp2 11mo ago
  What kind of citation or evidence would you expect? How would you measure productivity gains? From my personal life and activities however it is very clear. I have simply been able to do so many things I would have never been able to do without AI.
  - adobesubmarine 11mo ago
    I feel similarly, but the other guy's point is still good; there is no empirical evidence that AI is really as helpful as we think, and there is empirical evidence that among those who perceive productivity gains, most are wrong and the rest are overestimating the effect. Our experiences don't agree with the established facts, but the plural of "anecdote" is not "data."
    - mewpmewp2 10mo ago
      I don't think it is something you can scientifically measure that easily and the studies that have tried are flawed. You can measure factory work, but you can't measure AI and software eng productivity gains or loss objectively.
  - swiftcoder 11mo ago
    > How would you measure productivity gains? I'm not entirely sure - certainly anecdotes abound (as do anecdotes to the contrary). I am however sure that when someone claims that a technology is world-changing, and that it has already led to significant productivity gains... the onus is on them to provide evidence for such claims