4 ms·

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and ho

I think this has put into words a reason why I bounced off using AI this way, when I need something done I often have a rough idea of how I want it done, and how AI does it often doesn't match what I want, but because it's gone off and written a 2,000 lines of code it's suddenly more work for me to go through and say "Ok, so first off, strip all these comments out, you're doubling the file with trivial explanations of simple code. I don't want X to be abstracted this way, I want that...." etc. And then when I give it feedback 2,000 lines of code suddenly switch to 700 lines of completely different code and I can't keep up. And I don't want my codebase full of disjoint scripts that I don't really understand and all have weirdly different approaches to the problem. I want an AI that I have similar opinions to, which is obviously tough. It's like working with someone on their first day. I don't know if it's giving the tools less self-confidence per se, but I think it's exposing more the design process. Like ideally you want your designer to go "Ok, I'm thinking of this approach, i'll probably have these sorts of functions or classes, this state will be owned here" and we can approve that first, rather than going straight from prompt -> implementation.

searls 1y ago
Yeah, and then you just wind up feeling exhausted AND unsatisfied with where you wind up. You are exactly who I posted this for. 100% of my positive experiences with agent coding are when I don't have reason to care about the intrinsic qualities of the code (one-off scripts or genuine leaf node functions that can't impact anything else.
artursapek 1y ago
You need to tune it.
theshrike79 1y ago
The trick is to have rules specific to the project and your programming style & preferences. Think of the AI like an outsourced consultant who will never say no. It'll always do everything it can to solve the problem you've given it. If it doesn't know how, it'll write a thousand lines when a single additional dependency and 2 lines of code would've done it.
Spooky23 1y ago
Personally, I’ve gone from working with the AI to code to working on it to develop specifications. It’s also useful at troubleshooting issues. I’m no longer a developer by trade and it’s more a hobby or specific problem solving scenario now. But I find using it to identify gaps in my thinking and edit English is ultimately better than getting random code — I love editing English text, but find editing code without consistent style a drag for my purposes.
ozim 1y ago
At the start of prompt before project requirements I copy paste paragraph about the code I want. No emojis, no comments, no console log statements, no read me file, no error handling. Act as a senior developer working with other experienced developers. Otherwise it happily generates bunch of trash that is unreadable. Error handling generated will most of the times just hide errors instead of actually dealing with them.
- ta12653421 1y ago
  initial prompting like this has a huge impact, yes. also: I clean the chat and start over sometimes, because results may differ.
iamkoch 1y ago
This absolutely captures my experience. My successful AI written projects are those where I care solely on the output and have little to no knowledge about the subject matter. When I try to walk an agent through creating anything about which I have a deeply held opinion of what good looks like, I end up frustrated and abandoning the project. I've enjoyed using roo code's architect function to document an agreed upon approach, then been delighted and frustrated in equal measure by the implementation of code mode. On revelation is to always start new tasks and avoid continuing large conversations, because I would typically tackle any problem myself in smaller steps with verifiable outputs, whereas I tend to pose the entire problem space to the agent which it invariably fails at. I've settled on spending time finding what works for me. Earlier today I took 30 minutes to add functionality to an app that would've taken me days to write. And what's more I only put 30 minutes into the diary for it, because I knew what I wanted and didn't care how it got there. This leads me to conclude that using AI to write code that a(nother) human is one day to interact with is a no-go, for all the reasons listed.
- rlewkov 1y ago
  > "This leads me to conclude that using AI to write code that a(nother) human is one day to interact with is a no-go, for all the reasons listed." So, if one's goal is to develop code that is easily maintainable by others, do you think that AI writing code gets in the way of that goal?
bgro 1y ago
Your ai is generating 2000 line code chunks? Are you prompting it to create the entire Skyrim game for SNES? Then after taking long lunch, getting mad when you press run and you find out it made fallout with only melee weapons in a ps1 style?
cmrdporcupine 1y ago
I don't have this experience with Claude, frankly. I do have to correct it, but I try to give very specific prompts with very specific instructions. IT does well with highly commented code. Now, I have the best luck in my personal project codebase, which I know extremely intimately so can be very surgical with. Work, which has far less comments and is full of very high level abstractions that I don't know as well.. it struggles with. We both do. It's a fine pair programmer when one of the pairs knows the codebase extremely well. It's a bad companion elsewhere.
jstummbillig 1y ago
People are thinking too much of humans and LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort). This is simply not the case when doing AI coding, and you need to adjust how you work because of that and play to the strengths of this setup, if you want to get something out of it and not frustrate yourself. Here is how to do this: Have it generate something. That first 2000 lines of not so great first attempt code, don't even think about understanding all of that, or, worse, about correcting it. Review it loosely. You are not dealing with a human! There is absolutely no need to be thorough or nice. You are not hurting any feelings. Go for 80/20 (or the best ratio you think you can get). Then, think: - Anything you missed to inform the AI about? Update your initial prompt - Anything the AI simply does not do well or to your liking? Write general instructions (all of the IDEs have some way of doing that) that are very explicit about what you don't want to see again, and what you want to see instead. Then revert everything the ai did, and have it go again from the start. You should approach something that's better.
- Sharlin 1y ago
  This approach is essentially the PR workflow preferred by the author. Why let an LLM make huge changes to your working copy just for you to revert them next, instead of just writing patches to be asynchronously reviewed? What you propose is no way of doing pair programming in particular, and seems to support the author’s argument.
  - jstummbillig 1y ago
    1. There is not a mention of "pair programming" in the comment I was addressing. As often happens, the discussion evolves. 2. The point is, that you are training the AI through this process. You can do pair programming afterwards (or not). Aim to instruct it to give you ballpark answers first, and take it from there.
- afavour 1y ago
  > LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort) At this point AI generated code absolutely requires review by a human so LOC is still an important metric.
34679 1y ago
Just like with human engineers, you need to start with a planning session. This involves a back and forth discussion to hammer out the details before writing any code. I start off as vague as possible to see if the LLM recommends anything I hadn't thought of, then get more detailed as I go. When I'm satisfied, I have it create 2 documents, initialprompt.txt and TODO.md. The initial prompt file includes a summary of the project along with instructions to read the to do file and mark each step as complete after finishing it. This ensures the LLM has a complete understanding of the overall goals, along with a step by step list of tasks to get there. It also allows me to quickly get the LLM back up to speed when I need to start a new conversation due to context limits.
- globnomulous 1y ago
  In essence, I need to schedule a meeting with the LLM and 'hammer out a game plan.' Gotta make sure we're 'in sync' and everybody's 'on the same page.' Meeting-based programming. No wonder management loves it and thinks it should be the future.
  - woah 1y ago
    LLMs are stealing the jobs of developers who go off half-cocked and spend three days writing 2000 lines of code implementing the wrong feature instead of attending a 30 minute meeting
    - devmor 1y ago
      and the jobs of developers that want to schedule another breakout session to discuss the pros and cons of a 2-line change.
      - ge96 1y ago
        Yeah... I'm gonna need to circle back on that
    - hnthrow90348765 1y ago
      That's dumb, of course, but sometimes people really just do the bare minimum to describe what they want and they can only think clearly once there's something in front of them. The 2000 lines there should be considered a POC, even at 2000 lines.
slfnflctd 1y ago
> "Ok, I'm thinking of this approach, i'll probably have these sorts of functions or classes, this state will be owned here" This is the gist of what I've always wanted from a programming mentor, instructor, or tutor. It can be surprisingly hard to find. Knowing that current LLMs still struggle with it perhaps helps explain why.
tristor 1y ago
> It's like working with someone on their first day. This matches my experience exactly, but worse than working with a human on their first day, day 100 for an AI is still like working with them on their first day. Humans have effectively infinite context windows over a long enough period of time, AIs context windows are so constrained that it's not worthwhile to invest the effort to 'teach' it like you would a junior engineer.
- SirHumphrey 1y ago
  It’s not really that humans have infinite context windows, it’s more that the context windows are a very poor substitutes for long term memory. Memory even in a text heavy field like programming is not only text based so it’s often hard to describe for example an appropriate amount of error checking in prompt.md. Giving a person with anterograde amnesia a book of everything they know - no matter how well indexed or how searchable will not fix the lack of long term memory.
bakkoting 1y ago
Anthropic's guide to using Claude Code [1] is worth reading. Specifically, their recommended workflow is "first ask it to read the code, then ask it to make a plan to implement your change, then tell it to execute". That sounds like the workflow you're asking for - you can read its plan and make adjustments before it writes a single line of code. One of the weird things about using agents is that if they're doing things in a way you don't like, including things like writing code without first running the design by you, you can simply ask them to do things a different way. [1] https://www.anthropic.com/engineering/claude-code-best-practices https://www.anthropic.com/engineering/claude-code-best-pract...
- woah 1y ago
  > you can simply ask them to do things a different way Instead of a writing a blog post about how they didn't guess how you wanted things done?
- ta12653421 1y ago
  good one! I#m wondering how some can complain about ClaudeAI: - its actually enlightening - it saves a lot of time - by intuition, i did whats written in this blog from the beginning on YES: - sometimes the solution is rubish because i can see that its "randomly" is connecting/integrating stuff - ...but: In about 95% of the cases the output is exactly what i asked for
zild3d 1y ago
> I want an AI that I have similar opinions to, which is obviously tough. It's like working with someone on their first day. Most of what you're describing does apply to humans on the first day, and ais on their first day. If you aren't capturing these preferences somewhere and giving it to either a human or the ai, then why would they somehow know your preferences? For ai, the standard thats forming is you create some markdown file(s) with these so they only need to be explained once, and auto provided as context.
Horde 1y ago
There is an old issue, that's related, why one should avoid using equality relations in AI used for creating a proof. It will go back and forth and fill the log with trivial statements before it comes to the right proof path. This might end up being the majority of the proof and could be an unbounded part. Then somebody has to read that and spend a good deal of time deciding what's trivial and what isn't. Whereas if you remove equality, you have something that isn't very natural.
chermi 1y ago
I prompt it to come up with 3 potential implementation plans, choose which one it thinks is best, and explain its plan to me before it does anything. I also ask it to enumerate which files/functions it will modify in its chosen plan. Then you can give feedback on what it thinks and have it come up with a new plan if you don't like it. Every bit of context and constraints you give it helps. Having a design doc + a little description of your design/code "philosophy" helps. This is easy to do in cursor with rules, I'm guessing other tools have a similar feature. Also, if there's a somewhat similar feature implemented already or if you have a particular style, tell it to reference example files/code snippets.
nixpulvis 1y ago
I honestly don't expect to use AI tools extensively for code generation until we figure out how to have the models learn and become accustomed to me aside from clever context prompting. I want my own models derived from the baseline. That said, I also value not becoming too dependent on any service which isn't free and efficient. Relying on a CNC machine when you never learned how to whittle strips me of a sense of security and power I'm not comfortable with.
schwartzworld 1y ago
> but because it's gone off and written a 2,000 lines of code That’s a you problem, not an AI problem. You have to give it small tasks broken down the same way you would break them down.
flashgordon 1y ago
This is kind of why I hardly use one shot/Agentic flows that the cursors are recomending you do (after all it gets them more control and lockin). I mostly know "how" I want something done so put fences to ensure what is generated is in the X loc ballpark and in an incremental way. They are however great to reason through ideas and explore solutions before writing any code.

searls 1y ago

Yeah, and then you just wind up feeling exhausted AND unsatisfied with where you wind up. You are exactly who I posted this for. 100% of my positive experiences with agent coding are when I don't have reason to care about the intrinsic qualities of the code (one-off scripts or genuine leaf node functions that can't impact anything else.

artursapek 1y ago

You need to tune it.

theshrike79 1y ago

The trick is to have rules specific to the project and your programming style & preferences. Think of the AI like an outsourced consultant who will never say no. It'll always do everything it can to solve the problem you've given it. If it doesn't know how, it'll write a thousand lines when a single additional dependency and 2 lines of code would've done it.

Spooky23 1y ago

Personally, I’ve gone from working with the AI to code to working on it to develop specifications. It’s also useful at troubleshooting issues. I’m no longer a developer by trade and it’s more a hobby or specific problem solving scenario now. But I find using it to identify gaps in my thinking and edit English is ultimately better than getting random code — I love editing English text, but find editing code without consistent style a drag for my purposes.

ozim 1y ago

At the start of prompt before project requirements I copy paste paragraph about the code I want. No emojis, no comments, no console log statements, no read me file, no error handling. Act as a senior developer working with other experienced developers. Otherwise it happily generates bunch of trash that is unreadable. Error handling generated will most of the times just hide errors instead of actually dealing with them.

ta12653421 1y ago
initial prompting like this has a huge impact, yes. also: I clean the chat and start over sometimes, because results may differ.

iamkoch 1y ago

This absolutely captures my experience. My successful AI written projects are those where I care solely on the output and have little to no knowledge about the subject matter. When I try to walk an agent through creating anything about which I have a deeply held opinion of what good looks like, I end up frustrated and abandoning the project. I've enjoyed using roo code's architect function to document an agreed upon approach, then been delighted and frustrated in equal measure by the implementation of code mode. On revelation is to always start new tasks and avoid continuing large conversations, because I would typically tackle any problem myself in smaller steps with verifiable outputs, whereas I tend to pose the entire problem space to the agent which it invariably fails at. I've settled on spending time finding what works for me. Earlier today I took 30 minutes to add functionality to an app that would've taken me days to write. And what's more I only put 30 minutes into the diary for it, because I knew what I wanted and didn't care how it got there. This leads me to conclude that using AI to write code that a(nother) human is one day to interact with is a no-go, for all the reasons listed.

rlewkov 1y ago
> "This leads me to conclude that using AI to write code that a(nother) human is one day to interact with is a no-go, for all the reasons listed." So, if one's goal is to develop code that is easily maintainable by others, do you think that AI writing code gets in the way of that goal?

bgro 1y ago

Your ai is generating 2000 line code chunks? Are you prompting it to create the entire Skyrim game for SNES? Then after taking long lunch, getting mad when you press run and you find out it made fallout with only melee weapons in a ps1 style?

cmrdporcupine 1y ago

I don't have this experience with Claude, frankly. I do have to correct it, but I try to give very specific prompts with very specific instructions. IT does well with highly commented code. Now, I have the best luck in my personal project codebase, which I know extremely intimately so can be very surgical with. Work, which has far less comments and is full of very high level abstractions that I don't know as well.. it struggles with. We both do. It's a fine pair programmer when one of the pairs knows the codebase extremely well. It's a bad companion elsewhere.

jstummbillig 1y ago

People are thinking too much of humans and LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort). This is simply not the case when doing AI coding, and you need to adjust how you work because of that and play to the strengths of this setup, if you want to get something out of it and not frustrate yourself. Here is how to do this: Have it generate something. That first 2000 lines of not so great first attempt code, don't even think about understanding all of that, or, worse, about correcting it. Review it loosely. You are not dealing with a human! There is absolutely no need to be thorough or nice. You are not hurting any feelings. Go for 80/20 (or the best ratio you think you can get). Then, think: - Anything you missed to inform the AI about? Update your initial prompt - Anything the AI simply does not do well or to your liking? Write general instructions (all of the IDEs have some way of doing that) that are very explicit about what you don't want to see again, and what you want to see instead. Then revert everything the ai did, and have it go again from the start. You should approach something that's better.

Sharlin 1y ago
This approach is essentially the PR workflow preferred by the author. Why let an LLM make huge changes to your working copy just for you to revert them next, instead of just writing patches to be asynchronously reviewed? What you propose is no way of doing pair programming in particular, and seems to support the author’s argument.
- jstummbillig 1y ago
  1. There is not a mention of "pair programming" in the comment I was addressing. As often happens, the discussion evolves. 2. The point is, that you are training the AI through this process. You can do pair programming afterwards (or not). Aim to instruct it to give you ballpark answers first, and take it from there.
afavour 1y ago
> LOCs as something valuable or worth their consideration when working with AI (because usually that LOCs would have required human effort) At this point AI generated code absolutely requires review by a human so LOC is still an important metric.

34679 1y ago

Just like with human engineers, you need to start with a planning session. This involves a back and forth discussion to hammer out the details before writing any code. I start off as vague as possible to see if the LLM recommends anything I hadn't thought of, then get more detailed as I go. When I'm satisfied, I have it create 2 documents, initialprompt.txt and TODO.md. The initial prompt file includes a summary of the project along with instructions to read the to do file and mark each step as complete after finishing it. This ensures the LLM has a complete understanding of the overall goals, along with a step by step list of tasks to get there. It also allows me to quickly get the LLM back up to speed when I need to start a new conversation due to context limits.

globnomulous 1y ago
In essence, I need to schedule a meeting with the LLM and 'hammer out a game plan.' Gotta make sure we're 'in sync' and everybody's 'on the same page.' Meeting-based programming. No wonder management loves it and thinks it should be the future.
- woah 1y ago
  LLMs are stealing the jobs of developers who go off half-cocked and spend three days writing 2000 lines of code implementing the wrong feature instead of attending a 30 minute meeting
  - devmor 1y ago
    and the jobs of developers that want to schedule another breakout session to discuss the pros and cons of a 2-line change.
    - ge96 1y ago
      Yeah... I'm gonna need to circle back on that
  - hnthrow90348765 1y ago
    That's dumb, of course, but sometimes people really just do the bare minimum to describe what they want and they can only think clearly once there's something in front of them. The 2000 lines there should be considered a POC, even at 2000 lines.

slfnflctd 1y ago

> "Ok, I'm thinking of this approach, i'll probably have these sorts of functions or classes, this state will be owned here" This is the gist of what I've always wanted from a programming mentor, instructor, or tutor. It can be surprisingly hard to find. Knowing that current LLMs still struggle with it perhaps helps explain why.

tristor 1y ago

> It's like working with someone on their first day. This matches my experience exactly, but worse than working with a human on their first day, day 100 for an AI is still like working with them on their first day. Humans have effectively infinite context windows over a long enough period of time, AIs context windows are so constrained that it's not worthwhile to invest the effort to 'teach' it like you would a junior engineer.

SirHumphrey 1y ago
It’s not really that humans have infinite context windows, it’s more that the context windows are a very poor substitutes for long term memory. Memory even in a text heavy field like programming is not only text based so it’s often hard to describe for example an appropriate amount of error checking in prompt.md. Giving a person with anterograde amnesia a book of everything they know - no matter how well indexed or how searchable will not fix the lack of long term memory.

bakkoting 1y ago

Anthropic's guide to using Claude Code [1] is worth reading. Specifically, their recommended workflow is "first ask it to read the code, then ask it to make a plan to implement your change, then tell it to execute". That sounds like the workflow you're asking for - you can read its plan and make adjustments before it writes a single line of code. One of the weird things about using agents is that if they're doing things in a way you don't like, including things like writing code without first running the design by you, you can simply ask them to do things a different way. [1] https://www.anthropic.com/engineering/claude-code-best-practices https://www.anthropic.com/engineering/claude-code-best-pract...

woah 1y ago
> you can simply ask them to do things a different way Instead of a writing a blog post about how they didn't guess how you wanted things done?
ta12653421 1y ago
good one! I#m wondering how some can complain about ClaudeAI: - its actually enlightening - it saves a lot of time - by intuition, i did whats written in this blog from the beginning on YES: - sometimes the solution is rubish because i can see that its "randomly" is connecting/integrating stuff - ...but: In about 95% of the cases the output is exactly what i asked for

zild3d 1y ago

> I want an AI that I have similar opinions to, which is obviously tough. It's like working with someone on their first day. Most of what you're describing does apply to humans on the first day, and ais on their first day. If you aren't capturing these preferences somewhere and giving it to either a human or the ai, then why would they somehow know your preferences? For ai, the standard thats forming is you create some markdown file(s) with these so they only need to be explained once, and auto provided as context.

Horde 1y ago

There is an old issue, that's related, why one should avoid using equality relations in AI used for creating a proof. It will go back and forth and fill the log with trivial statements before it comes to the right proof path. This might end up being the majority of the proof and could be an unbounded part. Then somebody has to read that and spend a good deal of time deciding what's trivial and what isn't. Whereas if you remove equality, you have something that isn't very natural.

chermi 1y ago

I prompt it to come up with 3 potential implementation plans, choose which one it thinks is best, and explain its plan to me before it does anything. I also ask it to enumerate which files/functions it will modify in its chosen plan. Then you can give feedback on what it thinks and have it come up with a new plan if you don't like it. Every bit of context and constraints you give it helps. Having a design doc + a little description of your design/code "philosophy" helps. This is easy to do in cursor with rules, I'm guessing other tools have a similar feature. Also, if there's a somewhat similar feature implemented already or if you have a particular style, tell it to reference example files/code snippets.

nixpulvis 1y ago

I honestly don't expect to use AI tools extensively for code generation until we figure out how to have the models learn and become accustomed to me aside from clever context prompting. I want my own models derived from the baseline. That said, I also value not becoming too dependent on any service which isn't free and efficient. Relying on a CNC machine when you never learned how to whittle strips me of a sense of security and power I'm not comfortable with.

schwartzworld 1y ago

> but because it's gone off and written a 2,000 lines of code That’s a you problem, not an AI problem. You have to give it small tasks broken down the same way you would break them down.

flashgordon 1y ago

This is kind of why I hardly use one shot/Agentic flows that the cursors are recomending you do (after all it gets them more control and lockin). I mostly know "how" I want something done so put fences to ensure what is generated is in the X loc ballpark and in an incremental way. They are however great to reason through ideas and explore solutions before writing any code.