10 ms·

Prompt engineering playbook for programmers

464 points by vinhnx 1y ago

almosthere 1y ago
At this point it's a bit dogfooded, so why not just ask the LLM for a good prompt for the project you're working on?
sherdil2022 1y ago
So it looks like we all need to have firm understanding and tailor our prompts now to effectively use LLMs. Isn't this all subjective? I get different answers based upon how I word my question. Shouldn't things be little bit more objective? Isn't this random that I get different results based upon just wording? This whole thing is just discombobulating to me.
- fcoury 1y ago
  And to add to it, here's my experience: sometimes you spend a lot of time on this upfront prompt engineering and get bad results and sometimes you just YOLO it and get good results. It's hard to advocate for a determined strategy for prompt engineering when the tool you're prompting itself is non-deterministic. Edit: I also love that the examples come with "AI’s response to the poor prompt (simulated)"
  - emseetech 1y ago
    Also that non-determinism means every release will change the way prompting works. There's no guarantee of consistency like an API or a programming language release would have.
akkad33 1y ago
I'd rather write my own code than do all that
- echelon 1y ago
  Your boss (or CEO) probably wouldn't.
  - guappa 1y ago
    My boss cares about speed, because he doesn't sell AI.
- coffeefirst 1y ago
  Writing the code would also save you time over whatever I just read.
- Lich 1y ago
  Yeah I don’t get it. By the time I’m done writing all these prompts to get what I want, refining over and over not to mention the waiting time for the characters to appear on the screen, I could have written what I want myself. I find LLMs more useful as a quick documentation search and for the most basic refactoring and template building.
ColinEberhardt 1y ago
There are so many prompting guides at the moment. Personally I think they are quite unnecessary. If you take the time to use these tools, build familiarity with them and the way they work, the prompt you should use becomes quite obvious.
- Disposal8433 1y ago
  It reminds me that we had the same hype and FOMO when Google became popular. Books were being written on the subject and you had to buy those or you would become a caveman in a near future. What happened is that anyone could learn the whole thing in a day and that was it, no need to debate about whether you would miss anything if you didn't knew all those tools.
  - verbify 1y ago
    I certainly have better Google fu than some relatives who are always asking me to find something online.
    - Timwi 1y ago
      I love the term “Google fu”. We should call it prompt fu or LLM fu instead of “prompt engineering”.
  - wiseowise 1y ago
    You’re only proving the opposite: there’s definitely a difference between “experienced Google user” and someone who just puts random words and expects to find what they need.
    - marliechiller 1y ago
      Is there? I feel like google has optimised heavily for the caveman input rather than the enlightened search warrior nowadays
- sokoloff 1y ago
haolez 1y ago
Sometimes I get the feeling that making super long and intricate prompts reduces the cognitive performance of the model. It might give you a feel of control and proper engineering, but I'm not sure it's a net win. My usage has converged to making very simple and minimalistic prompts and doing minor adjustments after a few iterations.
- dwringer 1y ago
  I would simplify this as "irrelevant context is worse than no context", but it doesn't mean a long prompt of relevant context is bad.
- tgv 1y ago
  For another kind of task, a colleague had written a very verbose prompt. Since I had to integrate it, I added some CRUD ops for prompts. For a test, I made a very short one, something like "analyze this as a <profession>". The output was pretty much comparable, except that the output on the longer prompt contained (quite a few) references to literal parts of that prompt. It wasn't incoherent, but it was as if that model (gemini 2.5, btw) has a basic response for the task it extracts from the prompt, and merges the superfluous bits in. It would seem that, at least for this particular task, the model cannot (easily) be made to "think" differently.
- wslh 1y ago
  Same here: it starts with a relatively precise need, keeping a roadmap in mind rather than forcing one upfront. When it involves a technology I'm unfamiliar with, I also ask questions to understand what certain things mean before "copying and pasting". I've found that with more advanced prompts, the generated code sometimes fails to compile, and tracing the issues backward can be more time consuming than starting clean.
  - lodovic 1y ago
    I use specs in markdown for the more advanced prompts. I ask the llm to refine the markdown first and add implementation steps, so i can review what it will do. When it starts implementing, i can always ask it to "just implement step 1, and update the document when done". You can also ask it to verify if the spec has been implemented correctly.
neves 1y ago
Any tip that my fellow programmers find useful that's not in the article?
- didibus 1y ago
  Including a coding style guide can help the code looks like what you want. Also including an explanation of the project structure, and overall design of the code base. Always specify what libraries it should make use of (or it'll bring in anything or implement stuff a library has already). You can also make the AI review itself. Have it modify code, than ask to review the code, than ask to address review comments, and iterate until it has no more comments. Use an agentic tool like Claude Code or Amazon Q CLI. Then ask it to run tests after code changes and to address all issues until test pass. Make sure to tell it not to change the test code.
  - taosx 1y ago
    Unless your employer pays for you to use agentic tools, avoid them. They burn through money and tokens like there's no tomorrow.
- trcf22 1y ago
  I found that presenting your situation and asking for a plan/ideas + « do not give me code. Make sure you understand the requirements and ask questions if needed.» works much better for me. It also allows me to more easily control what the llm will do and not end up reviewing and throwing 200 lines of code. In a nextjs + vitest context, I try to really outline which tests I want and give it proper data examples so that it does not cheat around mocking fake objects. I do not buy into the whole you’re a senior dev etc. Most people use Claude for coding so I guess it’s engrained by default.
ofrzeta 1y ago
In the "Debugging example", the first prompt doesn't include the code but the second does? No wonder it can find the bug! I guess you may prompt as you like, as long as you provide the actual code, it usually finds bugs like this. About the roles: Can you measure a difference in code quality between the "expert" and the "junior"?
namanyayg 1y ago
It's cool to see Addy's prompts. I also wrote about some of mine! (ignoring the obvious ones): https://nmn.gl/blog/ai-prompt-engineering https://nmn.gl/blog/ai-prompt-engineering
b0a04gl 1y ago
[dead]
leshow 1y ago
using the term "engineering" for writing a prompt feels very unserious
- morkalork 1y ago
  For real. Editing prompts bares no resemblance to engineering at all, there is no accuracy or precision. Say you have a benchmark to test against and you're trying to make an improvement. Will your change to the prompt make the benchmark go up? Down? Why? Can you predict? No, it is not a science at all. It's just throwing shit and examples at the wall in hopes and prayers.
  - echelon 1y ago
    > Will your change to the prompt make the benchmark go up? Down? Why? Can you predict? No, it is not a science at all. Many prompt engineers do measure and quantitatively compare.
    - morkalork 1y ago
      Me too but it's after the fact. I make a change then measure, if it doesn't I roll back. But it's as good as witch craft or alchemy. Will I get I get gold with this adjustment? Nope, still lead. Tries variation #243 next
      - a2dam 1y ago
        This is literally how the light bulb filament was discovered.
        MegaButts 1y ago
        And Tesla famously described Edison as an idiot for this very reason. Then Tesla revolutionized the way we use electricity while Edison was busy killing elephants.
        SrslyJosh 1y ago
Avalaxy 1y ago
I feel like all of this is nonsense for people who want to pretend they are very good at using AI. Just copy pasting a stack trace with error message works perfectly fine, thank you.
- yoyohello13 1y ago
  Seriously, I seem to get good results by just "being a good communicator". Not only that, but as the tools get better, prompt engineering should get less important.
- abletonlive 1y ago
  In most HN LLM Programming discussion there's a vocal majority saying LLMs are useless. Now we have this commenter saying all they need to do is vibe and it all works out. WHICH IS IT?
  - rognjen 1y ago
    I downvoted your comment because of your first sentence. Your point makes is made even without it.
groby_b 1y ago
"State your constraints and requirements well and exhaustively". Meanwhile, I just can't get over the cartoon implying that a React Dev is just a Junior Dev who lost their hoodie.
yuvadam 1y ago
Seems like so much over (prompt) engineering. I get by just fine with pasting raw code or errors and asking plain questions, the models are smart enough to figure it out themselves.
orochimaaru 1y ago
A long time back for my MS CS I took a science of programming course. The way to verify has helped me craft prompts when I do data engineering work. Basically: Given input (…) and preconditions (…) write me spark code that gives me post conditions (…). If you can formally specify the input, preconditions and post conditions you usually get good working code. 1. Science of programming, David Gries 2. Verification of concurrent and sequential systems
nexoft 1y ago
"prompt engineering" ....ouch. I would say "common sense"\ also the problem with software engineering is that there is an inflation of SWE, too much people applying for it for compensation level rather than being good at it and really liking it, we ended up having a lot of bad software engineers that requires this crutch crap.
- adamhartenz 1y ago
  "Common sense" doesn't exist. It is a term people use when they can't explain what they actually mean.
  - ozim 1y ago
    Also how common sense can exist with LLM? There is no common sense with it - it is just an illusion.
    - wiseowise 1y ago
      Presumably it is your common sense.
  - marliechiller 1y ago
    Not sure I fully agree - sometimes maybe, but I think in the majority of cases it's used when people feel they dont need to explain exactly what they mean because it should already be obvious to most people. Example. "Always look when you cross the road" is a snippet common sense, with lack of heeding to that sense resulting in you potentially being hit by a car. Even a 4 year old wouldnt need the latter explanation, but most people could articulate that if they needed to. Its just a way of speeding up communication
    - lowbloodsugar 1y ago
      I was quite old when I realized that Common sense is literally “common experiences”. A colleague and I were lamenting a laughably bad outage that we thought showed a total lack of common sense resulting in an obvious issue. Then we both realized that the team had never had such an experience whereas the two of us had. Every member of that team now has that particular “common sense”. Likewise, “don’t run in front of cars”. As a kid, a mate broke his leg running onto the street and getting hit. I think near misses happen a lot when we’re kids. But far fewer has an “common sense” about prompt engineering because there’s just much less experience.
DebtDeflation 1y ago
In my experience there's really only three true prompt engineering techniques: - In Context Learning (providing examples, AKA one shot or few shot vs zero shot) - Chain of Thought (telling it to think step by step) - Structured output (telling it to produce output in a specified format like JSON) Maybe you could add what this article calls Role Prompting to that. And RAG is its own thing where you're basically just having the model summarize the context you provide. But really everything else just boils down to tell it what you want to do in clear plain language.
- faustocarva 1y ago
  Did you find it hard to create structured output while also trying to make it reason in the same prompt?
  - demosthanos 1y ago
    You use a two-phase prompt for this. Have it reason through the answer and respond with a clearly-labeled 'final answer' section that contains the English description of the answer. Then run its response through again in JSON mode with a prompt to package up what the previous model said into structured form. The second phase can be with a cheap model if you need it to be.
    - faustocarva 1y ago
      Great, will try this! But, in a chain-based prompt or full conversational flow?
      - demosthanos 1y ago
        You can do this conversationally, but I've had the most success with API requests, since that gives you the most flexibility. Pseudo-prompt: Prompt 1: Do the thing, describe it in detail, end with a clear summary of your answer that includes ${THINGS_YOU_NEED_FOR_JSON}. Prompt 2: A previous agent said ${CONTENT}, structure as JSON according to ${SCHEMA}. Ideally you use a model in Prompt 2 that supports JSON schemas so you have 100% guarantee that what you get back parses. Otherwise you can implement it yourself by validating it locally and sending the errors back with a prompt to fix them.
bongodongobob 1y ago
None of this shit is necessary. I feel like these prompt collections miss the point entirely. You can talk to an LLM and reason about stuff going back and forth. I mean, some things are nice to one shot, but I've found keeping it simple and just "being yourself" works much better than a page long prompt.
m3kw9 1y ago
The “you are a world class expert..” prompt to me is more like a superstious thing a sports player do before they play. I’ve used it and it still gives similar results, but maybe on a good day(that random seed) it will give me superior results
bsoles 1y ago
There is no such thing as "prompt engineering". Since when the ability to write proper and meaningful sentences became engineering? This is even worse than "software engineering". The unfortunate thing is that there will probably be job postings for such things and people will call themselves prompt engineers for their extraordinary abilities for writing sentences.
- yowlingcat 1y ago
  I would caution against thinking it's impossible even if it's not something you've personally experienced. Prompt engineering is necessary (but not sufficient) to creating high leverage outcomes from LLMs when solving complex problems. Without it, the chances of getting to a solution are slim. With it, the chances of getting to 90% of a solution and needing to fine tune the last mile are a lot higher but still not guaranteed. Maybe the phrase "prompt engineering" is bad and it really should be called "prompt crafting" because there is more an element of craft, taste, and judgment than there is durable, repeatable principles which are universally applicable.
  - SrslyJosh 1y ago
    > "high leverage outcomes" You're not talking to managers here, you can use plain english. > Maybe the phrase "prompt engineering" is bad and it really should be called "prompt crafting" because there is more an element of craft, taste, and judgment than there is durable, repeatable principles which are universally applicable. Yes, the biggest problem with the phrase is that "engineering" implies a well-defined process with predicable results (think of designing a bridge), and prompting doesn't check either of those boxes.
    - yowlingcat 1y ago
      My goals are outcomes on the key metrics of the business I run (such as opex, funnel pull-through, time to resolution) that translate the human effort and inputs into significantly more output than previous approaches we were using. That particular quotient (amplification of outputs over inputs) is the definition of leverage, and it is plain english. I don't think it's fair to hand-wave a word with a very specific meaning away as "manager speak" because you're unfamiliar with it. I have been using LLMs very specifically to drive towards those goals, and prompt engineering (or crafting given you dislike the phrase "engineering") is a crucial tool to get to those outcomes. And yes, sometimes that means writing my own code itself to interact with them, template prompts, post process, utilize them in workflows. And so, the more I think about it, the more that I see patterns in how I create prompts (that probably could be automated with the same tools used for content templating) that make it feel somewhere in the middle of engineering and crafting. My guess is that semantic ambiguity makes a lot of folks uncomfortable. If that's engineering, what isn't engineering, and doesn't it dilute engineering? Yet from the other angle, it's absolutely the case that LLM utilization is absolutely becoming a greater and greater part of how cutting edge companies write code, so much so that the "code" that the humans must put effort and intention into writing is just as much prompts as it is the fine tuned artifact created by running the prompt. If the reality of what actually is "code" is itself evolving, it's hard to imagine anything but that what constitutes software engineering must also itself be evolving in a very fundamental way.
MontagFTB 1y ago
Is there some way to quantify the effects of these prompt modifications versus not including them? How do we know the models are producing better output?
Kiyo-Lynn 1y ago
At first I kept thinking the model just wasn't good enough, it just couldn’t give me what I wanted. But over time, I realized the real problem was that I hadn’t figured out what I wanted in the first place. I had to make my own thinking clear first, then let the AI help organize it. The more specific and patient I am, the better it responds.
heisenburgzero 1y ago
In my own experience, if the problem is not solvable by a LLM. No amount of prompt "engineering" will really help. Only way to solve it would be by partially solving it (breaking down to sub-tasks / examples) and let it run its miles. I'll love to be wrong though. Please share if anyone has a different experience.
- TheCowboy 1y ago
  I think part of the skill in using LLMs is getting a sense for how to effectively break problems down, and also getting a sense of when and when not to do it. The article also mentions this. I think we'll also see ways of restructuring, organizing, and commenting code to improve interaction with LLMs. And also expect LLMs to get better at doing this, and maybe suggesting ways for programmers to break problems down that it is struggling with.
- stets 1y ago
  I think the intent of prompt engineering is to get better solutions quicker, in formats you want. But yeah, ideally the model just "knows" and you don't have to engineer your question
jwr 1y ago
I find the name "prompt engineering" so annoying. There is no engineering in throwing something at the wall and seeing if it sticks. There are no laws or rules that one can learn. It's not science, and it is certainly not engineering.
- dimitri-vs 1y ago
  It's really just technical writing. Majority of tricks from the GTP-4 era are obsolete with reasoning models.
vrnvu 1y ago
We are teaching programmers how to prompt engineer instead of programming… What a world we live in.
- awb 1y ago
  Your native language is now a programming language.
  - 1_08iu 1y ago
    If people feel that they need to learn different language patterns in order to communicate effectively in their native language with an LLM then I'm not sure if I agree. I think that if your native language truly was a programming language then there wouldn't be any need for prompt engineering. Regardless, I think that programmers are quite well-suited to the methods described in the article, but only in the same way that programmers are generally better at Googling things than the average person; they can imagine what the system needs to see in order to produce the result they want even if that isn't necessarily a natural description of their problem.
    - awb 1y ago
      As an engineer, if you’ve had a skilled Project Manager you’ve receive detailed, thoughtful “prompts” on how to complete a project that leads to a shared understanding. But with unskilled Project Managers you might receive a vague “prompt” that leads to ambiguity and misalignment. And Project Managers can go through training or read books on how to effectively “prompt” (aka communicate) more effectively. When I say that your native language is now a programming language, I mean that you are now the Project Manager, and the code is generated automatically. But because the code is auto-generated, the focus shifts towards the “prompt” as the first and potentially only step in generating code.
- wiseowise 1y ago
  What shall we teach them instead? Creating bazillion CRUDs? Serializing/Deserializing JSON?
- 1y ago
max_on_hn 1y ago
Some solid tips here but I think this bit really misses the point: > The key is to view the AI as a partner you can coach – progress over perfection on the first try This is not how to use AI. You cannot scale the ladder of abstraction if you are babysitting a task at one rung. If you feel that it’s not possible yet, that may be a sign that your test environment is immature. If it is possible to write acceptance tests for your project, then trying to manually coach the AI is just a cost optimization, you are simply reducing the tokens it takes the AI to get the answer. Whether that’s worth your time depends on the problem, but in general if you are manually coaching your AI you should stop and either: 1. Work on your pipeline for prompt generation. If you write down any relevant project context in a few docs, an AI will happily generate your prompts for you, including examples and nice formatting etc. Getting better at this will actually improve 2. Set up an end-to-end test command (unit/integration tests are fine too add later but less important than e2e) These processes are how people use headless agents like CheepCode[0] to move faster. Generate prompts with AI and put them in a task management app like Linear, then CheepCode works on the ticket and makes a PR. No more watching a robot work, check the results at the end and only read the thoughts if you need to debug your prompt. [0] the one I built - https://cheepcode.com https://cheepcode.com
mseepgood 1y ago
Is asking good questions or making clear requests what people now refer to as 'prompt engineering'?
bbuchalter 1y ago
I currently take a very causal approach to prompting. Have others started at that point and moved to the rigor discussed here? How much marginal benefit did you see? Does the investment of prompt time provide linear returns? Exponential? Logarithmic?
- SrslyJosh 1y ago
  Don't forget the emotional payoff of feeling like you're not having to do the actual work. I'd bet that some people are spending more time and energy writing prompts that it would take to just solve problems themselves.
jorge_cab 1y ago
I don't think it should be the role of developers to write "good prompts" I think ideally an intermediate layer should optimize the information passed to an LLM. Like what Agentic IDEs are starting to do. I don't copy paste code in the correct way to optimize my prompt, I select the code I want, with MCPs picking up you might not even have to paste input/output the Agent can run it and parse it into the LLM in an optimal way. Of course, the quality of your instructions matter but I think that falls outside of "prompt engineering"
julienchastang 1y ago
I did not read the entire blog post (it is pretty long), but from what I read, I actually disagree with the author. You want to provide the right amount of prompting, but not too much either. Also the LLM can derive a huge amount of information from the context you give it (e.g., you don't have to say it is looking at a k8s log, it will figure it out immediately from the information entered). However, in other situations, you do want to be more specific. Good prompting ends up being as much of an art as a science which you realize as you gain experience with it.
air7 1y ago
A few days ago Sergey Brin said "We don't circulate this too much in the AI community – not just our models but all models – tend to do better if you threaten them … with physical violence" -- https://www.theregister.com/2025/05/28/google_brin_suggests_threatening_ai/ https://www.theregister.com/2025/05/28/google_brin_suggests_...
- xigency 1y ago
  So that's why they dropped "Don't Be Evil."
- layman51 1y ago
  This reminds me of that funny detail in a YouTube video by “Programmers are also human” on professional vibe coders where he keeps ending his orders to the LLM with “.. or you go to jail.”
GuB-42 1y ago
Does the "you are an expert" kind of prompt really work? Role playing may help if you want to direct the focus of the LLM in a particular way (as explained in the article), for example by focusing on security, performance, style, etc... But I am skeptical over the idea that asking a LLM to be an expert will actually improve its expertise. I did a short test prompting ChatGPT do be an "average developer, just smart enough not to get fired", an "expert" and no persona. I got 3 different answers but I couldn't decide which one was the best. The first persona turned out quite funny, with "meh" comments and an explanation about what the code "barely" does, but the code itself is fine. I fear that but asking a LLM to be an expert, it will get the confidence of an expert rather than the skills of an experts, and a manipulative AI is something I'd rather not have.
- bfeynman 1y ago
  This used to work but new thinking models made this unnecessary for the most part.
BuyMyBitcoins 1y ago
I have a mental barrier around giving prompts in natural English. Part of me feels like I should be giving precise information as if I were writing a SQL query or providing arguments to an executable. It is fascinating that you can give instructions as if you were talking to someone, but part of me feels like doing it this way is imprecise. Nevertheless, having these tools process natural language for instructions is probably for the best. It does make these tools dramatically more accessible. That being said I still feel silly writing prompts as if I was talking to a person.
Noelia- 1y ago
[dead]

464 points by vinhnx 1y ago

At this point it's a bit dogfooded, so why not just ask the LLM for a good prompt for the project you're working on?

sherdil2022 1y ago

So it looks like we all need to have firm understanding and tailor our prompts now to effectively use LLMs. Isn't this all subjective? I get different answers based upon how I word my question. Shouldn't things be little bit more objective? Isn't this random that I get different results based upon just wording? This whole thing is just discombobulating to me.

fcoury 1y ago
And to add to it, here's my experience: sometimes you spend a lot of time on this upfront prompt engineering and get bad results and sometimes you just YOLO it and get good results. It's hard to advocate for a determined strategy for prompt engineering when the tool you're prompting itself is non-deterministic. Edit: I also love that the examples come with "AI’s response to the poor prompt (simulated)"
- emseetech 1y ago
  Also that non-determinism means every release will change the way prompting works. There's no guarantee of consistency like an API or a programming language release would have.

akkad33 1y ago

I'd rather write my own code than do all that

echelon 1y ago
Your boss (or CEO) probably wouldn't.
- guappa 1y ago
  My boss cares about speed, because he doesn't sell AI.
coffeefirst 1y ago
Writing the code would also save you time over whatever I just read.
Lich 1y ago
Yeah I don’t get it. By the time I’m done writing all these prompts to get what I want, refining over and over not to mention the waiting time for the characters to appear on the screen, I could have written what I want myself. I find LLMs more useful as a quick documentation search and for the most basic refactoring and template building.

ColinEberhardt 1y ago

There are so many prompting guides at the moment. Personally I think they are quite unnecessary. If you take the time to use these tools, build familiarity with them and the way they work, the prompt you should use becomes quite obvious.

Disposal8433 1y ago
It reminds me that we had the same hype and FOMO when Google became popular. Books were being written on the subject and you had to buy those or you would become a caveman in a near future. What happened is that anyone could learn the whole thing in a day and that was it, no need to debate about whether you would miss anything if you didn't knew all those tools.
- verbify 1y ago
  I certainly have better Google fu than some relatives who are always asking me to find something online.
  - Timwi 1y ago
    I love the term “Google fu”. We should call it prompt fu or LLM fu instead of “prompt engineering”.
- wiseowise 1y ago
  You’re only proving the opposite: there’s definitely a difference between “experienced Google user” and someone who just puts random words and expects to find what they need.
  - marliechiller 1y ago
    Is there? I feel like google has optimised heavily for the caveman input rather than the enlightened search warrior nowadays
sokoloff 1y ago

haolez 1y ago

Sometimes I get the feeling that making super long and intricate prompts reduces the cognitive performance of the model. It might give you a feel of control and proper engineering, but I'm not sure it's a net win. My usage has converged to making very simple and minimalistic prompts and doing minor adjustments after a few iterations.

dwringer 1y ago
I would simplify this as "irrelevant context is worse than no context", but it doesn't mean a long prompt of relevant context is bad.
tgv 1y ago
For another kind of task, a colleague had written a very verbose prompt. Since I had to integrate it, I added some CRUD ops for prompts. For a test, I made a very short one, something like "analyze this as a <profession>". The output was pretty much comparable, except that the output on the longer prompt contained (quite a few) references to literal parts of that prompt. It wasn't incoherent, but it was as if that model (gemini 2.5, btw) has a basic response for the task it extracts from the prompt, and merges the superfluous bits in. It would seem that, at least for this particular task, the model cannot (easily) be made to "think" differently.
wslh 1y ago
Same here: it starts with a relatively precise need, keeping a roadmap in mind rather than forcing one upfront. When it involves a technology I'm unfamiliar with, I also ask questions to understand what certain things mean before "copying and pasting". I've found that with more advanced prompts, the generated code sometimes fails to compile, and tracing the issues backward can be more time consuming than starting clean.
- lodovic 1y ago
  I use specs in markdown for the more advanced prompts. I ask the llm to refine the markdown first and add implementation steps, so i can review what it will do. When it starts implementing, i can always ask it to "just implement step 1, and update the document when done". You can also ask it to verify if the spec has been implemented correctly.

neves 1y ago

Any tip that my fellow programmers find useful that's not in the article?

didibus 1y ago
Including a coding style guide can help the code looks like what you want. Also including an explanation of the project structure, and overall design of the code base. Always specify what libraries it should make use of (or it'll bring in anything or implement stuff a library has already). You can also make the AI review itself. Have it modify code, than ask to review the code, than ask to address review comments, and iterate until it has no more comments. Use an agentic tool like Claude Code or Amazon Q CLI. Then ask it to run tests after code changes and to address all issues until test pass. Make sure to tell it not to change the test code.
- taosx 1y ago
  Unless your employer pays for you to use agentic tools, avoid them. They burn through money and tokens like there's no tomorrow.
trcf22 1y ago
I found that presenting your situation and asking for a plan/ideas + « do not give me code. Make sure you understand the requirements and ask questions if needed.» works much better for me. It also allows me to more easily control what the llm will do and not end up reviewing and throwing 200 lines of code. In a nextjs + vitest context, I try to really outline which tests I want and give it proper data examples so that it does not cheat around mocking fake objects. I do not buy into the whole you’re a senior dev etc. Most people use Claude for coding so I guess it’s engrained by default.

ofrzeta 1y ago

In the "Debugging example", the first prompt doesn't include the code but the second does? No wonder it can find the bug! I guess you may prompt as you like, as long as you provide the actual code, it usually finds bugs like this. About the roles: Can you measure a difference in code quality between the "expert" and the "junior"?

namanyayg 1y ago

It's cool to see Addy's prompts. I also wrote about some of mine! (ignoring the obvious ones): https://nmn.gl/blog/ai-prompt-engineering https://nmn.gl/blog/ai-prompt-engineering

b0a04gl 1y ago

[dead]

leshow 1y ago

using the term "engineering" for writing a prompt feels very unserious

morkalork 1y ago
For real. Editing prompts bares no resemblance to engineering at all, there is no accuracy or precision. Say you have a benchmark to test against and you're trying to make an improvement. Will your change to the prompt make the benchmark go up? Down? Why? Can you predict? No, it is not a science at all. It's just throwing shit and examples at the wall in hopes and prayers.
- echelon 1y ago
  > Will your change to the prompt make the benchmark go up? Down? Why? Can you predict? No, it is not a science at all. Many prompt engineers do measure and quantitatively compare.
  - morkalork 1y ago
    Me too but it's after the fact. I make a change then measure, if it doesn't I roll back. But it's as good as witch craft or alchemy. Will I get I get gold with this adjustment? Nope, still lead. Tries variation #243 next
    - a2dam 1y ago
      This is literally how the light bulb filament was discovered.
      - MegaButts 1y ago
        And Tesla famously described Edison as an idiot for this very reason. Then Tesla revolutionized the way we use electricity while Edison was busy killing elephants.
      - SrslyJosh 1y ago

Avalaxy 1y ago

I feel like all of this is nonsense for people who want to pretend they are very good at using AI. Just copy pasting a stack trace with error message works perfectly fine, thank you.

yoyohello13 1y ago
Seriously, I seem to get good results by just "being a good communicator". Not only that, but as the tools get better, prompt engineering should get less important.
abletonlive 1y ago
In most HN LLM Programming discussion there's a vocal majority saying LLMs are useless. Now we have this commenter saying all they need to do is vibe and it all works out. WHICH IS IT?
- rognjen 1y ago
  I downvoted your comment because of your first sentence. Your point makes is made even without it.

groby_b 1y ago

"State your constraints and requirements well and exhaustively". Meanwhile, I just can't get over the cartoon implying that a React Dev is just a Junior Dev who lost their hoodie.

yuvadam 1y ago

Seems like so much over (prompt) engineering. I get by just fine with pasting raw code or errors and asking plain questions, the models are smart enough to figure it out themselves.

orochimaaru 1y ago

A long time back for my MS CS I took a science of programming course. The way to verify has helped me craft prompts when I do data engineering work. Basically: Given input (…) and preconditions (…) write me spark code that gives me post conditions (…). If you can formally specify the input, preconditions and post conditions you usually get good working code. 1. Science of programming, David Gries 2. Verification of concurrent and sequential systems

nexoft 1y ago

"prompt engineering" ....ouch. I would say "common sense"\ also the problem with software engineering is that there is an inflation of SWE, too much people applying for it for compensation level rather than being good at it and really liking it, we ended up having a lot of bad software engineers that requires this crutch crap.

adamhartenz 1y ago
"Common sense" doesn't exist. It is a term people use when they can't explain what they actually mean.
- ozim 1y ago
  Also how common sense can exist with LLM? There is no common sense with it - it is just an illusion.
  - wiseowise 1y ago
    Presumably it is your common sense.
- marliechiller 1y ago
  Not sure I fully agree - sometimes maybe, but I think in the majority of cases it's used when people feel they dont need to explain exactly what they mean because it should already be obvious to most people. Example. "Always look when you cross the road" is a snippet common sense, with lack of heeding to that sense resulting in you potentially being hit by a car. Even a 4 year old wouldnt need the latter explanation, but most people could articulate that if they needed to. Its just a way of speeding up communication
  - lowbloodsugar 1y ago
    I was quite old when I realized that Common sense is literally “common experiences”. A colleague and I were lamenting a laughably bad outage that we thought showed a total lack of common sense resulting in an obvious issue. Then we both realized that the team had never had such an experience whereas the two of us had. Every member of that team now has that particular “common sense”. Likewise, “don’t run in front of cars”. As a kid, a mate broke his leg running onto the street and getting hit. I think near misses happen a lot when we’re kids. But far fewer has an “common sense” about prompt engineering because there’s just much less experience.

DebtDeflation 1y ago

In my experience there's really only three true prompt engineering techniques: - In Context Learning (providing examples, AKA one shot or few shot vs zero shot) - Chain of Thought (telling it to think step by step) - Structured output (telling it to produce output in a specified format like JSON) Maybe you could add what this article calls Role Prompting to that. And RAG is its own thing where you're basically just having the model summarize the context you provide. But really everything else just boils down to tell it what you want to do in clear plain language.

faustocarva 1y ago
Did you find it hard to create structured output while also trying to make it reason in the same prompt?
- demosthanos 1y ago
  You use a two-phase prompt for this. Have it reason through the answer and respond with a clearly-labeled 'final answer' section that contains the English description of the answer. Then run its response through again in JSON mode with a prompt to package up what the previous model said into structured form. The second phase can be with a cheap model if you need it to be.
  - faustocarva 1y ago
    Great, will try this! But, in a chain-based prompt or full conversational flow?
    - demosthanos 1y ago
      You can do this conversationally, but I've had the most success with API requests, since that gives you the most flexibility. Pseudo-prompt: Prompt 1: Do the thing, describe it in detail, end with a clear summary of your answer that includes ${THINGS_YOU_NEED_FOR_JSON}. Prompt 2: A previous agent said ${CONTENT}, structure as JSON according to ${SCHEMA}. Ideally you use a model in Prompt 2 that supports JSON schemas so you have 100% guarantee that what you get back parses. Otherwise you can implement it yourself by validating it locally and sending the errors back with a prompt to fix them.

bongodongobob 1y ago

None of this shit is necessary. I feel like these prompt collections miss the point entirely. You can talk to an LLM and reason about stuff going back and forth. I mean, some things are nice to one shot, but I've found keeping it simple and just "being yourself" works much better than a page long prompt.

m3kw9 1y ago

The “you are a world class expert..” prompt to me is more like a superstious thing a sports player do before they play. I’ve used it and it still gives similar results, but maybe on a good day(that random seed) it will give me superior results

bsoles 1y ago

There is no such thing as "prompt engineering". Since when the ability to write proper and meaningful sentences became engineering? This is even worse than "software engineering". The unfortunate thing is that there will probably be job postings for such things and people will call themselves prompt engineers for their extraordinary abilities for writing sentences.

yowlingcat 1y ago
I would caution against thinking it's impossible even if it's not something you've personally experienced. Prompt engineering is necessary (but not sufficient) to creating high leverage outcomes from LLMs when solving complex problems. Without it, the chances of getting to a solution are slim. With it, the chances of getting to 90% of a solution and needing to fine tune the last mile are a lot higher but still not guaranteed. Maybe the phrase "prompt engineering" is bad and it really should be called "prompt crafting" because there is more an element of craft, taste, and judgment than there is durable, repeatable principles which are universally applicable.
- SrslyJosh 1y ago
  > "high leverage outcomes" You're not talking to managers here, you can use plain english. > Maybe the phrase "prompt engineering" is bad and it really should be called "prompt crafting" because there is more an element of craft, taste, and judgment than there is durable, repeatable principles which are universally applicable. Yes, the biggest problem with the phrase is that "engineering" implies a well-defined process with predicable results (think of designing a bridge), and prompting doesn't check either of those boxes.
  - yowlingcat 1y ago
    My goals are outcomes on the key metrics of the business I run (such as opex, funnel pull-through, time to resolution) that translate the human effort and inputs into significantly more output than previous approaches we were using. That particular quotient (amplification of outputs over inputs) is the definition of leverage, and it is plain english. I don't think it's fair to hand-wave a word with a very specific meaning away as "manager speak" because you're unfamiliar with it. I have been using LLMs very specifically to drive towards those goals, and prompt engineering (or crafting given you dislike the phrase "engineering") is a crucial tool to get to those outcomes. And yes, sometimes that means writing my own code itself to interact with them, template prompts, post process, utilize them in workflows. And so, the more I think about it, the more that I see patterns in how I create prompts (that probably could be automated with the same tools used for content templating) that make it feel somewhere in the middle of engineering and crafting. My guess is that semantic ambiguity makes a lot of folks uncomfortable. If that's engineering, what isn't engineering, and doesn't it dilute engineering? Yet from the other angle, it's absolutely the case that LLM utilization is absolutely becoming a greater and greater part of how cutting edge companies write code, so much so that the "code" that the humans must put effort and intention into writing is just as much prompts as it is the fine tuned artifact created by running the prompt. If the reality of what actually is "code" is itself evolving, it's hard to imagine anything but that what constitutes software engineering must also itself be evolving in a very fundamental way.

MontagFTB 1y ago

Is there some way to quantify the effects of these prompt modifications versus not including them? How do we know the models are producing better output?

Kiyo-Lynn 1y ago

At first I kept thinking the model just wasn't good enough, it just couldn’t give me what I wanted. But over time, I realized the real problem was that I hadn’t figured out what I wanted in the first place. I had to make my own thinking clear first, then let the AI help organize it. The more specific and patient I am, the better it responds.

heisenburgzero 1y ago

In my own experience, if the problem is not solvable by a LLM. No amount of prompt "engineering" will really help. Only way to solve it would be by partially solving it (breaking down to sub-tasks / examples) and let it run its miles. I'll love to be wrong though. Please share if anyone has a different experience.

TheCowboy 1y ago
I think part of the skill in using LLMs is getting a sense for how to effectively break problems down, and also getting a sense of when and when not to do it. The article also mentions this. I think we'll also see ways of restructuring, organizing, and commenting code to improve interaction with LLMs. And also expect LLMs to get better at doing this, and maybe suggesting ways for programmers to break problems down that it is struggling with.
stets 1y ago
I think the intent of prompt engineering is to get better solutions quicker, in formats you want. But yeah, ideally the model just "knows" and you don't have to engineer your question

jwr 1y ago

I find the name "prompt engineering" so annoying. There is no engineering in throwing something at the wall and seeing if it sticks. There are no laws or rules that one can learn. It's not science, and it is certainly not engineering.

dimitri-vs 1y ago
It's really just technical writing. Majority of tricks from the GTP-4 era are obsolete with reasoning models.

vrnvu 1y ago

We are teaching programmers how to prompt engineer instead of programming… What a world we live in.

awb 1y ago
Your native language is now a programming language.
- 1_08iu 1y ago
  If people feel that they need to learn different language patterns in order to communicate effectively in their native language with an LLM then I'm not sure if I agree. I think that if your native language truly was a programming language then there wouldn't be any need for prompt engineering. Regardless, I think that programmers are quite well-suited to the methods described in the article, but only in the same way that programmers are generally better at Googling things than the average person; they can imagine what the system needs to see in order to produce the result they want even if that isn't necessarily a natural description of their problem.
  - awb 1y ago
    As an engineer, if you’ve had a skilled Project Manager you’ve receive detailed, thoughtful “prompts” on how to complete a project that leads to a shared understanding. But with unskilled Project Managers you might receive a vague “prompt” that leads to ambiguity and misalignment. And Project Managers can go through training or read books on how to effectively “prompt” (aka communicate) more effectively. When I say that your native language is now a programming language, I mean that you are now the Project Manager, and the code is generated automatically. But because the code is auto-generated, the focus shifts towards the “prompt” as the first and potentially only step in generating code.
wiseowise 1y ago
What shall we teach them instead? Creating bazillion CRUDs? Serializing/Deserializing JSON?
1y ago

max_on_hn 1y ago

Some solid tips here but I think this bit really misses the point: > The key is to view the AI as a partner you can coach – progress over perfection on the first try This is not how to use AI. You cannot scale the ladder of abstraction if you are babysitting a task at one rung. If you feel that it’s not possible yet, that may be a sign that your test environment is immature. If it is possible to write acceptance tests for your project, then trying to manually coach the AI is just a cost optimization, you are simply reducing the tokens it takes the AI to get the answer. Whether that’s worth your time depends on the problem, but in general if you are manually coaching your AI you should stop and either: 1. Work on your pipeline for prompt generation. If you write down any relevant project context in a few docs, an AI will happily generate your prompts for you, including examples and nice formatting etc. Getting better at this will actually improve 2. Set up an end-to-end test command (unit/integration tests are fine too add later but less important than e2e) These processes are how people use headless agents like CheepCode[0] to move faster. Generate prompts with AI and put them in a task management app like Linear, then CheepCode works on the ticket and makes a PR. No more watching a robot work, check the results at the end and only read the thoughts if you need to debug your prompt. [0] the one I built - https://cheepcode.com https://cheepcode.com

mseepgood 1y ago

Is asking good questions or making clear requests what people now refer to as 'prompt engineering'?

bbuchalter 1y ago

I currently take a very causal approach to prompting. Have others started at that point and moved to the rigor discussed here? How much marginal benefit did you see? Does the investment of prompt time provide linear returns? Exponential? Logarithmic?

SrslyJosh 1y ago
Don't forget the emotional payoff of feeling like you're not having to do the actual work. I'd bet that some people are spending more time and energy writing prompts that it would take to just solve problems themselves.

jorge_cab 1y ago

I don't think it should be the role of developers to write "good prompts" I think ideally an intermediate layer should optimize the information passed to an LLM. Like what Agentic IDEs are starting to do. I don't copy paste code in the correct way to optimize my prompt, I select the code I want, with MCPs picking up you might not even have to paste input/output the Agent can run it and parse it into the LLM in an optimal way. Of course, the quality of your instructions matter but I think that falls outside of "prompt engineering"

julienchastang 1y ago

I did not read the entire blog post (it is pretty long), but from what I read, I actually disagree with the author. You want to provide the right amount of prompting, but not too much either. Also the LLM can derive a huge amount of information from the context you give it (e.g., you don't have to say it is looking at a k8s log, it will figure it out immediately from the information entered). However, in other situations, you do want to be more specific. Good prompting ends up being as much of an art as a science which you realize as you gain experience with it.

air7 1y ago

A few days ago Sergey Brin said "We don't circulate this too much in the AI community – not just our models but all models – tend to do better if you threaten them … with physical violence" -- https://www.theregister.com/2025/05/28/google_brin_suggests_threatening_ai/ https://www.theregister.com/2025/05/28/google_brin_suggests_...

xigency 1y ago
So that's why they dropped "Don't Be Evil."
layman51 1y ago
This reminds me of that funny detail in a YouTube video by “Programmers are also human” on professional vibe coders where he keeps ending his orders to the LLM with “.. or you go to jail.”

GuB-42 1y ago

Does the "you are an expert" kind of prompt really work? Role playing may help if you want to direct the focus of the LLM in a particular way (as explained in the article), for example by focusing on security, performance, style, etc... But I am skeptical over the idea that asking a LLM to be an expert will actually improve its expertise. I did a short test prompting ChatGPT do be an "average developer, just smart enough not to get fired", an "expert" and no persona. I got 3 different answers but I couldn't decide which one was the best. The first persona turned out quite funny, with "meh" comments and an explanation about what the code "barely" does, but the code itself is fine. I fear that but asking a LLM to be an expert, it will get the confidence of an expert rather than the skills of an experts, and a manipulative AI is something I'd rather not have.

bfeynman 1y ago
This used to work but new thinking models made this unnecessary for the most part.

BuyMyBitcoins 1y ago

I have a mental barrier around giving prompts in natural English. Part of me feels like I should be giving precise information as if I were writing a SQL query or providing arguments to an executable. It is fascinating that you can give instructions as if you were talking to someone, but part of me feels like doing it this way is imprecise. Nevertheless, having these tools process natural language for instructions is probably for the best. It does make these tools dramatically more accessible. That being said I still feel silly writing prompts as if I was talking to a person.

Noelia- 1y ago

[dead]