5 ms·

Prompting LLMs is not engineering

104 points by Bluestein 1y ago

VirgilShelton 1y ago
It's just a buzz word but sadly it will (And already has) effected real engineers :(
- LtWorf 1y ago
  I don't think they were good engineers to begin with anyway.
dlevine 1y ago
My theory is that, when done properly, it’s much closer to science than engineering. And by “done properly,” i mean done in a regimented way with evals to verify that a wide range of inputs produce the desired outputs. Prompting is much closer to discovering the properties of an already existing system than building something using engineering methods.
- ankit219 1y ago
  "Done properly" anything can be science/engineering. Just make a process, give it a KPI on how well it's performing, and you have an engineering. Movie making done properly is where audience is engaged with every act, you test screen it, see audience reaction, tweak, have a specific set of evals, and then you have something scoring high on the metrics. Part of that happens today, but you would not call that science. It's not what people mean when they call something as science or engineering. Your's is too broad of a definition to mean anything specific.
- NotAnOtter 1y ago
  My definition of science is something like "a methodical approach to reach some truth" whereas engineering is something like "a methodical approach to reach some functionality". And I think some version of that is pretty universally accepted. So by that definition, prompt engineering is much closer to engineering than science. That said, I would consider it closer to product development than either of the above two; I don't count 'tell an llm you'll torture it until your website is hopefully less buggy' a methodical approach.
crowcroft 1y ago
This sounds like the same kind of issue ‘real’ engineers have with software ‘engineering’.
labrador 1y ago
Prompting is more art than engineering. But understanding the basics of LLMs, which are engineered systems, helps you get better results. I think of it like being a good interviewer: you’re not controlling the conversation, but you know how to guide it. A good prompter, like a good interviewer, knows how to ask the kind of question that gets a meaningful answer. I think of LLMs like a giant library and my task is to steer it to the right sections for a good answer.
- andy99 1y ago
  No,the prompt is a hyperparameter. If you're not treating it as such and systematically experimenting, it's just superstition.
  - labrador 1y ago
    You're view of it is correct for agents but for my purposes, which are mostly brainstorming, researching and asa thinking tool, I have GPT 4o memory systems turned on. This allows it to be more personable and allows crafting of answers more attuned to my meaning. System and chat history memory obviously introduce global state which is not exactly a tunable hyperparameter for more consistent results. You should probably turn it off for agents so your view is accurate.
  - Bluestein 1y ago
    > hyperparameter That's an interesting way to put it. indeed.-
mystraline 1y ago
Doesn't matter that much. Management thinks they can do our jobs by replacing us with a shell scrip... LLM prompt. We're always moving up and down the tech ladder, solving problems we thought were solved, and, we fix them. But some prompt is supposedly going to replace understanding of your infra?! Hardly. But you'll save money for this quarter. Past that is definitely up for discussion.... With your shareholders.
nimchimpsky 1y ago
[dead]
sincerely 1y ago
I think as people get increasingly aware of the nebulous consistency of hosted LLMs, local models will become more valuable
ThrowOregonAway 1y ago
Technically you're not an engineer unless you're licensed and bonded and belong to a professional engineering society, or you drive a train, or have invented a unique engine. Period, end of discussion. I know, I know. California changed their law so coders could puff themselves up and put on airs and make their resumes look more impressive. But this is a sad anomaly.
- crustycoder 1y ago
  I'm licensed by The Engineering Council in the UK for doing computery stuff, and by the relevant European body. I've been writing software for a living for most of my 40+ year career, and still do. So Chartered Software Engineers do exist. But perhaps not in CA.
  - ThrowOregonAway 1y ago
    We have no such thing in the US. There is no license for software "engineering" here, it's a made up term.
    - gnabgib 1y ago
      There is in Canada, licensed and bonded.
- rapatel0 1y ago
  As someone with a PhD in electrical engineering, I care exactly zero percent about a paid for stamp from an org calling an me engineer or not.
  - ThrowOregonAway 1y ago
    This is why the planes are falling out of the sky now, this attitude of yours. As an eminent master and founder of the entire field of electrical engineering, I am much more qualified than you to have an opinion on this matter as well.
gundmc 1y ago
I have no problem taking issue with using the term "engineering" for this, but this reads like OP does not believe that you can get better results based on the way you structure your prompt and what information you include. That is wild to me and should be obvious to anyone who spends more than a few hours using LLMs. For what it's worth, I think about the "engineering" in "Prompt/Context Engineering" almost more akin to how it's used in "Social Engineering". You are influencing the model to produce a desirable result.
- troupo 1y ago
  > That is wild to me and should be obvious to anyone who spends more than a few hours using LLMs. And here you're assuming I'm (the author) not doing that. Just the fact that the US wakes up and the available compute goes down affects model output significantly more than any magical prompting. I've literally had the same prompt on the same code produce exactly two different results (proper magic and complete broken hallucination). I've had one-line "do it"s one-shot complex problems, and I've had detailed precise instructions completely ignored producing horrendous code (and vice versa). Until you have a way to measure your "should be obviously", it's nothing but wishful thinking.
  - fleischhauf 1y ago
    how do you measure ? genuinely interested, because I'm confronted with the same problem
    - theshrike79 1y ago
      Not OP, but what I do is that I have specific test prompts saved I feed to new models. Something that's only from my brain and not copied from the internet. For example Simon Willison (@simonw) has them draw a SVG of a pelican on a bicycle[0], something that has never ever been photographed ever, nor is it something the LLM can have a reference on, so it has to figure it out. I have similar things, but not as visual. I know what the result should be and what it should look like. Then I compare and contrast. Mistral.ai, for example, is by far the fastest (hardware clearly overprovisioned compared to load), but it also produces utter bullshit and hallucinations. With ChatGPT and Claude I can kinda feel the results slowing down or getting worse when USAians are awake and hogging the resources (They're either throttling or just plain using a shittier model unrer load). Deepseek even has a cheaper API price for off-hours. [0]https://simonwillison.net/2025/Jun/6/six-months-in-llms/ https://simonwillison.net/2025/Jun/6/six-months-in-llms/
arresin 1y ago
You won’t get far without engineering if all you can do is prompt.
floppyd 1y ago
This article is just gatekeeping the word "engineer" for no reason. When I was developing an agent recently I picked a model (thus fixing most of the "uncertainties" mentioned) and started to construct the system prompt, from simpler to more complex, sometimes back to simpler, discovering that to this particular model the name choice means a lot, observing how often I was or wasn't getting desired results, etc etc. I think it would be challenging to find a definition of the word "engineer" that wouldn't include this process, even if it, obviously, wasn't as complex as some other engineering projects.
- exe34 1y ago
  Honestly if we call software "engineering" in the first place, without any kind of certification/liability, I don't see the problem with applying it to everything else.
  - School-Cotton 1y ago
    Indeed. Engineer is a silly word to gatekeep. Its original literal meaning is just someone who builds engines (in the original, broader sense of “engines” which basically means “machines”). The appropriation of the term to mean formalized categories with certifications etc. came hundreds of years later.
    - LtWorf 1y ago
      I doubt that's the origin, since "ingegno" means cleverness in italian…
      - School-Cotton 1y ago
        Well instead of doubting, you can just look it up on wiktionary.
        LtWorf 1y ago
        You could have done it yourself but you chose to post disinformation instead.
baal80spam 1y ago
In my org they told us clearly: either we will learn prompt engineering, or they will replace us with people who already know it.
- deleted 1y ago
  [deleted]
jonplackett 1y ago
I don’t mind people criticising AI. It’s bad in loads of ways. But it also has some good bits. To call it all snake oil is just as much bullshit as to claim it will solve all humanity’s problems.
- NotAnOtter 1y ago
  To claim AI is useless is ignorant. To claim AI is as good as 99% of people claim it to be, is also ignorant. A lot of people have drank the 'exponential growth' kool aid but don't have enough understanding of the underlying tech to realize (1) there could be some fundamental limits to how good LLM based AI can be. Or AI in general for that matter. (2) a big part of why it got so good so fast is because we started pumping trillions of dollars of microchips & electricity into it. The hardware and energy consumption cannot continue exponentially.
lloydjones 1y ago
This article seems like LinkedIn style controversy-bait, or is written from a place of extreme pedantry. Evals can come pretty close to getting “deterministic” output from LLMs, and I’d argue that this is reasonably considered engineering.
deleted 1y ago
[deleted]
nikanj 1y ago
And taking photographs is not painting, but it doesn't mean it's not a form of visual art that requires skill to do well
rapatel0 1y ago
There are plenty of engineering disciplines that need to have statistical approaches to solving problems. Prompt engineering, should be approached by creating benchmark datasets and defining measures of efficacy and reliability. More generally, IMHO engineering is use of physical and informational tools to build a solution. Tools may have unpredictability and reliability concerns. Its the job of an engineer to utilize the power of these tools while overcoming the reliability issues that might be present. Example: Semiconductor manufacturing involves shooting gases at piece of silicon. There are all sorts of random scattering, distributional anomalies, and patterning problems that arise largely at random. I think you would still call it engineering.
simonw 1y ago
> "Prompt engineers" will tell you that some specific ways of prompting some specific models will result in a "better result"... without any criteria for what a "better result" might signify. That's what evals are for. The best developers working on LLM applications are the ones who are addressing the problem described in this quote. Here's a recent thread about that: https://news.ycombinator.com/item?id=44430117 https://news.ycombinator.com/item?id=44430117
socalgal2 1y ago
I just tried Gemini Canvas which I've seen lot of cool results but I failed and I found it infuriating, similar to the worst version of trying to navigate a phone maze to talk to a real person. First, an example I saw online https://www.reddit.com/r/WebVR/comments/1lh9d3h/webvr_game_created_in_gemini_canvas_it_supports/ https://www.reddit.com/r/WebVR/comments/1lh9d3h/webvr_game_c... I've seen others. They seemed impressive. In particular, I thought it might be fun to ask Gemini to make a fireworks display for the 4th of July. What I imagined was Fantavision https://www.youtube.com/watch?v=pD2AnDhyynM https://www.youtube.com/watch?v=pD2AnDhyynM I didn't expect it to get there but I did hope one a scale of 1 to 10 (10 = Fantavision) it would get to 6 ish. I'd say it got to 3 of 10. What I got was some cubes with yellow planes on the side for buildings. This means things are massively inefficient since instead of a simple textured cube for each building, each building is 1000s of polygons, 2 for each window. Lots of Z fighting. I asked for a river with reflections. The reflections were practically invisible. I think it was generating reflection fireworks under the water plane, not actually "reflecting" the ones above. Because they were under the plane they were faded out because of blending which is not how reflections of fireworks work with water. Then, the fireworks themsevles were horrible, made with 1 pixel particles, no glow, no trails, no substance. They were further very un-random in their velocities and looked nothing like real fireworks. I tried for 90 minutes to get it to make something better but mostly failed. I did manage to get it to stop using pixels for particles and use textured quads. It's first version started with "click to launch fireworks". That didn't work at all, blank screen. I told it I didn't want to launch firework, I just wanted them to auto-launch. It made a new one, this time it had a caption "with auto launching fireworks" (and still blank). I told it there was no point in such a caption as it would be obvious from seeing the fireworks that they were auto-launching. It got rid of that caption but still a blank screen. I started over, this time adding "auto-launching, auto camera" type elements to my intial prompt. I tried several times to get it to make an interesting city around both sides of a river (because this was supposed to be an NYC fireworks display). It failed in all kinds of ways. I'd asked it to fly the camera around the fireworks and never have it go behind any buildings. I put the camera in the buildings so you couldn't see any fireworks. I told it "not behind the builings". It moved the camera further way, twice. I finally got it to put the camera only above the river between the two shorelines (if you can call a plane with some cubes on both sides a river with shorelines). The other way it failed was it just got too slow. The way it was inefficiently making buildings mean it started running at 3fps. I started over. At least twice it started with code that had an error. I'd click "Fix Error" and it would fail to fix it. I'd click "Fix Error", repeat 5-6 times. Eventually give up and start over with a slightly different prompt. I tried asking Gemini to write the prompt as someone suggestion. First use gemini to write the prompt. Then paste that into to Gemini Canvas mode. It didn't help nor generate anything noticable different than my hand written prompt. The issue I spent the most time on, it would "launch a firework" (which it referred to as a rocket). The "rocket" (a particle? a sphere? no idea) would go straight up verically (as in y += velocity * deltaTime). As it did it would leave a trail of static exhaust particles just hanging in the air motionless. The firework would then explode and all of the static hanging in the air exhaust particles from the rocket trail would unfreeze fall a little and fade out. My goal was to get those particles to fall and fade as they were emitted, not wait for the rocket to hit it's peak. I tried 15 or so prompts but it utterly failed to understand the problem. It kept tweaking the velocity of the rocket trail particles after the firework explodes. It never got that them hanging static in the air was the issue no matter how many times or ways I explained the issue. Ever time it would just adjust the velocity of the trail particles, after the firefox exploded and make up nonsense about how it "solved the issue now". If I'd gotten that working I'd have asked it to stop launching the rockets straight vertically but give them a parabolic simple physics based arc. I could have done the entire thing myself in 15-30 minutes and got exactly what I wanted. Instead I faught with non-understanding LLM right out of the a Sci-Fi comedy movie. I know this will work some day and I've seen examples of it working. Still ... ugh for now.
ineedasername 1y ago
Prompting "ChatGPT" is very different than prompting LLMs. For the latter, whether you want to use the E word or not, every one of the unknowns the author cites either don't exist or are greatly reduced when you're using LLMs to construct something purpose-built. Anyone working in tech should take a few hours, download LangFlow and choose a small model in ollama and just run one of the template flows and peek at their prompt setups. Maybe the author just chose a poor title, there's so much writing like this that is the equivalent of saying "Drills can't make good holes" when what you mean is the $15 harbor freight special hooked up to a janky power source and unsteady hand.
kriro 1y ago
[flagged]
acedTrex 1y ago
This seems very obvious to literally any one that is half competent at actual engineering.
toofy 1y ago
If a founder hires an engineer who develops a product, the founder is not magically an engineer. He hired the work out. This isn’t complicated but people struggle to understand it for some weird reason. If I have someone photograph my wedding, and then sit at my computer, once the pictures arrive to me, distribute the photographs, I’m not suddenly magically a photographer. I could go on and on. Using an LLM to write a biology paper does not make that person a biologist. Typing out a request to my friend to write me a poem about chickens with red mittens doesn’t magically somehow make me a poet. It wild that so many struggle with this basic ass concept.
msgodel 1y ago
Coding in general isn't engineering. Software engineering is a relatively boring administrative exercise and unless you just want to maximize your salary you should probably do everything you can to avoid a "software engineering" role.
ChrisArchitect 1y ago
Post from February. Related, more recent post (today) from the author: Everything around LLMs is still magical and wishful thinking https://news.ycombinator.com/item?id=44467949 https://news.ycombinator.com/item?id=44467949
tibastral2 1y ago
This guy should read Bruno Latour, it would make him understand that engineering and more generally science is also a belief system, based on some way of validating the knowledge that can be (or not) what occidental people call logical. But it's another ethnocentrism. See symetry proof of azandé for more insights
deleted 1y ago
[deleted]
seanhunter 1y ago
If a farmer describes something as “organic” (meaning produced in a certain way without artificial fertilizers etc), a chemist sneers because to them all produce is organic (consisting of carbon compounds). If a chemist calls something a metal, well, to an astrophysicist everything heavier than helium is a metal. So someone describes themselves as a “prompt engineer” and a software engineer sneers. Well, many mechanical/chemical/electrical engineers would take issue at the use of the word “engineer” from someone who can’t convert a vector between rectangular and polar coordinates, find and solve the differential equations governing the motion of a harmonic oscillator etc. Language is dynamic and used in context and no-one really gets to hold back the tide if society decides to use words in a particular way.
- OldfieldFund 1y ago
  I'm getting tired of these posts. 3 on the front page in the last 24 hours? 1. Is a significant number of people claiming that vibe coding is engineering? 2. Are people vibe coding in serious companies/on serious projects? 3. Are true engineers getting replaced by vibe coders? 4. Are we holding a prototype for a social app for dog walkers to the same standard as the persistence layer for a bank? 5. If a vibe coder's PR gets merged into a critical system, isn't that a catastrophic failure of code review, automated testing, and senior oversight? This whole vibe coding panic feels like experienced engineers complaining that junior engineers are, well, junior. The entire point of a career is to move from 'what works' (vibes) to 'why it works' (first principles).
brianjking 1y ago
The same people who are arguing that words have meaning are the same ones arguing both for and against prompting/context management and construction being "engineering".

104 points by Bluestein 1y ago

VirgilShelton 1y ago

It's just a buzz word but sadly it will (And already has) effected real engineers :(

LtWorf 1y ago
I don't think they were good engineers to begin with anyway.

dlevine 1y ago

My theory is that, when done properly, it’s much closer to science than engineering. And by “done properly,” i mean done in a regimented way with evals to verify that a wide range of inputs produce the desired outputs. Prompting is much closer to discovering the properties of an already existing system than building something using engineering methods.

ankit219 1y ago
"Done properly" anything can be science/engineering. Just make a process, give it a KPI on how well it's performing, and you have an engineering. Movie making done properly is where audience is engaged with every act, you test screen it, see audience reaction, tweak, have a specific set of evals, and then you have something scoring high on the metrics. Part of that happens today, but you would not call that science. It's not what people mean when they call something as science or engineering. Your's is too broad of a definition to mean anything specific.
NotAnOtter 1y ago
My definition of science is something like "a methodical approach to reach some truth" whereas engineering is something like "a methodical approach to reach some functionality". And I think some version of that is pretty universally accepted. So by that definition, prompt engineering is much closer to engineering than science. That said, I would consider it closer to product development than either of the above two; I don't count 'tell an llm you'll torture it until your website is hopefully less buggy' a methodical approach.

crowcroft 1y ago

This sounds like the same kind of issue ‘real’ engineers have with software ‘engineering’.

labrador 1y ago

Prompting is more art than engineering. But understanding the basics of LLMs, which are engineered systems, helps you get better results. I think of it like being a good interviewer: you’re not controlling the conversation, but you know how to guide it. A good prompter, like a good interviewer, knows how to ask the kind of question that gets a meaningful answer. I think of LLMs like a giant library and my task is to steer it to the right sections for a good answer.

andy99 1y ago
No,the prompt is a hyperparameter. If you're not treating it as such and systematically experimenting, it's just superstition.
- labrador 1y ago
  You're view of it is correct for agents but for my purposes, which are mostly brainstorming, researching and asa thinking tool, I have GPT 4o memory systems turned on. This allows it to be more personable and allows crafting of answers more attuned to my meaning. System and chat history memory obviously introduce global state which is not exactly a tunable hyperparameter for more consistent results. You should probably turn it off for agents so your view is accurate.
- Bluestein 1y ago
  > hyperparameter That's an interesting way to put it. indeed.-

mystraline 1y ago

Doesn't matter that much. Management thinks they can do our jobs by replacing us with a shell scrip... LLM prompt. We're always moving up and down the tech ladder, solving problems we thought were solved, and, we fix them. But some prompt is supposedly going to replace understanding of your infra?! Hardly. But you'll save money for this quarter. Past that is definitely up for discussion.... With your shareholders.

nimchimpsky 1y ago

[dead]

sincerely 1y ago

I think as people get increasingly aware of the nebulous consistency of hosted LLMs, local models will become more valuable

ThrowOregonAway 1y ago

Technically you're not an engineer unless you're licensed and bonded and belong to a professional engineering society, or you drive a train, or have invented a unique engine. Period, end of discussion. I know, I know. California changed their law so coders could puff themselves up and put on airs and make their resumes look more impressive. But this is a sad anomaly.

crustycoder 1y ago
I'm licensed by The Engineering Council in the UK for doing computery stuff, and by the relevant European body. I've been writing software for a living for most of my 40+ year career, and still do. So Chartered Software Engineers do exist. But perhaps not in CA.
- ThrowOregonAway 1y ago
  We have no such thing in the US. There is no license for software "engineering" here, it's a made up term.
  - gnabgib 1y ago
    There is in Canada, licensed and bonded.
rapatel0 1y ago
As someone with a PhD in electrical engineering, I care exactly zero percent about a paid for stamp from an org calling an me engineer or not.
- ThrowOregonAway 1y ago
  This is why the planes are falling out of the sky now, this attitude of yours. As an eminent master and founder of the entire field of electrical engineering, I am much more qualified than you to have an opinion on this matter as well.

gundmc 1y ago

I have no problem taking issue with using the term "engineering" for this, but this reads like OP does not believe that you can get better results based on the way you structure your prompt and what information you include. That is wild to me and should be obvious to anyone who spends more than a few hours using LLMs. For what it's worth, I think about the "engineering" in "Prompt/Context Engineering" almost more akin to how it's used in "Social Engineering". You are influencing the model to produce a desirable result.

troupo 1y ago
> That is wild to me and should be obvious to anyone who spends more than a few hours using LLMs. And here you're assuming I'm (the author) not doing that. Just the fact that the US wakes up and the available compute goes down affects model output significantly more than any magical prompting. I've literally had the same prompt on the same code produce exactly two different results (proper magic and complete broken hallucination). I've had one-line "do it"s one-shot complex problems, and I've had detailed precise instructions completely ignored producing horrendous code (and vice versa). Until you have a way to measure your "should be obviously", it's nothing but wishful thinking.
- fleischhauf 1y ago
  how do you measure ? genuinely interested, because I'm confronted with the same problem
  - theshrike79 1y ago
    Not OP, but what I do is that I have specific test prompts saved I feed to new models. Something that's only from my brain and not copied from the internet. For example Simon Willison (@simonw) has them draw a SVG of a pelican on a bicycle[0], something that has never ever been photographed ever, nor is it something the LLM can have a reference on, so it has to figure it out. I have similar things, but not as visual. I know what the result should be and what it should look like. Then I compare and contrast. Mistral.ai, for example, is by far the fastest (hardware clearly overprovisioned compared to load), but it also produces utter bullshit and hallucinations. With ChatGPT and Claude I can kinda feel the results slowing down or getting worse when USAians are awake and hogging the resources (They're either throttling or just plain using a shittier model unrer load). Deepseek even has a cheaper API price for off-hours. [0]https://simonwillison.net/2025/Jun/6/six-months-in-llms/ https://simonwillison.net/2025/Jun/6/six-months-in-llms/

arresin 1y ago

You won’t get far without engineering if all you can do is prompt.

floppyd 1y ago

This article is just gatekeeping the word "engineer" for no reason. When I was developing an agent recently I picked a model (thus fixing most of the "uncertainties" mentioned) and started to construct the system prompt, from simpler to more complex, sometimes back to simpler, discovering that to this particular model the name choice means a lot, observing how often I was or wasn't getting desired results, etc etc. I think it would be challenging to find a definition of the word "engineer" that wouldn't include this process, even if it, obviously, wasn't as complex as some other engineering projects.

exe34 1y ago
Honestly if we call software "engineering" in the first place, without any kind of certification/liability, I don't see the problem with applying it to everything else.
- School-Cotton 1y ago
  Indeed. Engineer is a silly word to gatekeep. Its original literal meaning is just someone who builds engines (in the original, broader sense of “engines” which basically means “machines”). The appropriation of the term to mean formalized categories with certifications etc. came hundreds of years later.
  - LtWorf 1y ago
    I doubt that's the origin, since "ingegno" means cleverness in italian…
    - School-Cotton 1y ago
      Well instead of doubting, you can just look it up on wiktionary.
      - LtWorf 1y ago
        You could have done it yourself but you chose to post disinformation instead.

baal80spam 1y ago

In my org they told us clearly: either we will learn prompt engineering, or they will replace us with people who already know it.

deleted 1y ago
[deleted]

jonplackett 1y ago

I don’t mind people criticising AI. It’s bad in loads of ways. But it also has some good bits. To call it all snake oil is just as much bullshit as to claim it will solve all humanity’s problems.

NotAnOtter 1y ago
To claim AI is useless is ignorant. To claim AI is as good as 99% of people claim it to be, is also ignorant. A lot of people have drank the 'exponential growth' kool aid but don't have enough understanding of the underlying tech to realize (1) there could be some fundamental limits to how good LLM based AI can be. Or AI in general for that matter. (2) a big part of why it got so good so fast is because we started pumping trillions of dollars of microchips & electricity into it. The hardware and energy consumption cannot continue exponentially.

lloydjones 1y ago

This article seems like LinkedIn style controversy-bait, or is written from a place of extreme pedantry. Evals can come pretty close to getting “deterministic” output from LLMs, and I’d argue that this is reasonably considered engineering.

deleted 1y ago

[deleted]

nikanj 1y ago

And taking photographs is not painting, but it doesn't mean it's not a form of visual art that requires skill to do well

rapatel0 1y ago

There are plenty of engineering disciplines that need to have statistical approaches to solving problems. Prompt engineering, should be approached by creating benchmark datasets and defining measures of efficacy and reliability. More generally, IMHO engineering is use of physical and informational tools to build a solution. Tools may have unpredictability and reliability concerns. Its the job of an engineer to utilize the power of these tools while overcoming the reliability issues that might be present. Example: Semiconductor manufacturing involves shooting gases at piece of silicon. There are all sorts of random scattering, distributional anomalies, and patterning problems that arise largely at random. I think you would still call it engineering.

simonw 1y ago

> "Prompt engineers" will tell you that some specific ways of prompting some specific models will result in a "better result"... without any criteria for what a "better result" might signify. That's what evals are for. The best developers working on LLM applications are the ones who are addressing the problem described in this quote. Here's a recent thread about that: https://news.ycombinator.com/item?id=44430117 https://news.ycombinator.com/item?id=44430117

socalgal2 1y ago

I just tried Gemini Canvas which I've seen lot of cool results but I failed and I found it infuriating, similar to the worst version of trying to navigate a phone maze to talk to a real person. First, an example I saw online https://www.reddit.com/r/WebVR/comments/1lh9d3h/webvr_game_created_in_gemini_canvas_it_supports/ https://www.reddit.com/r/WebVR/comments/1lh9d3h/webvr_game_c... I've seen others. They seemed impressive. In particular, I thought it might be fun to ask Gemini to make a fireworks display for the 4th of July. What I imagined was Fantavision https://www.youtube.com/watch?v=pD2AnDhyynM https://www.youtube.com/watch?v=pD2AnDhyynM I didn't expect it to get there but I did hope one a scale of 1 to 10 (10 = Fantavision) it would get to 6 ish. I'd say it got to 3 of 10. What I got was some cubes with yellow planes on the side for buildings. This means things are massively inefficient since instead of a simple textured cube for each building, each building is 1000s of polygons, 2 for each window. Lots of Z fighting. I asked for a river with reflections. The reflections were practically invisible. I think it was generating reflection fireworks under the water plane, not actually "reflecting" the ones above. Because they were under the plane they were faded out because of blending which is not how reflections of fireworks work with water. Then, the fireworks themsevles were horrible, made with 1 pixel particles, no glow, no trails, no substance. They were further very un-random in their velocities and looked nothing like real fireworks. I tried for 90 minutes to get it to make something better but mostly failed. I did manage to get it to stop using pixels for particles and use textured quads. It's first version started with "click to launch fireworks". That didn't work at all, blank screen. I told it I didn't want to launch firework, I just wanted them to auto-launch. It made a new one, this time it had a caption "with auto launching fireworks" (and still blank). I told it there was no point in such a caption as it would be obvious from seeing the fireworks that they were auto-launching. It got rid of that caption but still a blank screen. I started over, this time adding "auto-launching, auto camera" type elements to my intial prompt. I tried several times to get it to make an interesting city around both sides of a river (because this was supposed to be an NYC fireworks display). It failed in all kinds of ways. I'd asked it to fly the camera around the fireworks and never have it go behind any buildings. I put the camera in the buildings so you couldn't see any fireworks. I told it "not behind the builings". It moved the camera further way, twice. I finally got it to put the camera only above the river between the two shorelines (if you can call a plane with some cubes on both sides a river with shorelines). The other way it failed was it just got too slow. The way it was inefficiently making buildings mean it started running at 3fps. I started over. At least twice it started with code that had an error. I'd click "Fix Error" and it would fail to fix it. I'd click "Fix Error", repeat 5-6 times. Eventually give up and start over with a slightly different prompt. I tried asking Gemini to write the prompt as someone suggestion. First use gemini to write the prompt. Then paste that into to Gemini Canvas mode. It didn't help nor generate anything noticable different than my hand written prompt. The issue I spent the most time on, it would "launch a firework" (which it referred to as a rocket). The "rocket" (a particle? a sphere? no idea) would go straight up verically (as in y += velocity * deltaTime). As it did it would leave a trail of static exhaust particles just hanging in the air motionless. The firework would then explode and all of the static hanging in the air exhaust particles from the rocket trail would unfreeze fall a little and fade out. My goal was to get those particles to fall and fade as they were emitted, not wait for the rocket to hit it's peak. I tried 15 or so prompts but it utterly failed to understand the problem. It kept tweaking the velocity of the rocket trail particles after the firework explodes. It never got that them hanging static in the air was the issue no matter how many times or ways I explained the issue. Ever time it would just adjust the velocity of the trail particles, after the firefox exploded and make up nonsense about how it "solved the issue now". If I'd gotten that working I'd have asked it to stop launching the rockets straight vertically but give them a parabolic simple physics based arc. I could have done the entire thing myself in 15-30 minutes and got exactly what I wanted. Instead I faught with non-understanding LLM right out of the a Sci-Fi comedy movie. I know this will work some day and I've seen examples of it working. Still ... ugh for now.

ineedasername 1y ago

Prompting "ChatGPT" is very different than prompting LLMs. For the latter, whether you want to use the E word or not, every one of the unknowns the author cites either don't exist or are greatly reduced when you're using LLMs to construct something purpose-built. Anyone working in tech should take a few hours, download LangFlow and choose a small model in ollama and just run one of the template flows and peek at their prompt setups. Maybe the author just chose a poor title, there's so much writing like this that is the equivalent of saying "Drills can't make good holes" when what you mean is the $15 harbor freight special hooked up to a janky power source and unsteady hand.

kriro 1y ago

[flagged]

acedTrex 1y ago

This seems very obvious to literally any one that is half competent at actual engineering.

toofy 1y ago

If a founder hires an engineer who develops a product, the founder is not magically an engineer. He hired the work out. This isn’t complicated but people struggle to understand it for some weird reason. If I have someone photograph my wedding, and then sit at my computer, once the pictures arrive to me, distribute the photographs, I’m not suddenly magically a photographer. I could go on and on. Using an LLM to write a biology paper does not make that person a biologist. Typing out a request to my friend to write me a poem about chickens with red mittens doesn’t magically somehow make me a poet. It wild that so many struggle with this basic ass concept.

msgodel 1y ago

Coding in general isn't engineering. Software engineering is a relatively boring administrative exercise and unless you just want to maximize your salary you should probably do everything you can to avoid a "software engineering" role.

ChrisArchitect 1y ago

Post from February. Related, more recent post (today) from the author: Everything around LLMs is still magical and wishful thinking https://news.ycombinator.com/item?id=44467949 https://news.ycombinator.com/item?id=44467949

tibastral2 1y ago

This guy should read Bruno Latour, it would make him understand that engineering and more generally science is also a belief system, based on some way of validating the knowledge that can be (or not) what occidental people call logical. But it's another ethnocentrism. See symetry proof of azandé for more insights

deleted 1y ago

[deleted]

seanhunter 1y ago

If a farmer describes something as “organic” (meaning produced in a certain way without artificial fertilizers etc), a chemist sneers because to them all produce is organic (consisting of carbon compounds). If a chemist calls something a metal, well, to an astrophysicist everything heavier than helium is a metal. So someone describes themselves as a “prompt engineer” and a software engineer sneers. Well, many mechanical/chemical/electrical engineers would take issue at the use of the word “engineer” from someone who can’t convert a vector between rectangular and polar coordinates, find and solve the differential equations governing the motion of a harmonic oscillator etc. Language is dynamic and used in context and no-one really gets to hold back the tide if society decides to use words in a particular way.

OldfieldFund 1y ago
I'm getting tired of these posts. 3 on the front page in the last 24 hours? 1. Is a significant number of people claiming that vibe coding is engineering? 2. Are people vibe coding in serious companies/on serious projects? 3. Are true engineers getting replaced by vibe coders? 4. Are we holding a prototype for a social app for dog walkers to the same standard as the persistence layer for a bank? 5. If a vibe coder's PR gets merged into a critical system, isn't that a catastrophic failure of code review, automated testing, and senior oversight? This whole vibe coding panic feels like experienced engineers complaining that junior engineers are, well, junior. The entire point of a career is to move from 'what works' (vibes) to 'why it works' (first principles).

brianjking 1y ago

The same people who are arguing that words have meaning are the same ones arguing both for and against prompting/context management and construction being "engineering".