5 ms·

i feel like people that get really into prompt engineering seem to forget that other people are working on building models that are even better. Do they really

i feel like people that get really into prompt engineering seem to forget that other people are working on building models that are even better. Do they really think that the tricks they developed to make gpt3.5 work well will still be necessary on gpt9 (or whatever new model replaces gpt)? if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff.

JimtheCoder 3y ago
"if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff." I think this is more for people, though, that want to maximize the use of AI in their own field, isn't it? The "knowledge worker enhancer"? I am not sure they really need to work on the actual AI stuff...
- soderfoo 3y ago
  It felt like a mix to me. People seeking to enhance their work and the low effort high reward seeking hustle bros. The latter latches on to the surface level of any new tech but proceeds with swagger befitting a Nobel laureate.
- kaycey2022 3y ago
  You know I'm something of a "Google User" myself
pessimizer 3y ago
I don't agree. There's a huge need for people to figure out how to integrate AI into larger things, and to establish ways we'll work with it - and at every step, including GPT 3.5 prompt engineering. There's a lot you can come up with if you just assume that good generative AI exists, and will probably get better. Different iterations of AI might as well be pluggable modules.
golergka 3y ago
> if you want a long-lasting career in internet, you need to work on the actual internet stuff, not just using the internet - a C developer that writes modem drivers at a telecom company in the 90s about html/perl developers building first interactive websites
mathgeek 3y ago
I tend to think that as long as these models remain tools (rather than AI that is self aware, etc.), we will have careers available in both “mechanics and engineers” so to speak. Some will design, others will build, still others will integrate and maintain.
manojlds 3y ago
If you want a long lasting career in programming, work on building the languages, not just using the languages. Is probably what someone said many decades ago.
- Terr_ 3y ago
  And in those decades they were probably right.
- xmprt 3y ago
  Is that not true to some extent? In the early days of computer science, when there were so many new developments, it would serve you well to be the person working on writing the programs and building computer hardware, rather than the person using computers to get stuff done. The person using computers was still miles ahead of the person not using computers but where you really wanted to be was building languages and inventing new tools.
  - sshine 3y ago
    > where you really wanted to be was building languages and inventing new tools. Building computers, operating systems and compilers was always a niche. So if you're a computer scientist or an electrical engineer, sure. But if you're not, there was, is and will be "power users" who make the most of it. - Teach, fix shit and bridge the gap for mortals - Enjoy technology with a deep but not foundational understanding.
- parpfish 3y ago
  learning a programming language confers useful skills that you can generalize to new versions of that language or different languages. we can make these generalization becasue we know how the langauges work and people can draw connections from one to the other (i think people here would all agree that learning your second language is far easier than leraning your first). this is different from LLMs because they don't have explicitly constructed abilities that we can compare across models. so we need to step back and approach every model as brand new and figure out what they're capable of. just because you have a prompt that works amazingly in one model, there's no guarantee that that prompt will continue to work in bigger "better" models. so rather than working to devise way to trick a particular model into doing your task, it would probably be a better use of your time to learn how to train/modify models to explicitly solve the problem you care about.
8note 3y ago
I'd lean yes -- what data set can they feed into the training algorithm to replace the stuff that doesn't get into writing?
jw1224 3y ago
The culture war surrounding prompt engineering is so dumb. My eyes roll hard into the back of my head every time I read a comment like this. Prompt engineering is not the same as programming. In some regards, it is “better”. In other regards, it is “worse”. They’re two different yet similar disciplines, each with their own strengths and weaknesses, but both equally legitimate. Programming is simply the skill of being able to communicate with a compiler/interpreter effectively. Prompt engineering is simply the skill of being able to communicate with an LLM effectively. I wish we weren’t so divided over an incredible new technology.
- minimaxir 3y ago
  It’s more of a No True Scotsman thing about “prompt engineering is just inputting text, it’s not actual engineering” https://www.reddit.com/r/datascience/comments/14nbwfv/where_is_the_engineering_part_in_prompt_engineer/ https://www.reddit.com/r/datascience/comments/14nbwfv/where_...
  - yousif_123123 3y ago
    Anyone using a computer is "just" clicking on specific pixels and triggering keyboard events.. it still could be "actual engineering" imo if one is building a system following engineering methods, solving problems etc.
- Detrytus 3y ago
  Working on new LLM model is like doing genetic engineering trying to come up with a new animal species. Prompt engineering is like feeding said animal and shoveling its manure.
  - roughly 3y ago
    Which one of those folks do I pay for the hamburgers I’m having tonight?
    - Detrytus 3y ago
beachy 3y ago
> if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff. Back in the day, when those new fangled relationship database things came on the scene, do you think people would have been well advised to try and find work on the actual database engine itself, instead of the more frivolous work of using the new technology to, say, solve actual business problems?
- dopidopHN 3y ago
  I see the analogy but early RDBMS had SQL. And concepts like primary keys, normalization. I don’t think LLM offer standardized API like that ?
- noogle 3y ago
  "prompt engineering" is a self-destructing field. If you use any rigorous approach to optimizing the prompt, you end up with essentially supervised machine -learning: models can (and do) learn the optimal prompt once there is a yardstick for the goodness of the model's response. That's a classical for a data-scientist, but the skill set has little to do with prompts. If you are not rigorous, then what you are doing is essentially "black art". It may work for some tasks ad-hoc, but with the rapid pace of model improvement your skill will likely become irrelevant/not needed quickly.
  - vidarh 3y ago
    Focusing on specific incantations, yes. Focusing on how to decomposing a problem, probably not, but then you get very close to designing systems / data design and analysis methodologies more than "prompt engineering", so I guess I mostly agree with you in as much as the relevant field is not really about AI as it is about picking up more structured design and analysis practices (again).
  - unaindz 3y ago
    I don't have much experience with GPT but with image generation. You need some amount of experimentation to get the best results but in my experience what works for one model does nothing or worsens the output in others. Adding loras and different types of images into the equation makes this so variable that I would never consider it useful besides keeping a few key words I used to get x good result on y model and experimenting with those when I start a new project. Calling it "prompt engineering" seems odd.
arsome 3y ago
I'm not so sure - while I agree, prompt engineering will go away, there's probably a huge market for pre-built integrations in every sector. Bind this AI platform to this industry specific tooling in a polished way, etc.
BoorishBears 3y ago
I feel like people who write stuff like this don't understand the difference between software and software that delivers value. I mean, you think those graphics optimizations we pour millions of dollars into before releasing AAA games will matter when the GTX 8020 outperforms a 4090? Hint: Delivering value for actual people is rarely the result of sitting on your hands and waiting for the next big platform, or even rolling up your sleeves and trying to learn how to build the next big thing. You have this idea that by investing energy in something that will be obsoleted you're losing out, but spoiler, that's how 99% of software that delivers actual value works. The cutting edge of tech almost always ends up being PaaS/SaaS serving itself: Your mail gets to you because someone is working on software with limitations we solved decades ago. Your paycheck ends up in your bank account because people invest a ton of time in codebases subject to problems we solved long ago. Your anti-lock brakes aren't built on a Rust codebase, but some horrible memory unsafe mess running on a processor that's a decade out of date. — The reality is: 99% of the effort that goes into trying to build the next big thing goes nowhere. The expected value of you trying to learn "the actual AI stuff" to the greater world is near 0 compared to you "just prompt engineering" and putting out something that solves a pain-point nicely with GPT 3.5. At the end of the day most of the value that gets delivered to actual users comes from engineers who went deeper into extracting value from the current thing. We still need people to work on the next big thing so that the 1% of effort that isn't wasted can actually materialize... but in my experience the most successful engineers in that regard are still able to realize the delusion it requires, without being paralyzed by the cognitive dissonance that realization invites.
- parpfish 3y ago
  prompt engineering can definitely deliver value for a customer, but i don't think it will deliver value to a developer in terms of building a sustainable career. a different analogy that gets at my original concern: becoming an expert prompt engineer for an particular LLM is like becoming a power user for a piece of proprietary software that isn't getting any more updates.
  - BoorishBears 3y ago
    The first point doesn't follow the second: there will always be a sustainable career in building software that delivers value. In tech we take it for granted that just because there's some new hotness everyone wants to jump on it day 1. GPT 5 could drop tomorrow and if your tool delivers value using 3.5, it's not going to magically stop delivering value, and in most verticals people will prefer your battle tested 3.5 to some brand new 5. And if 5 does simplify prompts for your use case and there'll still only be two options: - making a the same thing as what you made with 3.5 is now trivial... in which case you still have the mindshare and the distribution solved to a degree your newly enabled clones don't. - making a better version of what you made is now trivial... in which case you can just as trivially improve your version and already have the mindshare and distribution solved. At the end of the day software developers often struggle to fit software into the larger ecosystem it slots into before it becomes something valuable, and to be GPT has been an amazing case study in the fact. I think the use of the word engineering did half the damage, and I think people confusing the twitter memes with the interesting attempts at prompt engineering (via ReACT, Gorilla, etc) did the other half, but at the end of the day I think a lot of people will be left kicking themselves when "misguided prompt engineers" end up solving real useful problems in ways they didn't think were possible well before we reach the arbitrary goalposts for foundation models that people keep setting up.

JimtheCoder 3y ago

"if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff." I think this is more for people, though, that want to maximize the use of AI in their own field, isn't it? The "knowledge worker enhancer"? I am not sure they really need to work on the actual AI stuff...

soderfoo 3y ago
It felt like a mix to me. People seeking to enhance their work and the low effort high reward seeking hustle bros. The latter latches on to the surface level of any new tech but proceeds with swagger befitting a Nobel laureate.
kaycey2022 3y ago
You know I'm something of a "Google User" myself

pessimizer 3y ago

I don't agree. There's a huge need for people to figure out how to integrate AI into larger things, and to establish ways we'll work with it - and at every step, including GPT 3.5 prompt engineering. There's a lot you can come up with if you just assume that good generative AI exists, and will probably get better. Different iterations of AI might as well be pluggable modules.

golergka 3y ago

> if you want a long-lasting career in internet, you need to work on the actual internet stuff, not just using the internet - a C developer that writes modem drivers at a telecom company in the 90s about html/perl developers building first interactive websites

mathgeek 3y ago

I tend to think that as long as these models remain tools (rather than AI that is self aware, etc.), we will have careers available in both “mechanics and engineers” so to speak. Some will design, others will build, still others will integrate and maintain.

manojlds 3y ago

If you want a long lasting career in programming, work on building the languages, not just using the languages. Is probably what someone said many decades ago.

Terr_ 3y ago
And in those decades they were probably right.
xmprt 3y ago
Is that not true to some extent? In the early days of computer science, when there were so many new developments, it would serve you well to be the person working on writing the programs and building computer hardware, rather than the person using computers to get stuff done. The person using computers was still miles ahead of the person not using computers but where you really wanted to be was building languages and inventing new tools.
- sshine 3y ago
  > where you really wanted to be was building languages and inventing new tools. Building computers, operating systems and compilers was always a niche. So if you're a computer scientist or an electrical engineer, sure. But if you're not, there was, is and will be "power users" who make the most of it. - Teach, fix shit and bridge the gap for mortals - Enjoy technology with a deep but not foundational understanding.
parpfish 3y ago
learning a programming language confers useful skills that you can generalize to new versions of that language or different languages. we can make these generalization becasue we know how the langauges work and people can draw connections from one to the other (i think people here would all agree that learning your second language is far easier than leraning your first). this is different from LLMs because they don't have explicitly constructed abilities that we can compare across models. so we need to step back and approach every model as brand new and figure out what they're capable of. just because you have a prompt that works amazingly in one model, there's no guarantee that that prompt will continue to work in bigger "better" models. so rather than working to devise way to trick a particular model into doing your task, it would probably be a better use of your time to learn how to train/modify models to explicitly solve the problem you care about.

8note 3y ago

I'd lean yes -- what data set can they feed into the training algorithm to replace the stuff that doesn't get into writing?

jw1224 3y ago

The culture war surrounding prompt engineering is so dumb. My eyes roll hard into the back of my head every time I read a comment like this. Prompt engineering is not the same as programming. In some regards, it is “better”. In other regards, it is “worse”. They’re two different yet similar disciplines, each with their own strengths and weaknesses, but both equally legitimate. Programming is simply the skill of being able to communicate with a compiler/interpreter effectively. Prompt engineering is simply the skill of being able to communicate with an LLM effectively. I wish we weren’t so divided over an incredible new technology.

minimaxir 3y ago
It’s more of a No True Scotsman thing about “prompt engineering is just inputting text, it’s not actual engineering” https://www.reddit.com/r/datascience/comments/14nbwfv/where_is_the_engineering_part_in_prompt_engineer/ https://www.reddit.com/r/datascience/comments/14nbwfv/where_...
- yousif_123123 3y ago
  Anyone using a computer is "just" clicking on specific pixels and triggering keyboard events.. it still could be "actual engineering" imo if one is building a system following engineering methods, solving problems etc.
Detrytus 3y ago
Working on new LLM model is like doing genetic engineering trying to come up with a new animal species. Prompt engineering is like feeding said animal and shoveling its manure.
- roughly 3y ago
  Which one of those folks do I pay for the hamburgers I’m having tonight?
  - Detrytus 3y ago

beachy 3y ago

> if you want a long-lasting career in AI you need to work on the actual AI stuff, not just using the AI stuff. Back in the day, when those new fangled relationship database things came on the scene, do you think people would have been well advised to try and find work on the actual database engine itself, instead of the more frivolous work of using the new technology to, say, solve actual business problems?

dopidopHN 3y ago
I see the analogy but early RDBMS had SQL. And concepts like primary keys, normalization. I don’t think LLM offer standardized API like that ?
noogle 3y ago
"prompt engineering" is a self-destructing field. If you use any rigorous approach to optimizing the prompt, you end up with essentially supervised machine -learning: models can (and do) learn the optimal prompt once there is a yardstick for the goodness of the model's response. That's a classical for a data-scientist, but the skill set has little to do with prompts. If you are not rigorous, then what you are doing is essentially "black art". It may work for some tasks ad-hoc, but with the rapid pace of model improvement your skill will likely become irrelevant/not needed quickly.
- vidarh 3y ago
  Focusing on specific incantations, yes. Focusing on how to decomposing a problem, probably not, but then you get very close to designing systems / data design and analysis methodologies more than "prompt engineering", so I guess I mostly agree with you in as much as the relevant field is not really about AI as it is about picking up more structured design and analysis practices (again).
- unaindz 3y ago
  I don't have much experience with GPT but with image generation. You need some amount of experimentation to get the best results but in my experience what works for one model does nothing or worsens the output in others. Adding loras and different types of images into the equation makes this so variable that I would never consider it useful besides keeping a few key words I used to get x good result on y model and experimenting with those when I start a new project. Calling it "prompt engineering" seems odd.

arsome 3y ago

I'm not so sure - while I agree, prompt engineering will go away, there's probably a huge market for pre-built integrations in every sector. Bind this AI platform to this industry specific tooling in a polished way, etc.

BoorishBears 3y ago

I feel like people who write stuff like this don't understand the difference between software and software that delivers value. I mean, you think those graphics optimizations we pour millions of dollars into before releasing AAA games will matter when the GTX 8020 outperforms a 4090? Hint: Delivering value for actual people is rarely the result of sitting on your hands and waiting for the next big platform, or even rolling up your sleeves and trying to learn how to build the next big thing. You have this idea that by investing energy in something that will be obsoleted you're losing out, but spoiler, that's how 99% of software that delivers actual value works. The cutting edge of tech almost always ends up being PaaS/SaaS serving itself: Your mail gets to you because someone is working on software with limitations we solved decades ago. Your paycheck ends up in your bank account because people invest a ton of time in codebases subject to problems we solved long ago. Your anti-lock brakes aren't built on a Rust codebase, but some horrible memory unsafe mess running on a processor that's a decade out of date. — The reality is: 99% of the effort that goes into trying to build the next big thing goes nowhere. The expected value of you trying to learn "the actual AI stuff" to the greater world is near 0 compared to you "just prompt engineering" and putting out something that solves a pain-point nicely with GPT 3.5. At the end of the day most of the value that gets delivered to actual users comes from engineers who went deeper into extracting value from the current thing. We still need people to work on the next big thing so that the 1% of effort that isn't wasted can actually materialize... but in my experience the most successful engineers in that regard are still able to realize the delusion it requires, without being paralyzed by the cognitive dissonance that realization invites.

parpfish 3y ago
prompt engineering can definitely deliver value for a customer, but i don't think it will deliver value to a developer in terms of building a sustainable career. a different analogy that gets at my original concern: becoming an expert prompt engineer for an particular LLM is like becoming a power user for a piece of proprietary software that isn't getting any more updates.
- BoorishBears 3y ago
  The first point doesn't follow the second: there will always be a sustainable career in building software that delivers value. In tech we take it for granted that just because there's some new hotness everyone wants to jump on it day 1. GPT 5 could drop tomorrow and if your tool delivers value using 3.5, it's not going to magically stop delivering value, and in most verticals people will prefer your battle tested 3.5 to some brand new 5. And if 5 does simplify prompts for your use case and there'll still only be two options: - making a the same thing as what you made with 3.5 is now trivial... in which case you still have the mindshare and the distribution solved to a degree your newly enabled clones don't. - making a better version of what you made is now trivial... in which case you can just as trivially improve your version and already have the mindshare and distribution solved. At the end of the day software developers often struggle to fit software into the larger ecosystem it slots into before it becomes something valuable, and to be GPT has been an amazing case study in the fact. I think the use of the word engineering did half the damage, and I think people confusing the twitter memes with the interesting attempts at prompt engineering (via ReACT, Gorilla, etc) did the other half, but at the end of the day I think a lot of people will be left kicking themselves when "misguided prompt engineers" end up solving real useful problems in ways they didn't think were possible well before we reach the arbitrary goalposts for foundation models that people keep setting up.