8 ms·

Good article. The rise of "prompt influencers" is frustrating, since it makes it a bit trickier to find the signal of interesting AI content through the noise.

by calny 3y ago

Good article. The rise of "prompt influencers" is frustrating, since it makes it a bit trickier to find the signal of interesting AI content through the noise. I miss when the influencers were all just hawking altcoins. Guess I'll stick to watching Two Minute Papers[0] and reading papers that ak[1] tweets. > Being “good at prompting” is a temporary state of affairs. Valid point. Sam A said he thinks prompt engineering won't even be a thing in 5 years. And in the shorter term, prompts that work with a specific model like GPT-4 might not work well for future models, or even updates to the same model. That said, I see prompt engineering as the beginning of a new paradigm of "intent engineering"--where developers use AI to understand and anticipate user intent with minimal user effort. It'll be fun to see what that looks like in 5 years. [0] https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg [1] https://twitter.com/_akhaliq https://twitter.com/_akhaliq

nonethewiser 3y ago
How can prompt engineering not be a thing so long as prompts are needed? If a different range of inputs produces a different range of outputs how could there be no room for nuances? Or how is intent engineering really any different? It’s all based off an prompt input right? I think part of this comes from the fact that the same input will produce a different output.
- newswasboring 3y ago
  Prompt engineering will be a thing, but I don't think it will be as prevalent as people think it will be. Look at projects like langchain. To me its biggest value is the library of standard prompts it provides. So I think prompt engineering will probably be a "niche" job the same way C programming is a "niche" job. Its super specialized and most people doing C programming are also experts in specific architectures they are programming in.
  - simonw 3y ago
    Saying that LangChain removes the value of learning to write prompts sounds to me like saying that the existence of ORMs removes the value of learning SQL.
    - newswasboring 3y ago
      I never said langchain "removes the value of learning to write prompts". My point is it abstracts it out enough that not everyone working with LLMs will need to know how to do it at a very high level. Just like most programmers can't write assembly/C, but we have tools which abstract it out so that experts can write/generate it for us. I don't know about SQL and ORM to respond to your analogy.
    - mejutoco 3y ago
      > the existence of ORMs removes the value of learning SQL. No that SQL has no value, but this is exactly what ORMs do for a lot of people. They stay in their language instead of having to learn SQL (not saying this is ideal).
noobcoder 3y ago
Yeah, while all the other stuff is important, prompts are a big deal, especially for generative image tasks. They need to be just right for the checkpoint, but if you nail it, you can get an awesome image straight away. Sadly, most people just give up too quickly. But with ChatGPT, even if you give it a terrible prompt, it can still "get" what you're trying to say. You can keep chatting until you get the image you want. It's way easier than with Txt2img, which can be pretty unforgiving. And those courses? Total scam, don't bother with them.
ChatGTP 3y ago
Valid point. Sam A said he thinks prompt engineering won't even be a thing in 5 years. Who cares what Sam A thinks on the matter. Of course he is going to say things like that he wants to make his product sound as amazing as possible.
- coding123 3y ago
  Probably a lot of people, as he has more knowledge of upcoming features.
  - deleted 3y ago
    [deleted]
  - fortyseven 3y ago
    Kind of hoped we'd learned a lesson about putting any one person in a field on a pedestal.
    - ChatGTP 3y ago
      Felt the same
Jevon23 3y ago
>Sam A said he thinks prompt engineering won't even be a thing in 5 years. I don’t understand what this could mean. I have to do “prompt engineering” when I talk to other people all the time - when I need to ask them to do tasks, when I need to clarify requirements, etc. As long as we’re communicating through text and not mind reading, some level of “engineering” will be required. AI is intelligent but it’s not magic.
- ekanes 3y ago
  Prompt engineering now (as I understand it) is you're refining and refining the prompt itself. Perhaps what we might see is you give it a prompt then just keep adjusting the response dynamically, and then it can remember the entire concept as a saved prompt.
  - newswasboring 3y ago
    I already use ChatGPT like this. After every good outcome I say something like "modify my original prompt to include all the things we learned in this chat session". It has mixed results, context window being the biggest factor I guess. Longer conversations (where I am trying to write stories specially) do not produce good prompts as it becomes super specific even if I tell it to generalize specific things.
    - daveguy 3y ago
      Edit: I may have misunderstood. You instruct it to modify the original prompt of that session to include what you learned? Do you ask it to repeat the new prompt? This is interesting, but it wouldn't help for any future sessions. --- It has zero effect. Your prompt consists of the OpenAI prompt + your conversation so far. There is no cross-prompt learning, knowledge or intelligence [except for, in the future, what OpenAI decides to include in model training of new releases]. The main reason for this is there are a limited number of tokens that can be used as the prompt. If you "include everything we learned in this chat session" it would have to include the entire chat session as part of all future prompts and you would quickly run out of tokens for the prompt. Training happens on a corpus, but not during regular use. Perceived "learning" is just context provided by the session-limited prompt.
j0hnyl 3y ago
I don't think prompt engineering is temporary as long as we are using LLMs. Prompt engineering is about creating workflows that squeeze the most impact out of these tools. I don't see how that will go away.
- scotu 3y ago
  the expectation is that we won't be prompting this models the way we do now down the road. As in: prompting is the command line of LLM, at some point we'll get the equivalent of a GUI (either because we can be clueless on how to prompt because the LLM is so good, or it's so good at eliciting your requirements, or because there is no prompt at all and you interface with the LLM completely differently) You could foresee that under the covers there is always going to be some prompting but it's going to be performed rarely by few people?
  - j0hnyl 3y ago
    I agree that there will be new and interesting abstractions for prompting, but I have this feeling that the promise of LLMs for the foreseeable future is to apply them to new and unique business cases. I think this will always require interacting with them at a lower level to some extent.
layoric 3y ago
I am likely wrong here, but I thought the term “prompt engineering” was also related to the act of building a system around generating a prompt dynamically based on limited input? Eg not just trial and error over single prompts but using broader techniques like in-context learning, chain of thought reasoning, other AI models (BERT), vector DB etc to build prompts to send to the LLM? I’m likely wrong cause I can’t remember where I read this definition, but IMO it makes more sense that it relates to building systems around promoting, rather than how a single model reacts to very limited circumstances. Models etc are going to change pretty regularly, tiny adjustments in wording will change along with it, so seem to have limited ROI. Broader techniques will become wide spread pretty quickly, how you use them all together in a system I think will be more important over time, but I don’t know, everything is moving so quickly shrugs.

by calny 3y ago

nonethewiser 3y ago

How can prompt engineering not be a thing so long as prompts are needed? If a different range of inputs produces a different range of outputs how could there be no room for nuances? Or how is intent engineering really any different? It’s all based off an prompt input right? I think part of this comes from the fact that the same input will produce a different output.

newswasboring 3y ago
Prompt engineering will be a thing, but I don't think it will be as prevalent as people think it will be. Look at projects like langchain. To me its biggest value is the library of standard prompts it provides. So I think prompt engineering will probably be a "niche" job the same way C programming is a "niche" job. Its super specialized and most people doing C programming are also experts in specific architectures they are programming in.
- simonw 3y ago
  Saying that LangChain removes the value of learning to write prompts sounds to me like saying that the existence of ORMs removes the value of learning SQL.
  - newswasboring 3y ago
    I never said langchain "removes the value of learning to write prompts". My point is it abstracts it out enough that not everyone working with LLMs will need to know how to do it at a very high level. Just like most programmers can't write assembly/C, but we have tools which abstract it out so that experts can write/generate it for us. I don't know about SQL and ORM to respond to your analogy.
  - mejutoco 3y ago
    > the existence of ORMs removes the value of learning SQL. No that SQL has no value, but this is exactly what ORMs do for a lot of people. They stay in their language instead of having to learn SQL (not saying this is ideal).

noobcoder 3y ago

Yeah, while all the other stuff is important, prompts are a big deal, especially for generative image tasks. They need to be just right for the checkpoint, but if you nail it, you can get an awesome image straight away. Sadly, most people just give up too quickly. But with ChatGPT, even if you give it a terrible prompt, it can still "get" what you're trying to say. You can keep chatting until you get the image you want. It's way easier than with Txt2img, which can be pretty unforgiving. And those courses? Total scam, don't bother with them.

ChatGTP 3y ago

Valid point. Sam A said he thinks prompt engineering won't even be a thing in 5 years. Who cares what Sam A thinks on the matter. Of course he is going to say things like that he wants to make his product sound as amazing as possible.

coding123 3y ago
Probably a lot of people, as he has more knowledge of upcoming features.
- deleted 3y ago
  [deleted]
- fortyseven 3y ago
  Kind of hoped we'd learned a lesson about putting any one person in a field on a pedestal.
  - ChatGTP 3y ago
    Felt the same

Jevon23 3y ago

>Sam A said he thinks prompt engineering won't even be a thing in 5 years. I don’t understand what this could mean. I have to do “prompt engineering” when I talk to other people all the time - when I need to ask them to do tasks, when I need to clarify requirements, etc. As long as we’re communicating through text and not mind reading, some level of “engineering” will be required. AI is intelligent but it’s not magic.

ekanes 3y ago
Prompt engineering now (as I understand it) is you're refining and refining the prompt itself. Perhaps what we might see is you give it a prompt then just keep adjusting the response dynamically, and then it can remember the entire concept as a saved prompt.
- newswasboring 3y ago
  I already use ChatGPT like this. After every good outcome I say something like "modify my original prompt to include all the things we learned in this chat session". It has mixed results, context window being the biggest factor I guess. Longer conversations (where I am trying to write stories specially) do not produce good prompts as it becomes super specific even if I tell it to generalize specific things.
  - daveguy 3y ago
    Edit: I may have misunderstood. You instruct it to modify the original prompt of that session to include what you learned? Do you ask it to repeat the new prompt? This is interesting, but it wouldn't help for any future sessions. --- It has zero effect. Your prompt consists of the OpenAI prompt + your conversation so far. There is no cross-prompt learning, knowledge or intelligence [except for, in the future, what OpenAI decides to include in model training of new releases]. The main reason for this is there are a limited number of tokens that can be used as the prompt. If you "include everything we learned in this chat session" it would have to include the entire chat session as part of all future prompts and you would quickly run out of tokens for the prompt. Training happens on a corpus, but not during regular use. Perceived "learning" is just context provided by the session-limited prompt.

j0hnyl 3y ago

I don't think prompt engineering is temporary as long as we are using LLMs. Prompt engineering is about creating workflows that squeeze the most impact out of these tools. I don't see how that will go away.

scotu 3y ago
the expectation is that we won't be prompting this models the way we do now down the road. As in: prompting is the command line of LLM, at some point we'll get the equivalent of a GUI (either because we can be clueless on how to prompt because the LLM is so good, or it's so good at eliciting your requirements, or because there is no prompt at all and you interface with the LLM completely differently) You could foresee that under the covers there is always going to be some prompting but it's going to be performed rarely by few people?
- j0hnyl 3y ago
  I agree that there will be new and interesting abstractions for prompting, but I have this feeling that the promise of LLMs for the foreseeable future is to apply them to new and unique business cases. I think this will always require interacting with them at a lower level to some extent.

layoric 3y ago

I am likely wrong here, but I thought the term “prompt engineering” was also related to the act of building a system around generating a prompt dynamically based on limited input? Eg not just trial and error over single prompts but using broader techniques like in-context learning, chain of thought reasoning, other AI models (BERT), vector DB etc to build prompts to send to the LLM? I’m likely wrong cause I can’t remember where I read this definition, but IMO it makes more sense that it relates to building systems around promoting, rather than how a single model reacts to very limited circumstances. Models etc are going to change pretty regularly, tiny adjustments in wording will change along with it, so seem to have limited ROI. Broader techniques will become wide spread pretty quickly, how you use them all together in a system I think will be more important over time, but I don’t know, everything is moving so quickly shrugs.