6 ms·

Favorites from our prompt engineering tournament

39 points by jzone3 2y ago

NegativeLatency 2y ago
I couldn't find any results, just prompts, did I miss something?
- furyofantares 2y ago
  I guess it was an accurately titled article for once
a1371 2y ago
Prompt engineering doesn't feel like an activity that creates sustainable AI advancement. A prompt may work well with one model, in most situations, but even the best practices seem too experimental. For their competition to avoid a PR disaster, isn't it better to look in the model? Perhaps observe the weights, when the AI says something that you want to avoid in the future. A safeguard could trigger if the model is going in that direction.
- LZ_Khan 2y ago
  LLM's are trained in much the same way, so while your point stands, most/all of the tips here are going to be useful for LLM's for at least a year or so. If a tip was like "use XML tags to give clarity to the model," then it wouldn't be sustainable.
- dvt 2y ago
  > Prompt engineering doesn't feel like an activity that creates sustainable AI advancement. Agreed, it should really be rolled into fine tuning. If you're building a model for PR, for example, it should already be fine tuned so it can't say anything disastrous. Prompt engineering is only really relevant to general-purpose models which aren't that useful to begin with (other than "fun" chatting).
- kang 2y ago
  > Prompt engineering doesn't feel like an activity that creates sustainable AI advancement. Chatgpt was created from gpt via prompt engineering? An inverse chatgpt where user answers questions instead of the other way around also has applications.
  - rcxdude 2y ago
    Only in that the very initial version was GPT + prompt. ChatGPT is heavily fine-tuned using both handwritten examples and later on rated examples from its own output. The prompt is still relevant but the fine-tuning is the biggest thing that makes it work like a chatbot.
snapcaster 2y ago
Interesting analogue to books like "How to win friends and influence people". This genre of self help books include a lot of things that when you squint look like prompt engineering on humans
- babyshake 2y ago
  Although doesn't the Dale Carnegie book focus mostly on being a good listener and using emotional intelligence? Prompt engineering doesn't seem to be very similar to this AFAIK.
  - snapcaster 2y ago
    Things like "Begin with praise and honest appreciation. Call attention to people's mistakes indirectly. Talk about your own mistakes before criticizing the other person" seem more like prompting tricks
xyst 2y ago
Am I the only one annoyed by the term “prompt engineer{ing}”? I thought this was a meme, but I have actually seen some job posts for “prompt engineer”.
- n4r9 2y ago
  I get a feeling of incongruity about the term. I think it's because prompts aren't a "system". At least as far as I understand it.
- kataklasm 2y ago
  Just as much as by the equivalence of software developer and software engineer. Both are a misuse of the term "engineer". Maybe one more than the other, but still.
- deleted 2y ago
  [deleted]
- makk 2y ago
  Those job posts are seeking engineers who show up on time and deliver their work expeditiously.
agys 2y ago
Last year I’ve seen a live “prompt battle” and it was great: a single-elimination tournament with an applause meter, a hype-man and music! https://promptbattle.com https://promptbattle.com
TheDudeMan 2y ago
So silly to call it engineering.
eggfriedrice 2y ago
I'm not sure I understood much except the photo of the cake
six_four_eight 2y ago
Towards the end they state: ‘… just adding “do not hallucinate” has been shown to reduce the odds a model hallucinates.’ I find this surprising and doesn’t fit with my understanding of how a language model works. But I’m very much a novice. Would this be due to update training including feedback that marks bad responses with the term “hallucinate”?
- ManuelKiessling 2y ago
  My mental model is “telling a LLM to not make any mistakes is like telling a depressed person to stop feeling bad”.
- anotheryou 2y ago
  my model is that you need to tell give it ways to make the hallucination not the most plausible thing. I prefer to tell it that it can say "I don't know"
deleted 2y ago
[deleted]

39 points by jzone3 2y ago

NegativeLatency 2y ago

I couldn't find any results, just prompts, did I miss something?

furyofantares 2y ago
I guess it was an accurately titled article for once

a1371 2y ago

Prompt engineering doesn't feel like an activity that creates sustainable AI advancement. A prompt may work well with one model, in most situations, but even the best practices seem too experimental. For their competition to avoid a PR disaster, isn't it better to look in the model? Perhaps observe the weights, when the AI says something that you want to avoid in the future. A safeguard could trigger if the model is going in that direction.

LZ_Khan 2y ago
LLM's are trained in much the same way, so while your point stands, most/all of the tips here are going to be useful for LLM's for at least a year or so. If a tip was like "use XML tags to give clarity to the model," then it wouldn't be sustainable.
dvt 2y ago
> Prompt engineering doesn't feel like an activity that creates sustainable AI advancement. Agreed, it should really be rolled into fine tuning. If you're building a model for PR, for example, it should already be fine tuned so it can't say anything disastrous. Prompt engineering is only really relevant to general-purpose models which aren't that useful to begin with (other than "fun" chatting).
kang 2y ago
> Prompt engineering doesn't feel like an activity that creates sustainable AI advancement. Chatgpt was created from gpt via prompt engineering? An inverse chatgpt where user answers questions instead of the other way around also has applications.
- rcxdude 2y ago
  Only in that the very initial version was GPT + prompt. ChatGPT is heavily fine-tuned using both handwritten examples and later on rated examples from its own output. The prompt is still relevant but the fine-tuning is the biggest thing that makes it work like a chatbot.

snapcaster 2y ago

Interesting analogue to books like "How to win friends and influence people". This genre of self help books include a lot of things that when you squint look like prompt engineering on humans

babyshake 2y ago
Although doesn't the Dale Carnegie book focus mostly on being a good listener and using emotional intelligence? Prompt engineering doesn't seem to be very similar to this AFAIK.
- snapcaster 2y ago
  Things like "Begin with praise and honest appreciation. Call attention to people's mistakes indirectly. Talk about your own mistakes before criticizing the other person" seem more like prompting tricks

xyst 2y ago

Am I the only one annoyed by the term “prompt engineer{ing}”? I thought this was a meme, but I have actually seen some job posts for “prompt engineer”.

n4r9 2y ago
I get a feeling of incongruity about the term. I think it's because prompts aren't a "system". At least as far as I understand it.
kataklasm 2y ago
Just as much as by the equivalence of software developer and software engineer. Both are a misuse of the term "engineer". Maybe one more than the other, but still.
deleted 2y ago
[deleted]
makk 2y ago
Those job posts are seeking engineers who show up on time and deliver their work expeditiously.

agys 2y ago

Last year I’ve seen a live “prompt battle” and it was great: a single-elimination tournament with an applause meter, a hype-man and music! https://promptbattle.com https://promptbattle.com

TheDudeMan 2y ago

So silly to call it engineering.

eggfriedrice 2y ago

I'm not sure I understood much except the photo of the cake

six_four_eight 2y ago

Towards the end they state: ‘… just adding “do not hallucinate” has been shown to reduce the odds a model hallucinates.’ I find this surprising and doesn’t fit with my understanding of how a language model works. But I’m very much a novice. Would this be due to update training including feedback that marks bad responses with the term “hallucinate”?

ManuelKiessling 2y ago
My mental model is “telling a LLM to not make any mistakes is like telling a depressed person to stop feeling bad”.
anotheryou 2y ago
my model is that you need to tell give it ways to make the hallucination not the most plausible thing. I prefer to tell it that it can say "I don't know"

deleted 2y ago

[deleted]