5 ms·

> It’s not hard to imagine future startups for which the secret sauce of their product is a carefully crafted prompt. If your startup's entire technology stack

by jhugo 4y ago

> It’s not hard to imagine future startups for which the secret sauce of their product is a carefully crafted prompt. If your startup's entire technology stack is "provide a prompt to a system that you don't understand, which may or may not follow it accurately, and concatenate user-supplied strings directly into it", I don't really know what you expect...

cl42 4y ago
On the one hand, I am inclined to agree with you... On the other hand, having spent time with large language models (LLMs) and GPT-3 in particular, there really is an art to writing prompts, structuring them, etc. I find it hard to defend a business model that has one specific prompt that makes or breaks the company... But there are definitely people who are skilled at writing prompts and I can see companies protecting their prompts (via patents? Trade secrets? Something else?) the way gene sequences are protected, ingredients for popular soft drinks are protected, or the way special molecules are protected. Anyway, I find it fascinating how deep this line of thinking can go, all from the idea of a "one prompt company" protecting its special sequence of characters that unlocks a specific set of sequences in a LLM matrix! :)
- hackerlight 4y ago
  > GPT-3 in particular, there really is an art to writing prompts, structuring them, etc. That'll stop being the case in GPT-N. Prompt engineering might be a thing for a short time only.
  - smsm42 4y ago
    They said it for SEO too, and yet here we are...
  - muzani 4y ago
    I expect it to be like black hat SEO. It could well be a decade. And the result might end up being some heavily neutered output, that's does the job, but is not very interesting.
  - sebzim4500 4y ago
    I imagine that the benefits of well written prompts will decrease over time but they will probably still exist. There is still a knack to getting useful information out of google/bing/etc., even if it is much easier than it was decades ago.
  - samatman 4y ago
    I think this basically misunderstands what the mulchers actually are. The prompt is the only thing there is, it's all they will ever understand. Therefore, engineering prompts is the only way to get behavior out of them. That won't change unless the architecture changes, in which case, it isn't GPT-N.
  - NBJack 4y ago
    Given we are effectively modeling humans, and there is often plenty of room for misinterpretation in human natural language, we will see. There are good reasons things that 'must not fail' (i.e. civil engineering, accepted medical practice, etc.) are meant to go through rigorous formal review to get it both right and agreed on.
    - melagonster 4y ago
      I trust they spent many years to convert normal language to professional terms, this is a way to reduce influences from natural language.
jstanley 4y ago
Prompts that we run into in blog posts are generally quite small, but it's not hard to imagine prompts of comparable complexity to a typical contemporary startup's entire code base.
nxmnxm99 4y ago
It would be fairly straightforward to reverse engineer that though
TheCoreh 4y ago
> provide a prompt to a system that you don't understand, which may or may not follow it accurately Isn't this also the case whenever you employ a human being to do anything? These attacks are not unlike "social engineering", I guess the AI is just particularly naïve. Maybe the solution is to have a second AI inspect the behavior of the first one to double check it was not fooled.

by jhugo 4y ago

cl42 4y ago

On the one hand, I am inclined to agree with you... On the other hand, having spent time with large language models (LLMs) and GPT-3 in particular, there really is an art to writing prompts, structuring them, etc. I find it hard to defend a business model that has one specific prompt that makes or breaks the company... But there are definitely people who are skilled at writing prompts and I can see companies protecting their prompts (via patents? Trade secrets? Something else?) the way gene sequences are protected, ingredients for popular soft drinks are protected, or the way special molecules are protected. Anyway, I find it fascinating how deep this line of thinking can go, all from the idea of a "one prompt company" protecting its special sequence of characters that unlocks a specific set of sequences in a LLM matrix! :)

hackerlight 4y ago
> GPT-3 in particular, there really is an art to writing prompts, structuring them, etc. That'll stop being the case in GPT-N. Prompt engineering might be a thing for a short time only.
- smsm42 4y ago
  They said it for SEO too, and yet here we are...
- muzani 4y ago
  I expect it to be like black hat SEO. It could well be a decade. And the result might end up being some heavily neutered output, that's does the job, but is not very interesting.
- sebzim4500 4y ago
  I imagine that the benefits of well written prompts will decrease over time but they will probably still exist. There is still a knack to getting useful information out of google/bing/etc., even if it is much easier than it was decades ago.
- samatman 4y ago
  I think this basically misunderstands what the mulchers actually are. The prompt is the only thing there is, it's all they will ever understand. Therefore, engineering prompts is the only way to get behavior out of them. That won't change unless the architecture changes, in which case, it isn't GPT-N.
- NBJack 4y ago
  Given we are effectively modeling humans, and there is often plenty of room for misinterpretation in human natural language, we will see. There are good reasons things that 'must not fail' (i.e. civil engineering, accepted medical practice, etc.) are meant to go through rigorous formal review to get it both right and agreed on.
  - melagonster 4y ago
    I trust they spent many years to convert normal language to professional terms, this is a way to reduce influences from natural language.

jstanley 4y ago

Prompts that we run into in blog posts are generally quite small, but it's not hard to imagine prompts of comparable complexity to a typical contemporary startup's entire code base.

nxmnxm99 4y ago

It would be fairly straightforward to reverse engineer that though

TheCoreh 4y ago

> provide a prompt to a system that you don't understand, which may or may not follow it accurately Isn't this also the case whenever you employ a human being to do anything? These attacks are not unlike "social engineering", I guess the AI is just particularly naïve. Maybe the solution is to have a second AI inspect the behavior of the first one to double check it was not fooled.