4 ms·

If you have been hired as a "Prompt Engineer", can you please explain your work? Are you just finding better ways to communicate clearly in human language to th

by dpflan 3y ago

If you have been hired as a "Prompt Engineer", can you please explain your work? Are you just finding better ways to communicate clearly in human language to the current model you are interacting with? Or are you doing statistical analyses of prompts, etc? It is hard for me not think a Prompt Engineer as most basic is someone comfortable and knowledgeable about how to express a concept in human language to an LLM, which to me, really sounds like at minimum: clearly communicate with little ambiguity.

hanniabu 3y ago
It's not as simple as being descriptive, but what you're of descriptive and knowing the terms to use. You also need to know the tools to use and there's a lot of settings you play with that are hidden from the user in mainstream apps.
- deleted 3y ago
  [deleted]
binarymax 3y ago
I'm not a full time "prompt engineer" but it's a small role in the work that I do for my business. Prompt engineering is kinda like a basic form of data science. You have a dataset and some manually labelled results, and you hypothesize on prompts and try to improve on some metric. You'd be surprised how the tiniest of changes will alter a result. Thinks like a hyphen, semicolon, or capitalization can sway the metric. It's very finicky and very annoying.
- msp26 3y ago
  Tokenization errors are also really, really annoying to deal with. e.g. why does the JSON output have silly whitespace/quotation sometimes? Obviously it's because the first token the model output was `{` and not `{"` like it should be. Obviously.
  - binarymax 3y ago
    Totally. When using GPT I exclusively use function calling, which is far more reliable for JSON output. When using other models I don't even bother with strict JSON because of the error rate - and instead opt for HTML which is more forgiving when parsing.
    - msp26 3y ago
      nono, the same thing happens with OpenAI's JSON mode and function calling. It does output parsable JSON very reliably but it comes out mangled with a bunch of whitespace sometimes. GPT-4-turbo's output context window is limited to 4096 so fixing this is relevant. You can use logit_bias for it.

by dpflan 3y ago

hanniabu 3y ago

It's not as simple as being descriptive, but what you're of descriptive and knowing the terms to use. You also need to know the tools to use and there's a lot of settings you play with that are hidden from the user in mainstream apps.

deleted 3y ago
[deleted]

binarymax 3y ago

I'm not a full time "prompt engineer" but it's a small role in the work that I do for my business. Prompt engineering is kinda like a basic form of data science. You have a dataset and some manually labelled results, and you hypothesize on prompts and try to improve on some metric. You'd be surprised how the tiniest of changes will alter a result. Thinks like a hyphen, semicolon, or capitalization can sway the metric. It's very finicky and very annoying.

msp26 3y ago
Tokenization errors are also really, really annoying to deal with. e.g. why does the JSON output have silly whitespace/quotation sometimes? Obviously it's because the first token the model output was `{` and not `{"` like it should be. Obviously.
- binarymax 3y ago
  Totally. When using GPT I exclusively use function calling, which is far more reliable for JSON output. When using other models I don't even bother with strict JSON because of the error rate - and instead opt for HTML which is more forgiving when parsing.
  - msp26 3y ago
    nono, the same thing happens with OpenAI's JSON mode and function calling. It does output parsable JSON very reliably but it comes out mangled with a bunch of whitespace sometimes. GPT-4-turbo's output context window is limited to 4096 so fixing this is relevant. You can use logit_bias for it.