4 ms·

If I take advantage of this for allowing my customers to ask questions about our product documentation, how do I limit questions to questions about my product d

If I take advantage of this for allowing my customers to ask questions about our product documentation, how do I limit questions to questions about my product documentation? I don't want to be paying Open OPI for questions unrelated to my product.

th3h4mm3r 4y ago
This is really a good question. Could be interesting also which is the best method to provide to GPT a Product documentato and ask question inherents to that and nothing else.
shagie 4y ago
One idea would be to use a much cheaper (and faster) classifier to come back with a "yes" or "no" if the question asked is about your product documentation. Using Ada or Babbage is about 1% of the cost of Davinci (and Curie is 10% the cost of Davinci). Without any real tuning, this responds quite promptly (and the various tests I've done, correctly): curl https://api.openai.com/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "text-ada-001", "prompt": "Identify if the following question is about the Olympic Games. Answer with {yes}, {no}, {maybe}.\n\nWhat category is pole vaulting in?", "temperature": 0.7, "max_tokens": 38, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }'
- bongobingo1 4y ago
  > What category is pole vaulting in? >> Yes Well, at least you can't say the documentation is out of date.
  - jesselawson 4y ago
    Honestly the best comment I've read on here in a long time. Cheers.
  - shagie 4y ago
    I was just after something simple in its training set to demonstrate the simple classification that could be used as a check before passing the request on to a more expensive fine tuned model.
- keithwhor 4y ago
  It's way easier and cheaper to use embeddings for these sorts of use cases.
  - shagie 4y ago
    It probably is - I just haven't toyed with that part of the system yet.
amrb 4y ago
I'm going to coin the cyber-security finding "Bill exhaustion via open ML prompts".. wonder how long till it's in the Owasp top X

th3h4mm3r 4y ago

This is really a good question. Could be interesting also which is the best method to provide to GPT a Product documentato and ask question inherents to that and nothing else.

shagie 4y ago

One idea would be to use a much cheaper (and faster) classifier to come back with a "yes" or "no" if the question asked is about your product documentation. Using Ada or Babbage is about 1% of the cost of Davinci (and Curie is 10% the cost of Davinci). Without any real tuning, this responds quite promptly (and the various tests I've done, correctly): curl https://api.openai.com/v1/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "text-ada-001", "prompt": "Identify if the following question is about the Olympic Games. Answer with {yes}, {no}, {maybe}.\n\nWhat category is pole vaulting in?", "temperature": 0.7, "max_tokens": 38, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }'

bongobingo1 4y ago
> What category is pole vaulting in? >> Yes Well, at least you can't say the documentation is out of date.
- jesselawson 4y ago
  Honestly the best comment I've read on here in a long time. Cheers.
- shagie 4y ago
  I was just after something simple in its training set to demonstrate the simple classification that could be used as a check before passing the request on to a more expensive fine tuned model.
keithwhor 4y ago
It's way easier and cheaper to use embeddings for these sorts of use cases.
- shagie 4y ago
  It probably is - I just haven't toyed with that part of the system yet.

amrb 4y ago

I'm going to coin the cyber-security finding "Bill exhaustion via open ML prompts".. wonder how long till it's in the Owasp top X