5 ms·

It's time to bind "Please be concise in your answer and only mention important details. Use a single paragraph and avoid lists. Keep me in the discussion, I'll

by wruza 2y ago

It's time to bind "Please be concise in your answer and only mention important details. Use a single paragraph and avoid lists. Keep me in the discussion, I'll ask for details later." to F1.

telchior 2y ago
You've just made me realize that I actually do need that as a macro. Probably type that ten times per day lately. Others might include "in one sentence" or "only answer yes or no, and link sources proving your assertion".
- atonse 2y ago
  If you’re using ChatGPT, add it to your memory so it always remembers that you prefer that.
  - protocolture 2y ago
    Yep. Somehow I made mine noticably grumpier but I dont know which setting or memory piece did the job. I really like not being complimented on literally everything with a wall of text anymore.
  - Hard_Space 2y ago
    No matter how many times I get ChatGPT to write my rules to long-term memory (I checked, and multiple rules exist in LTM multiple times), it inevitably forgets some or all of the rules because after a while, it can only see what's right in front of it, and not (what should be) the defining schema that you might provide.
    - nickpsecurity 2y ago
      I haven't used ChatGPT in a while. I used to run into a problem that sounds similar. If you're talking about: 1. Rules that get prefixed in front of your prompt as part of the real prompt ChatGPT gets. Like what they do with the system prompt. And 2. Some content makes your prompt too big for the context windows where the rules get cut off. Then, it might help to measure the tokens in the overall prompt, have a max number, and warn if it goes over it. I had a custom, chat app that used their API's with this feature built in. Another possibility is, when this is detected, it asks you if you want to use one with a larger, context window. Those cost more. So, it would be presented as an option. My app let me select any of their models to do that manually.
mips_avatar 2y ago
Yeah but it kind of kneecaps the model. They need tokens to "think". It's better to have them create a long response then distill it down later.
- wruza 2y ago
  Is there a well-known benchmark for this? I don't feel that short vs long answers make any difference, but ofc feelings aren't what we can measure. Also, if that works, why doesn't copilot/cursor write lots of excessive code mixed with lots of prose only to distill it later?
  - teruakohatu 2y ago
    > don't feel that short vs long answers make any difference The “thinking” models are really verbose output models that summarise the thinking at the end. These tend to outperform non-thinking models, but at a higher cost. Anthropic lets you see some/all of the thinking so you can see how the model arrived at the answer.
    - wruza 2y ago
      So if I replace "answer" with "summarize" that should work then?
- dailykoder 2y ago
  You need tokens to create more revenue for the company that is running the LLM. Nothing more, nothing less

by wruza 2y ago

It's time to bind "Please be concise in your answer and only mention important details. Use a single paragraph and avoid lists. Keep me in the discussion, I'll ask for details later." to F1.

telchior 2y ago

You've just made me realize that I actually do need that as a macro. Probably type that ten times per day lately. Others might include "in one sentence" or "only answer yes or no, and link sources proving your assertion".

atonse 2y ago
If you’re using ChatGPT, add it to your memory so it always remembers that you prefer that.
- protocolture 2y ago
  Yep. Somehow I made mine noticably grumpier but I dont know which setting or memory piece did the job. I really like not being complimented on literally everything with a wall of text anymore.
- Hard_Space 2y ago
  No matter how many times I get ChatGPT to write my rules to long-term memory (I checked, and multiple rules exist in LTM multiple times), it inevitably forgets some or all of the rules because after a while, it can only see what's right in front of it, and not (what should be) the defining schema that you might provide.
  - nickpsecurity 2y ago
    I haven't used ChatGPT in a while. I used to run into a problem that sounds similar. If you're talking about: 1. Rules that get prefixed in front of your prompt as part of the real prompt ChatGPT gets. Like what they do with the system prompt. And 2. Some content makes your prompt too big for the context windows where the rules get cut off. Then, it might help to measure the tokens in the overall prompt, have a max number, and warn if it goes over it. I had a custom, chat app that used their API's with this feature built in. Another possibility is, when this is detected, it asks you if you want to use one with a larger, context window. Those cost more. So, it would be presented as an option. My app let me select any of their models to do that manually.

mips_avatar 2y ago

Yeah but it kind of kneecaps the model. They need tokens to "think". It's better to have them create a long response then distill it down later.

wruza 2y ago
Is there a well-known benchmark for this? I don't feel that short vs long answers make any difference, but ofc feelings aren't what we can measure. Also, if that works, why doesn't copilot/cursor write lots of excessive code mixed with lots of prose only to distill it later?
- teruakohatu 2y ago
  > don't feel that short vs long answers make any difference The “thinking” models are really verbose output models that summarise the thinking at the end. These tend to outperform non-thinking models, but at a higher cost. Anthropic lets you see some/all of the thinking so you can see how the model arrived at the answer.
  - wruza 2y ago
    So if I replace "answer" with "summarize" that should work then?
dailykoder 2y ago
You need tokens to create more revenue for the company that is running the LLM. Nothing more, nothing less