4 ms·

The fact that everyone asks it to be terse is interesting to me. I find the output to be of far greater quality if you let it talk. In fact, the default with no

The fact that everyone asks it to be terse is interesting to me. I find the output to be of far greater quality if you let it talk. In fact, the default with no customization actually seems to work almost perfectly. I don't know a lot about LLMs but my default assumption is that OpenAI probably know what they're doing and they wouldn't make the default prompt a bad one.

tomashubelbauer 2y ago
I'd be less inclined to put that instruction there now with the faster Omni, but GPT4 was too slow to let it ramble, it wouldn't get to the point fast enough by itself. And of course it would waste three seconds starting off by rewording your question to open its answer.
- p1esk 2y ago
  In my system prompt I ask it to always start with repeating my question in a rephrased form. Though it’s needed more for lesser models, gpt4 seems to always understand my questions perfectly.
drexlspivey 2y ago
You prefer this response instead of the one line command? https://chatgpt.com/share/8c97085e-70cc-4e62-8a54-3a64f95744bd https://chatgpt.com/share/8c97085e-70cc-4e62-8a54-3a64f95744...
- LeoPanthera 2y ago
  A single example does not prove the rule.
matsemann 2y ago
My experience as well. Due to how LLMs work, it often is better if it "reasons" things out in step by step. Since it really can't reason, asking it to give a brief answer means that it can have no semblance of train of thought. Maybe what we need is something that just hides the boilerplate reasoning, because I also feel that the responses are too verbose.
- anticensor 2y ago
  That one is easy: Generate the long answer behind the scenes, and then feed it to a special-purpose summarisation model (the type that lets you determine the output length) to summarise it.
striking 2y ago
Most folks don't realize that each token produced is an opportunity for it to do more computation, and that they are actively making it dumber by asking for as brief a response as possible. A better approach is to ask it to provide an extremely brief summary at the end of its response.
- drexlspivey 2y ago
  Does more computation mean a better answer? If I ask it who was the king of England in 1850 the answer is a single name, everything else is completely useless.
  - striking 2y ago
    I mean in the general case. I have my instructions for brevity gated behind a key phrase, because I generally use ChatGPT as a vibe-y computation tool rather than a fact finding tool. I don't know that I'd trust it to spit out just one fact without a justification unless I didn't actually care much for the validity of the answer.
  - acchow 2y ago
    It gives better reuslts with “chain of thought”
  - have_faith 2y ago
    It's potentially a problem for follow up questions. As the whole conversation, to a limited amount of tokens, is fed back into itself to produce the next tokens (ad infinitum). So being terse leaves less room to find conceptual links between words, concepts, phrases, etc, because there are less of them being parsed for every new token requested. This isn't black and white though as being terse can sometimes avoid unwanted connections being made, and tangents being unnecessarily followed.
  - card_zero 2y ago
    King Victoria. Does that not benefit from a few clarifying words? Or is your whole point that "Victoria" is sufficient?
jamesponddotco 2y ago
It's even more interesting if you take into consideration that for Claude, making it be more verbose and "think" about its answer improves the output. I imagine that something similar happens with GPT, but I never tested that.
- dinkleberg 2y ago
  I have been wondering that now that the context windows are larger if letting it “think” more will result in higher quality results. The big problem I had earlier on, especially when doing code related chats, would be be it printing out all source code in every message and almost instantly forgetting what the original topic was.
hn_throwaway_99 2y ago
> my default assumption is that OpenAI probably know what they're doing and they wouldn't make the default prompt a bad one. That's not really a great assumption. Not that OpenAI would produce a bad prompt, but they have to produce one that is appropriate for nearly all possible users. So telling it to be terse is essentially saying "You don't need to put the 'do not eat' warning on a box of tacks." Also, a lot of these comments are not just about terseness, e.g. many request step-by-step, chain-of-thought style reasoning. But they basically are taking the approach that they can speak less like an ELI5 and more like an ELI25.
mrbombastic 2y ago
I am not sure assuming they know what they are doing is too reasonable but it might be reasonable to assume they will optimize for the default so straying too far might be a bad idea anyway
illusive4080 2y ago
I didn’t know that. I always try to make it terse because by default it is far too verbose for my liking. I’ll have to try this out. What if I just ask it for a terse summary at the end? Maybe I’ll get the best of both worlds.
- deleted 2y ago
  [deleted]
sandspar 2y ago
I'd rather have a buddy with an IQ of 115 who I enjoy talking to than one with an IQ of 120 who I find annoying.
itomato 2y ago
Maybe an artifact of the 4K token limit
BiteCode_dev 2y ago
Because it works. We tried the alternative, and it's less productive. At some point, there is the theory and practice. Since LLM output are anything but an exact science from the users perspective, trials and errors are what's up. You can state all day long how it works internally and how people should use it, but people I've not waited for you, they used it intensively, for million of hours. And they know.
cal85 2y ago
It works. I agree, more words seem to result in better critical rigour. But for the majority of my casual use cases it is capable of perfectly accurate and complete answers in just a few tokens, so I configure it to prefer short, direct answers. But this is just a suggestion. It seems to understand when a task is complex enough to require more verbiage for more careful reasoning. Or I can easily steer it towards longer answers when I think they’re needed, by telling it to go through something in detail or step by step etc. The main benefit of asking for terseness in your preferences is that it significantly reduces pleasantries etc. (Not that I want it completely dry and robotic, but it just waffles too much out of the box.)

tomashubelbauer 2y ago

I'd be less inclined to put that instruction there now with the faster Omni, but GPT4 was too slow to let it ramble, it wouldn't get to the point fast enough by itself. And of course it would waste three seconds starting off by rewording your question to open its answer.

p1esk 2y ago
In my system prompt I ask it to always start with repeating my question in a rephrased form. Though it’s needed more for lesser models, gpt4 seems to always understand my questions perfectly.

drexlspivey 2y ago

You prefer this response instead of the one line command? https://chatgpt.com/share/8c97085e-70cc-4e62-8a54-3a64f95744bd https://chatgpt.com/share/8c97085e-70cc-4e62-8a54-3a64f95744...

LeoPanthera 2y ago
A single example does not prove the rule.

matsemann 2y ago

My experience as well. Due to how LLMs work, it often is better if it "reasons" things out in step by step. Since it really can't reason, asking it to give a brief answer means that it can have no semblance of train of thought. Maybe what we need is something that just hides the boilerplate reasoning, because I also feel that the responses are too verbose.

anticensor 2y ago
That one is easy: Generate the long answer behind the scenes, and then feed it to a special-purpose summarisation model (the type that lets you determine the output length) to summarise it.

striking 2y ago

Most folks don't realize that each token produced is an opportunity for it to do more computation, and that they are actively making it dumber by asking for as brief a response as possible. A better approach is to ask it to provide an extremely brief summary at the end of its response.

drexlspivey 2y ago
Does more computation mean a better answer? If I ask it who was the king of England in 1850 the answer is a single name, everything else is completely useless.
- striking 2y ago
  I mean in the general case. I have my instructions for brevity gated behind a key phrase, because I generally use ChatGPT as a vibe-y computation tool rather than a fact finding tool. I don't know that I'd trust it to spit out just one fact without a justification unless I didn't actually care much for the validity of the answer.
- acchow 2y ago
  It gives better reuslts with “chain of thought”
- have_faith 2y ago
  It's potentially a problem for follow up questions. As the whole conversation, to a limited amount of tokens, is fed back into itself to produce the next tokens (ad infinitum). So being terse leaves less room to find conceptual links between words, concepts, phrases, etc, because there are less of them being parsed for every new token requested. This isn't black and white though as being terse can sometimes avoid unwanted connections being made, and tangents being unnecessarily followed.
- card_zero 2y ago
  King Victoria. Does that not benefit from a few clarifying words? Or is your whole point that "Victoria" is sufficient?

jamesponddotco 2y ago

It's even more interesting if you take into consideration that for Claude, making it be more verbose and "think" about its answer improves the output. I imagine that something similar happens with GPT, but I never tested that.

dinkleberg 2y ago
I have been wondering that now that the context windows are larger if letting it “think” more will result in higher quality results. The big problem I had earlier on, especially when doing code related chats, would be be it printing out all source code in every message and almost instantly forgetting what the original topic was.

hn_throwaway_99 2y ago

> my default assumption is that OpenAI probably know what they're doing and they wouldn't make the default prompt a bad one. That's not really a great assumption. Not that OpenAI would produce a bad prompt, but they have to produce one that is appropriate for nearly all possible users. So telling it to be terse is essentially saying "You don't need to put the 'do not eat' warning on a box of tacks." Also, a lot of these comments are not just about terseness, e.g. many request step-by-step, chain-of-thought style reasoning. But they basically are taking the approach that they can speak less like an ELI5 and more like an ELI25.

mrbombastic 2y ago

I am not sure assuming they know what they are doing is too reasonable but it might be reasonable to assume they will optimize for the default so straying too far might be a bad idea anyway

illusive4080 2y ago

I didn’t know that. I always try to make it terse because by default it is far too verbose for my liking. I’ll have to try this out. What if I just ask it for a terse summary at the end? Maybe I’ll get the best of both worlds.

deleted 2y ago
[deleted]

sandspar 2y ago

I'd rather have a buddy with an IQ of 115 who I enjoy talking to than one with an IQ of 120 who I find annoying.

itomato 2y ago

Maybe an artifact of the 4K token limit

BiteCode_dev 2y ago

Because it works. We tried the alternative, and it's less productive. At some point, there is the theory and practice. Since LLM output are anything but an exact science from the users perspective, trials and errors are what's up. You can state all day long how it works internally and how people should use it, but people I've not waited for you, they used it intensively, for million of hours. And they know.

cal85 2y ago

It works. I agree, more words seem to result in better critical rigour. But for the majority of my casual use cases it is capable of perfectly accurate and complete answers in just a few tokens, so I configure it to prefer short, direct answers. But this is just a suggestion. It seems to understand when a task is complex enough to require more verbiage for more careful reasoning. Or I can easily steer it towards longer answers when I think they’re needed, by telling it to go through something in detail or step by step etc. The main benefit of asking for terseness in your preferences is that it significantly reduces pleasantries etc. (Not that I want it completely dry and robotic, but it just waffles too much out of the box.)