2 ms·
Given how Chain-of-Thought is being used more and more to improve performance, I wonder if system prompt items like "Give the answer immediately. Provide detail
by fastball 2y ago
Given how Chain-of-Thought is being used more and more to improve performance, I wonder if system prompt items like "Give the answer immediately. Provide detailed explanations and restate my query in your own words if necessary after giving the answer" will actually hurt effectiveness of the LLM.
- TeMPOraL 2y agoIt always hurt the effectiveness of the LLMs. Asking them to be concise and give answers without explanations has always been an easy way to degrade model performance.
- ithkuil 2y agoDecoupling internal rumination from the final human friendly summary is the essence of "reasoning" models. It's not that I don't want the model to emit more tokens. I just don't need to read them all. Hence a formal split between thinking phase and communication phase provides the best of both worlds (at the expense of latency(
- porridgeraisin 2y agoOn a "pure" LLM, yes. But it is possible that LLM providers circumvent this. For example, it might be the case that claude when set to concise mode, doesn't apply that to the thinking tokens, and only applies it to the summary. Or, the provider could be augmenting your prompt. From my simple tests on chatgpt, it seems that this is not the case, and asking it to be terse cuts the CoT tokens short as well. Someone needs to test on Claude 3.7 with the reasoning settings.