2 ms·
I don’t think this is a plausible argument, as they’re generally capacity constrained, and everyone would like shorter (= faster) responses. I’m fairly certain
by stingraycharles 6mo ago
I don’t think this is a plausible argument, as they’re generally capacity constrained, and everyone would like shorter (= faster) responses.
I’m fairly certain that in a few more releases we’ll have models with shorter CoT chains. Whether they’ll still let us see those is another question, as it seems like Anthropic wants to start hiding their CoT, potentially because it reveals some secret sauce.
- Ifkaluva 6mo agoI guess mainly they don’t want you to distill on their CoT
- stingraycharles 6mo agoYes, which I understand, but I think they’re crippling their product for users this way. I don’t think it’s just this, because the thinking tokens often reveal more about Anthropic’s inner workings. For example, it’s how the whole existence of Claude’s soul document was reverse engineered, it often leaks details about “system reminders” (eg long conversation reminders). I think it’s also just very convenient for Anthropic to do this. The fact that they’re also presenting this as a “performance optimization” suggests they’re not giving the real reason they do this.