3 ms·
It seems consistent with the behavior we see when using GPT4 in chat mode. Every once in a while it will change its answer as it’s generating it, as though it’s
by woeirua 3y ago
It seems consistent with the behavior we see when using GPT4 in chat mode. Every once in a while it will change its answer as it’s generating it, as though it’s switched which model it favors to produce the response. GPT3.5 doesn’t do that.
- famouswaffles 3y agoMoE models don't work like that though
- SgtBastard 3y agoToken-at-time MoE models could result in mid-output-stream directional change. I agree with you that whole-output MoE models don’t.