3 ms·
sorry, my mistake, I meant draft not predict > it states that OpenAI choose not to disaggregate prefill and decode They disaggregate INSIDE the chip, not by h
by dist-epoch 2mo ago
sorry, my mistake, I meant draft not predict
> it states that OpenAI choose not to disaggregate prefill and decode
They disaggregate INSIDE the chip, not by having separate machines for the 3 phases. the slides:
https://x.com/beffjezos/status/2092416851737518190 https://x.com/beffjezos/status/2092416851737518190