2 ms·
Not standard but one of several techniques, you can see them in our open source inference proxy - https://github.com/codelion/optillm https://github.com/codelio
by codelion 1y ago
Not standard but one of several techniques, you can see them in our open source inference proxy - https://github.com/codelion/optillm https://github.com/codelion/optillm
Cerebras has used optillm for optimising inference with techniques like CePO and LongCePO.