2 ms·
You mean this https://lmsys.org/blog/2024-02-05-compressed-fsm/ https://lmsys.org/blog/2024-02-05-compressed-fsm/
by imtringued 2y ago
You mean this https://lmsys.org/blog/2024-02-05-compressed-fsm/ https://lmsys.org/blog/2024-02-05-compressed-fsm/
- OutOfHere 2y agoIt has a clever way to decode multiple valid tokens at once, rather than just one token at a time. Corresponding project link: https://github.com/sgl-project/sglang https://github.com/sgl-project/sglang