> The academic literature has recently caught on to this approach as well and we want to highlight the excellent papers on Sarathi (link 1, link 2) and DeepSpee

> The academic literature has recently caught on to this approach as well and we want to highlight the excellent papers on Sarathi (link 1, link 2) and DeepSpeed-FastGen (link). The academic literature calls the technique “chunked prefill”. The Sarathi paper is more than a year old and implemented in both vllm and trtllm. There doesn't seem to be much else of substance in the article. Edit: to be fair it seems the article was written some time ago and only "updated" two weeks ago. So maybe it was more novel at the time.