Jrxing

3 ms·

1. 1.▲
   Who Routes LLM Routers? RouterArena Builds the Eval Foundation for LLM Routing(huggingface.co)

   1 points by Jrxing 11mo ago|1 comments
2. 2.▲
   by Jrxing 11mo ago

   As LLMs continue to diversifying, model routers become increasingly critical for connecting various models and achieving the best performance-cost trade off. RouterArena is the first open platform & leaderboard for rigorous and comprehe
3. 3.▲
   by Jrxing 1y ago

   Sharing the big GPU cluster with non-latency critical load is one solution we also explored. For this work, we are targeting more on the problem of smaller models running SOTA GPUs. Distilled/fine-tuned small models have shown comparab
4. 4.▲
   by Jrxing 1y ago

   Hi, thanks for digging out who I am. Yes, I am the author of the blog and the project. We polished the blog for several days. I didn't get how you could conclude that this is AI generated. Is it too good to be human written?
5. 5.▲
   Kvcached: Virtualized, elastic KV cache for LLM serving on shared GPUs(notion.so)

   69 points by Jrxing 1y ago|13 comments