6 ms·Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Pythonby crowwork 2y agoScale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python