3 ms·
Yes. TGI is Huggingface's version of LLVM (some nuance, of course). LLVM also launched grammar support recently too, so we'll be looking into it.
by edunteman 3y ago
Yes. TGI is Huggingface's version of LLVM (some nuance, of course). LLVM also launched grammar support recently too, so we'll be looking into it.
- memothon 3y agoAll of these acronyms are so confusing. I'm assuming LLVM isn't the compiler tool, but searching "LLVM ai" doesn't give me any good results.
- reliableturing 3y agoThey probably meant vLLM https://docs.vllm.ai/en/latest/ https://docs.vllm.ai/en/latest/
- edunteman 3y agoah shoot, yes I meant vLLM, sorry for the confusion, lots of comments to reply to :)