3 ms·
Cool! HF Transformers is great for prototyping and research, but should not an interactive tool like this be based on something more speed-focused, like llama.
by akreal 2y ago
Cool!
HF Transformers is great for prototyping and research, but should not an interactive tool like this be based on something more speed-focused, like llama.cpp?
Any plans for languages beyond English?
- juberti 2y agoWe're running it on vLLM and are working with others in the community to bring it to other optimized inference frameworks.