4 ms·
Token/second performance has been excellent for me. You can use this to run any if the thousands of of GGUF models on Hugging Face, see note here: https://simo
by simonw 3y ago
Token/second performance has been excellent for me.
You can use this to run any if the thousands of of GGUF models on Hugging Face, see note here: https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-trying-other-models https://simonwillison.net/2023/Nov/29/llamafile/#llamafile-t...