4 ms·
> This Space has been paused by its owner. Looks interesting but I'd like to see how well it does with my prompt.
by ganzuul 3y ago
> This Space has been paused by its owner.
Looks interesting but I'd like to see how well it does with my prompt.
- dmezzetti 3y agoThis is a small model that will easily run on most modern machines. There are many ways to run this either using Hugging Face Transformers directly or a library built on Transformers that makes it easier. For example, you can run LLM inference with txtai (https://github.com/neuml/txtai https://github.com/neuml/txtai) in Python using the following 3 lines of code. from txtai.pipeline import LLM llm = LLM("TinyLlama/TinyLlama-1.1B-Chat-v1.0") llm("""<|im_start|>user What is the speed of light?<|im_end|> <|im_start|>assistant""") I expect they'll be GGUF and AWQ versions of this model soon that will make inference possible on even smaller devices.
- ganzuul 3y agoThank you but I'm currently on a tablet and can't access my workstation.