2 ms·
So you mean firing up ollama and them later if it scales. going for llm service?
by sonderotis 1y ago
So you mean firing up ollama and them later if it scales. going for llm service?
- toomuchtodo 1y agoYes. I have seen examples of servers for developing countries that host local copies of Wikipedia and other open data sets, as well as an aggressive caching proxy for low bandwidth internet uplinks. Same idea, but local serving of LLM inference.