9 ms·
A practical guide to starting out with Ollama LLM and Open WebUI
- cranberryturkey 2y agoThis is the best setup I've seen so far, only problem is if you want to integrate it into your product you'll need a cloud GPU which can be quite expensive.
- BoredPositron 2y agoYou can use serverless endpoints on runpod or similar services the delay is a bit high for the first inference but for concurrent requests it's fine with a reasonable keep alive.