3 ms·

I wanted my students to be able to run open source models. Easiest way was using ollama, and ollama web ui (now open web ui) on google cloud for .18 cents an ho

by mrtimo 3y ago

I wanted my students to be able to run open source models. Easiest way was using ollama, and ollama web ui (now open web ui) on google cloud for .18 cents an hour with a 4-core nvidia and 16 GB of ram on a spot instance. I created a tutorial for my students: https://docs.google.com/document/d/1OpZl4P3d0WKH9XtErUZib5_2EA_QTy50YjJM0kjX1g4/edit#heading=h.2vck2x5a8jz https://docs.google.com/document/d/1OpZl4P3d0WKH9XtErUZib5_2...

BOOSTERHIDROGEN 3y ago
Who good for everyday tasks ?
pknerd 3y ago
Quite expensive. Heard groq API is cheaper?
pknerd 3y ago
Quite costly. I heard that the grow API is cheaper
- pknerd 3y ago
  Well thanks to downvote, I know I misspelled groq, anyway here it is: https://wow.groq.com/ https://wow.groq.com/
internet101010 3y ago
Open WebUI is what I settled on as a centralized hub for LLM queries. It connects to both ollama and OpenAI (via api key) and gives me a single drop down to pick. I can download ollama models from within Open WebUI, which is nice. Built-in LangChain lets people quickly see possibilities and limitations of RAG (spoiler: good for lookups but don't expect to sumif a csv).
- toomuchtodo 3y ago
  Can you share results like ChatGPT with convo uuid link?