3 ms·
> Ollama is definitely the easiest way to run LLMs locally Nitro outstripped them, 3 MB executable with OpenAI HTTP server and persistent model load
by refulgentis 3y ago
> Ollama is definitely the easiest way to run LLMs locally
Nitro outstripped them, 3 MB executable with OpenAI HTTP server and persistent model load
- jmorgan 3y agoPersistent model loading will be possible with: https://github.com/ollama/ollama/pull/2146 https://github.com/ollama/ollama/pull/2146 – sorry it isn't yet! More to come on filesize and API improvements
- akulbe 3y agoI just wanted to say thank you for being communicative and approachable and nice.
- evantbyrne 3y agoWho cares about executable size when the models are measured in gigabytes lol. I would prefer a Go/Node/Python/etc server for a HTTP service even at 10x the size over some guy's bespoke c++ any day of the week. Also, measuring the size of an executable after zipping is a nonsense benchmark in of itself
- refulgentis 3y agoNot some guy, agree on zip, disagree entirely with tone of the comment (what exactly separates ollama from those same exact hyperbolic descriptions?)