4 ms·
When I'm writing a script that requires some kind of call to an LLM, I use this: https://github.com/simonw/llm https://github.com/simonw/llm. This is of course
by lcian 9mo ago
When I'm writing a script that requires some kind of call to an LLM, I use this: https://github.com/simonw/llm https://github.com/simonw/llm.
This is of course cross-platform and works with both models accessible through an API and local ones.
I'm afraid this might not solve your problem though, as this is not an out of the box solution, it requires the user to either provide their own API key or to install Ollama and wire it up on their own.
- kristopolous 9mo agoI've been working on a more unixy version of his tool I call llcat. Composable, stateless, agnostic, and generic: https://github.com/day50-dev/llcat https://github.com/day50-dev/llcat It might help things get closer.. It's under 2 days old and it's already really fundamentally changing how I do things. Also for edge running look into the LFM 2.5 class of models: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct
- mirror_neuron 9mo agoI love this concept. Looks great, I will definitely check it out.
- kristopolous 9mo agoPlease use it and give me feedback. I'm going to give a lightning talk on it tonight at sfvlug
- nvader 9mo agoI think this is definitely a step in the right direction, and is exactly the kind of answer I was looking for. Thank you! `llm` gives my tool a standard bin to call to invoke completions, and configuring and managing it is the user's responsibility. If more tools started expecting something like this, it could become a defacto standard. Then maybe the OS would begin to provide it.