3 ms·
This feels like using a harrier jet to take in your groceries. Seems like something that could be built using one of the very small edge models - a 0.3 or 0.5b
by kennywinker 4mo ago
This feels like using a harrier jet to take in your groceries. Seems like something that could be built using one of the very small edge models - a 0.3 or 0.5b maybe with some fine tuning, and run on cpu on device for free without any real quality loss.
- iamsky 4mo agoIt definitely can, it just depends on the complexity of the commands you want it to generate. Most of the time a small local model will do what you want. If you know of a small local model fine tuned to generate shell commands I'm all ears. I usually run it with a 9B qwen model. Pairing this with dictation is fun though, and speeds things up a lot for terminal heavy work.
- kennywinker 4mo agohttps://huggingface.co/kushagragoyal/fine_tuned_model https://huggingface.co/kushagragoyal/fine_tuned_model https://github.com/Eng-Elias/qwen3-600M-terminal-instruct https://github.com/Eng-Elias/qwen3-600M-terminal-instruct https://huggingface.co/ajayk007/Qwen2.5-Coder-1.5B-Shellsmith https://huggingface.co/ajayk007/Qwen2.5-Coder-1.5B-Shellsmit... https://huggingface.co/distil-labs/distil-qwen3-0.6b-SHELLper https://huggingface.co/distil-labs/distil-qwen3-0.6b-SHELLpe... https://huggingface.co/distil-labs/distil-lfm25-shellper https://huggingface.co/distil-labs/distil-lfm25-shellper Those last two compared: https://www.distillabs.ai/blog/fine-tuning-liquids-lfm25-accurate-tool-calling-at-350m-parameters/ https://www.distillabs.ai/blog/fine-tuning-liquids-lfm25-acc... https://www.reddit.com/r/LocalLLaMA/comments/1or1e7p/i_finetuned_gemma_3_1b_for_cli_command/ https://www.reddit.com/r/LocalLLaMA/comments/1or1e7p/i_finet...