4 ms·
Hmm.. this might make it feasible to build something like a command line program where you can optionally just specify the arguments in natural language. Althou
by ilaksh 5mo ago
Hmm.. this might make it feasible to build something like a command line program where you can optionally just specify the arguments in natural language. Although I know people will object to including an extra 14 MB and the computation for "parsing" and it could be pretty bad if everyone started doing that.
But it's really interesting to me that that may be possible now. You can include a fine-tuned model that understands how to use your program.
E.g. `> toolcli what can you do` runs `toolcli --help summary`, `toolcli add tom to teamfutz group` = `toolcli --gadd teamfutz tom`
- HenryNdubuaku 5mo agoSo Needle is trained for INT4, what you see in the playground is INT4, only 14MB, same challenge though.
- ilaksh 5mo agoOh gotcha. Fixed my comment.
- matchaonmuffins 5mo agoI wonder if it will make sense to have a central natural language parser of sorts managed by the OS that includes a model like this? So all command line programs can hook into this model at runtime, and you can have adaptors for fine-tuning etc.
- ilaksh 5mo agoYes I assume some kind of LLM system will eventually make its way into OS package registries and eventually there will be a standard of some sort. Already OpenAI's API is the de facto standard available from all providers. But who knows what port it's on. Maybe implement mDNS/DNS-SD or SSDP, and if not there it could just go through default ports like the Ollama one and check for a certain endpoint.
- ainch 5mo agoApple already have this working on iOS - as discussed in this recent post. https://unix.foo/posts/local-ai-needs-to-be-norm/ https://unix.foo/posts/local-ai-needs-to-be-norm/
- sellmesoap 5mo agosounds loke a job for llamafile! https://github.com/mozilla-ai/llamafile https://github.com/mozilla-ai/llamafile