4 ms·
This was a workshop I gave in my https://llm.datasette.io/ https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the t
by simonw 2y ago
This was a workshop I gave in my https://llm.datasette.io/ https://llm.datasette.io/ CLI tool.
What other CLI tools are people using to work with LLMs in the terminal?
There one comment here about https://github.com/paul-gauthier/aider https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickstart https://github.com/ollama/ollama/blob/main/README.md#quickst...
- throwup238 2y ago> What other CLI tools are people using to work with LLMs in the terminal? I use aichat: https://github.com/sigoden/aichat https://github.com/sigoden/aichat I especially like the terminal integration where I can type a natural language request at the terminal and press Alt+E to have it converted to a command to run.
- kelvie 2y agoI also use aichat. For some reason (well, I know the reason, it's shell quoting) I don't like passing prompts in via command line, so having a simple text-based TUI like a readline prompt is nice.
- mikeqq2024 2y agoThere is similar ShellOracle: https://github.com/djcopley/ShellOracle https://github.com/djcopley/ShellOracle It is used as Bash widget, where you press CTL-F, describe your command, then press Enter, the generated command will be inserted into your shell prompt.
- bt1a 2y ago(aichat in tangential comment looks much better) Besides aider and ollama, I think shell_gpt https://github.com/TheR1D/shell_gpt https://github.com/TheR1D/shell_gpt is great for quick chats / lookups. Being able to quickly cat files to repl sessions saves a lot of time. I need to integrate distil whisper large v3, aider, and shell_gpt to tidy up a lot of my disjointed LLM use. As someone else mentioned, the commits created by aider allow me to "skip" some intermediate steps that would be required when working on coding tasks with other frameworks.
- pjot 2y agoopenai's python client includes a cli actually - im not sure if its really documented anywhere. $ openai api chat.completions.create -g 'user' 'say hello' -m 'gpt-3.5-turbo' $ Hello! How can I assist you today?
- evmar 2y agoI am using a hacky one I wrote myself. I looked at llm but it doesn't appear to have a mechanism for multi-shot prompting, where you provide both prompts and responses within your query. (Ref https://platform.openai.com/docs/guides/prompt-engineering/tactic-provide-examples https://platform.openai.com/docs/guides/prompt-engineering/t... .) Maybe take this as a feature request? It feels like the 'template' system in llm might be able to encompass this but the docs don't appear to provide a reference for the yaml format, only examples. I guess that is another feature request, sorry! (BTW if you haven't seen https://docs.divio.com/documentation-system/ https://docs.divio.com/documentation-system/ it really changed how I think about documentation)
- simonw 2y agoYeah, I've been thinking a bit about the multi shot thing. I've had great results from Claude by "faking" the previous conversation to include example question/answer pairs. With LLM you can do that using the undocumented Python Conversation API, but it's undocumented for a reason (I don't think it's good enough yet). You could also fake a previous conversation through the CLI tool but that is VERY undocumented - you would have to write fake rows into the SQLite database! I also want to support the Claude thing where you can prefill the start of the response - amazing for things like forcing HTML by refilling an HTML doctype.
- evmar 2y agoIn case it helps any, here are some more details about what I used it for. (Summary: providing examples of a specific translation task, to make it use a particular API in the output.) https://inuh.net/@evmar/112001414385042731 https://inuh.net/@evmar/112001414385042731 And what I expected of llm is for the template file to (optionally) contain an array of prompt/response pairs. You could even imagine the --save flag constructing one from an ongoing conversation of llm -c maybe.
- simonw 2y agoYeah, doing this with templates is a great idea.
- saulpw 2y agoI've appreciated https://github.com/cthulahoops/chatcli https://github.com/cthulahoops/chatcli which is simple and straightforward.
- nyellin 2y agoMost of the cli tools just wrap an LLM, but don't give it access to the data it needs to be useful. Aider is an exception of course - it gives great results because it feeds the LLM your source files. We built http://github.com/robusta-dev/holmesgpt/ http://github.com/robusta-dev/holmesgpt/ to investigate Prometheus/Jira/PagerDuty issues. We're able to get pretty good results (we benchmark extensively) because we use function-calling to give the LLM read acess to relevant data. I think we're the only open source AIOps tool, and the only AIOps tool period that does something more complex than RAG + summarization.
- snthpy 2y agoI haven't tried it yet but this appeared a few days ago and I'm a big fan of Textual. https://github.com/paulrobello/parllama https://github.com/paulrobello/parllama
- mark_l_watson 2y agoThanks for sharing that, I also like Textual. I will try parllama and compare it to Gollama.
- FergusArgyll 2y agoI made my own python wrapper around the Gemini API, I wanted one feature: there is a default system prompt but -s overrides that with a temporary system prompt. -p is for the prompt, if it is left out, the cli is interactive (one long chat until I exit) but with -p I can use regular linux utils for example: I wrote a short bash script which uses yt-dlp and ffmpeg to download a song, convert it to my preferred format and then uses gemini to add metadata. artist=$(gemini -s "Please respond with the name of the Artist based on the songs title. do not use any other words, just the artist name. example: 'Bruce Springsteen - Old Dan Tucker [S-GHbDFrwlU].opus' Bruce Springsteen" -p "$opus_file" | tr -d '\n')
- Jaydenaus 2y agoI've been using this one: https://github.com/egoist/shell-ask https://github.com/egoist/shell-ask very similar, although the ability to continue a conversation like you can in yours is a killer feature I wish it had.
- lynx23 2y agoI am experimenting with my own tool[1] which is built using asnycio and prompt-toolkit. Made it particularily easy to define function_tools via a decorator which can automagically transform async defs into pydantic models. [1] https://github.com/mlang/ass https://github.com/mlang/ass
- librasteve 2y agoThe raku LLM modules are excellent CLI and Jupyter notebook tools. Suggest you start here https://raku.land/?q=LLM%3A%3APrompts https://raku.land/?q=LLM%3A%3APrompts and review the comprehensive videos and examples from here https://www.reddit.com/r/rakulang/s/W1UqivfFA9 https://www.reddit.com/r/rakulang/s/W1UqivfFA9
- mark_l_watson 2y agoMy workspace for experimenting with LLMs is chaotic: currently about a 100 Python scripts that each have a single purpose. I sometimes use Ollama or Golamma from the command line. Since I am an old Lisp hacker, I also have a large collection of short Racket and Common Lisp LLM experiments. Chaos.
- nbbaier 2y agoYou should use an LLM to classify and consolidate them lol
- deathmonger5000 2y agoI created a CLI tool called Promptr which is an open source developer tool that allows the user to modify their codebase using plain language. The tool sends the user’s query as well as the relevant source code to an LLM. The changes from the LLM are applied directly to the user’s filesystem eliminating the need for copy pasting. Promptr is implemented in Javascript, and it incorporates liquidjs templating so users can build a library of reusable prompt templates for common tasks and contexts. You can find out more here: https://github.com/ferrislucas/promptr https://github.com/ferrislucas/promptr
- josephrmartinez 2y agoA CLI tool for generating tutorials based on the work you recently committed: https://github.com/josephrmartinez/mktute https://github.com/josephrmartinez/mktute npm i mktute You can select between local model (ollama), claude 3.5 sonnet, or gpt-4. I've been surprised to find sonnet much better in performance and price for this task.
- fragmede 2y agoOpen interpreter. Open Interpreter lets LLMs run code on your computer to complete tasks. Eg "fix my python" or "make a movie out of the pictures in this directory." https://github.com/OpenInterpreter/open-interpreter https://github.com/OpenInterpreter/open-interpreter