3 ms·
Why attach this to an IDE? I would think that simply letting an LLM control a terminal and make a commit would be the better approach. I recently tried it with
by mg 3y ago
Why attach this to an IDE? I would think that simply letting an LLM control a terminal and make a commit would be the better approach.
I recently tried it with "You are in a directory with a web app that does ... and you want to implement feature .... In each step, you can use ls or any other bash command and I will give you the output".
It was pretty hillarious how the LLM found its way around the codebase with ls, cat, find, grep awk and actually even managed to edit the code that way and do a commit.
Giving the LLM a bit better tools, like a version of "cat" that prefixes the lines with numbers, and a "swap" command that can do "swap 179 250 ..." to swap out the lines 179 to 250 with "..." would probably be enough to empower the LLM to be pretty efficient.
The next step might be to let the LLM manage its context window by allowing it to remove the last output with a command like "forget". So when the LLM does "cat somefile" and realizes that the output is not interesting, it can follow up with "forget" so the output will be replaced with "You deemed the output to be not interesting".
Those tools would probably evolve to make coding and managing the context window more and more efficient. Like "nicecat 100 200" to see the lines 100 to 200 with numbers prefixed. "keep 200 300" to forget the last output except lines 200 to 300 etc.
- deliriumchn 3y ago> Why attach this to an IDE? I would think that simply letting an LLM control a terminal and make a commit would be the better approach. This is a recipy for disaster imo, LLMs are not that good yet to give them the ability to act on their own
- PrimeMcFly 3y agoLet it work on a fork and see what it comes up with.
- jack_pp 3y agoIn thinking that of course you would review the commit and fix it if broken, you don't need to blindly trust it. The commit is just another way to see the code changes better
- hhsectech 3y agoDepends on the model you use. The tiny quantized 7B models, yeah its like having a high functioning teenage intern...but there are bigger more sophisticated models...like Falcon 180B...you need hella RAM to run that bad boy though.
- Lazare 3y agoThis is how https://aider.chat https://aider.chat works; it gives you answers in the form of git commits that include the prompt and a description of what it did. It's an interesting UX.
- dkersten 3y ago> In each step, you can use ls or any other bash command and I will give you the output If you're using the OpenAI API, you can also automate this using the "tools"/"tool_calls" function call feature. https://platform.openai.com/docs/api-reference/chat/create https://platform.openai.com/docs/api-reference/chat/create
- eurekin 3y ago> It was pretty hillarious how the LLM found its way around the codebase with ls, cat, find, grep awk and actually even managed to edit the code that way and do a commit. Please do write more. I'd love to replicate that. I keep coming across applications, where there is more "looking around the code" work than changing it. Even a slight help could mean a lot here!
- Zababa 3y ago> Giving the LLM a bit better tools, like a version of "cat" that prefixes the lines with numbers, and a "swap" command that can do "swap 179 250 ..." to swap out the lines 179 to 250 with "..." would probably be enough to empower the LLM to be pretty efficient. ed is the standard (LLM) editor
- mg 3y agoHow would ed help? Say one of the files is talk.py and there is a function inside of it at line 17 which the LLM wants to change: 17 def hello(): 18 print ("Hello!") My approach would be to provide the LLM with a swap command, which takes filename, first line and last line and then treats the rest of the standard input as the new content: swap talk.py 17 18 def hello(name): print(f"Hello {name}!") My feeling is that this is the most efficient way for an LLM to code.
- nareshshah139 3y agoLooks like the open interpreter project might be what you’re looking for.
- stevage 3y agoYeah I am really waiting for these tools to be proactive. I don't find the idea of asking if there is a but very sensible. It should tell me when it looks like I wrote a bug. Or if it has a suggestion for a refactor.