8 ms·
Coding agent in 94 lines of Ruby
- rbitar 1y agoRubyLLM has been a joy to work with so nice to see it’s being used here. This project is also great and will make it easier to build an agent that can fetch data outside of the codebase for context and/or experiment with different system prompts. I’ve been a personal fan of claude code but this will be fun to work with
- radanskoric 1y agoAuthor here. The code I made took me 3 hours (including getting up to speed on RubyLLM). I also intentionally DIDN'T use a coding assistant to write it (although I use Windsurf in my regular work). :D It's clearly not a full featured agent but the code is here and it's a nice starting point for a prototype: https://github.com/radanskoric/coding_agent https://github.com/radanskoric/coding_agent My best hope for it is that people will use it to experiment with their own ideas. So if you like it, please feel free to fork it. :)
- RangerScience 1y agoThis is very cool, somewhat inspiring, and (personally) very informative: I didn't actually know what "agentic" AI use was, but this did an excellent job (incidentally!) explaining it. Might poke around... What makes something a good potential tool, if the shell command can (technically) can do anything - like running tests? (or it is just the things requiring user permission vs not?)
- tough 1y ago> What makes something a good potential tool, if the shell command can (technically) can do anything - like running tests? Think of it as -semantic- wrappers so the LLM can -decide- what action to take at any given moment given its context, the user prompt, and available tools names and descriptions. creating wrappers for the most used basic tools even if they all pipe to terminal unix commands can be useful. also giving it speicif knowledge base it can consult on demand like a wiki of its own stack etc
- notpushkin 1y agoAlso it’s safer than just giving unrestricted shell access to an LLM.
- tough 1y agothat too, ideally autonomous agents will be only spawnrd in their own secure environments using docker or vm’s or posix / unix security but yeah
- radanskoric 1y agoThanks, sharing my learnings on how coding agents work was my main intention with the article. Personally I was a bit surprised by how much of the "magic" is coming directly from the underlying LLM. The shell command can run anything really. When I tested it, it asked me multiple times to run the tests and then I could see it fixing the tests in iterations. Very interesting to observe. If I was to improve this to be a better Ruby agent (which I don't plan to do, at least not yet), I would probably try adding some Rspec/Minitest specific tools that would parse the response and present it back to the LLM in a cleaned up format.
- elif 1y agoWhy stop there? Give it a capybara tool and make it a full TDD agent
- radanskoric 1y agoThat's a very neat idea, maybe even add something like browser-use to allow it to implement a Rails app and try it out automatically. I think you should try it. :) I'm being serious. This sounds like a fun project but I have to turn my attention to other projects for the near future. This was more of an experiment for me, but it would be cool to see someone try out that idea.
- RangerScience 1y agoDo you know of examples of other agents with more defined tools, to use as inspiration/etc? (Like - what would it look like to clean up test results for an LLM?)
- deleted 1y ago[deleted]
- fullstackwife 1y agoThis reminds me about PHP hello world programs which would take a string from GET, use it as a path, read a file from this path, and return the content in the response. You could make a website while not using any knowledge about websites. Agents are the new PHP scripts!
- zeckalpha 1y agoRCE as a service!
- behnamoh 1y ago[flagged]
- tptacek 1y agoYou could chuck RubyLLM and still golf it down pretty far. Nothing RubyLLM is doing is magic. Read the linked article that inspired it (which is in Go); there's even less magic there. Some of the complaints here are just about Ruby syntax, which: shrug.
- nomilk 1y ago> def execute(path:) > why the "incomplete" colon? Just ruby's syntax to denote 'keyword arguments', i.e. when calling this method, you must explicitly use the parameter name, like so: execute(path: "/some/path") > class ListFiles < RubyLLM::Tool > what's with the "<"? Just ruby's way of denoting inheritance i.e. class A < B means class A inherits from B > param :command, desc: "The command to execute" > again, the colons are confusing... The colons can be confusing, but you get used to them. The first one in :command is denoting a symbol. You could think of it as just a string. Any method that expects a symbol could be rewritten to uses a string instead, but symbols are a tad more elegant (one less keypress). The second one, desc:, is a keyword argument being passed to the param method. You could rewrite it as param(:command, desc: "The command to execute") I sometimes wonder what ruby would be like without symbols, and part of me thinks the tradeoff would be worth it (less elegant, since there'd be "strings" everywhere instead of :symbols, but it would be a bit easer to learn/read.
- deleted 1y ago[deleted]
- d4mi3n 1y agoAnother couple important aspects of symbols: 1. They refer to the same object in memory, so multiple usages of a symbol by the same name (e.g. `:param`) not not add additional memory overhead. 2. Symbols are also immutable in that they cannot be mutated in any way once they are referenced/created. These properties can be useful in some contexts, but in practice they're effectively used as immutable constant values.
- paulddraper 1y ago[flagged]
- dismalaf 1y agoUnless you're writing a kernel from scratch, you're calling lots of code someone else wrote once upon a time...
- radanskoric 1y agoEven if you're writing a kernel, you're using a compiler. And if you're writing a compiler it's probably bootstrapped and built by itself, so you're still using a mountain of code some other people wrote. :)
- CyberDildonics 1y agoIf you do that your program probably does more than just call what you're saying the program does. This is like someone saying they wrote a one line memory allocator and calling mmap or saying you wrote a one line video converter that just calls ffmpeg.
- paulddraper 1y agoNaturally. But I wouldn't necessary expect "Ruby" to be "Ruby plus 5k lines of more Ruby."
- jeffrwells 1y agoLOC or code golf is indeed not a metric to optimize to 0, but hating on importing libraries is a bad take. As CTO at Redo do you demand everything written in assembly? Does your entire company’s code run in one file or do you use abstraction to simplify? I’m not quite clear on how expressing a really complex and paradigm shifting approach of agents in a more concise way is a bad thing
- paulddraper 1y agoOf course not. Nor did I think that "94 lines of Ruby" was assembly. But "94 lines of Ruby" is actually "94 lines of Ruby plus 100x other lines of Ruby plus Anthropic." The high level form is indeed 94 lines. I think it could be 15 if you are going for concepts.
- Mystery-Machine 1y agoJust out of curiosity, I never understood why people do `ENV.fetch("ANTHROPIC_API_KEY", nil)` which is the equivalent of `ENV["ANTHROPIC_API_KEY"]`. I thought the whole point of calling `.fetch` was to "fail fast". Instead of assigning `nil` as default and having `NoMethodError: undefined method 'xxx' for nil` somewhere random down the line, you could fail on the actual line where a required (not optional) ENV var wasn't found. Can someone please explain?
- jaredsohn 1y agoThere might be code later that says that if the anthropic api key is not set, then turn off the LLM feature. Wouldn't make sense for this LLM-related code but the concept makes sense for using various APIs from dev.
- riffraff 1y agoBut if you do ENV[xxx] the value is also set to nil. Using .fetch with a default of nil is what's arguably not very useful. IMO it's just a robocop rule to use .fetch, which is useful in general for exploding on missing configuration but not useful if a missing value is handled.
- radanskoric 1y agoAuthor here. You're actually right here. I took the code from RubyLLM configuration documentation. If you're pulling in a lot of config options and some have default values then there's value in symmetry. Using fetch with nil communicates clearly "This config, unlike those others, has no default value". But in my case, that benefit is not there so I think I'll change it to your suggestion when I touch the code again.
- zeckalpha 1y agoThis may be a Pythonism, where the exception raising convention is inverse of Ruby's. {}["key"] # KeyError in Python
- sagarpatil 1y agoI don’t understand the hype in the original post. OpenAI launched function calls two years ago and it was always possible to create a simple coding agent.
- radanskoric 1y agoAuthor here. The part about coding agents that wasn't clear to me was how much of the "magic" is in the underlying LLM and how much in the code around it making it into an agent. When I realised that it's mostly in the LLM I found that a bit surprising. Also, since I'm not an AI Engineer, I was happy to realise that my "regular programming" skills would be enough if I wanted to build a coding agent. It sounds like you were aware of that for a while now, but I and a lot of other people weren't. :) That was my motivation for writing the article.
- ColinEberhardt 1y agoGreat post, thanks for sharing. I wrote something similar a couple of years ago, showing just how simple it is to work with LLMs directly rather than through LangChain, adding tool use etc … https://blog.scottlogic.com/2023/05/04/langchain-mini.html https://blog.scottlogic.com/2023/05/04/langchain-mini.html It is of course quite out of date now as LLMs have native tool use APIs. However, it proves a similar point to yours, in most applications 99% of the power is within the LLM. The rest is often just simple plumbing.
- radanskoric 1y agoThanks for sharing this. The field moves so yes, it's out of date, but it's useful to see how the tools concept evolved. Especially since I wasn't paying attention at that area of development back when you wrote your article. Very interesting.
- thih9 1y ago> Claude is trained to recognise the tool format and to respond in a specific format. Does that mean that it wouldn’t work with other LLMs? E.g. I run Qwen3-14B locally; would that or any other model similar in size work?
- simonw 1y agoQwen3 was trained for tool usage too. Most models are these days. https://qwenlm.github.io/blog/qwen3/#agentic-usages https://qwenlm.github.io/blog/qwen3/#agentic-usages
- radanskoric 1y agoIt would work with most other Tool enabled LLMs. RubyLLM abstracts away the format. Some will work better than the others, depending on the provider, but almost all have tool support. Claude is just an example. I pulled the actual payloads by looking at what is actually being sent to Claude and what it is responding. It might vary slightly for other providers. I used Clause because I already had a key ready from trying it out before.
- thih9 1y ago> return { error: "User declined to execute the command" } I wonder if AIs that receive this information within their prompt might try to change the user’s mind as part of reaching their objective. Perhaps even in a dishonest way. To be safe I’d write “error: Command cannot be executed at the time”, or “error: Authentication failure”. Unless you control the training set; or don’t care about the result. Interesting times.
- radanskoric 1y agoIf a certain user is susceptible to having the LLM convince them to run an unsafe command, I fear we can't fix that by trying to trick the LLM. :D Either the user needs to be educated or we need to restrict what the user themselves can do.
- johnisgood 1y agoI am leaning towards the former. Please let us have nice things despite the people unwilling to learn.
- radanskoric 1y agoWhy are people always the reason why we can't have nice things... :D
- johnisgood 1y agoSide-note: I do not understand the inclusion of "N lines of X". You import a library, which presumably consists of many lines. I do not see the point. It would be true that this is only 94 lines of Ruby if and only if there was no "require "ruby_llm/tool"" at the top.
- monooso 1y agoGiven that this post is a response to an article about achieving the same in "N lines of Go" (also using a library), it seems like an appropriate title.
- johnisgood 1y agoThe original post uses "github.com/anthropics/anthropic-sdk-go", the Ruby uses a different library, does it not? If they are two different libraries, then the comparison does not make too much sense.
- radanskoric 1y agoI didn't put the number into the title to make it a competition. LoC is a poor metric. I put it to communicate to the reader that they won't have to spend a lot of time reading the article to get a full understanding. I always put extra effort into trying to make my blog posts shorter without sacrificing the quality. I think good technical writing should transfer the knowledge while requesting the least amount of time possible from the reader.
- johnisgood 1y agoI know, I was only responding to the comment, I was not trying to claim that you are attempting to make it a competition, my bad if it came across as such.
- radanskoric 1y agoIt's good that you commented. I see more than a few people are getting caught up on the number of lines so it's good that I clarify.
- melvinroest 1y agoThe way I'd create extra functionality is to give command-line access with a permission step in between. I'd then create a folder of useful scripts and give it permission to execute those. You can make it much more than just a coding agent. I personally use my personal LLMs for data analysis by integrating it with some APIs. These type of LLM systems are basically acting as a frontend now that respond to very fuzzy user input. Such an LLM can reach out to your own defined functions (aka a backend). The app space that I think is interesting and that I'm working on is creating these systems combined with some solid data creating advicing/coaching/recommendation systems. If you want some input on building something like that, my email is in my profile. Currently I'm playing around with an LLM chat interface with database access that gives study advice based on: * HEXACO data (personality) * Motivational data (self-determination theory) * ESCO data (skills data) * Descriptions of study programs described in ESCO data If you want to chat about creating these systems, my email is in my profile. I'm currently also looking for freelance opportunities based on things like this as I think there are many LLM applications to which we've only scratched the surface.
- elif 1y agoThank you for showing off why ruby is useful not just in the current year, but particular to the current time and AI situation. When you're dealing with code written with hallucinations, you want an easy to understand quickly language (of which ruby is S tier) where out of place behavior cannot hide in code so repetitive and unnecessary that your mind tries to skip over it.
- radanskoric 1y agoThat's an excellent point. Code was always read more than written. With AI it shifts even more towards reading so language readability becomes even more important. And Ruby really shines there.
- dontlaugh 1y agoRuby, the language famous for lots of difficult to understand runtime “magic”?
- radanskoric 1y agoIt's a sharp knife. You can create a messy nightmare or a clean super readable codebase, it's up to how good the author is.
- mattbrewsbytes 1y agoWow, so that RubyLLM gem makes writing an agent more about basic IO operations. I have somehow thought there needed to be deep understanding of LLMs and/or AI APIs to build things like this where I would need to research and read a lot of docs, stay up to date on the endless updates the various AI systems have, etc. The example from the article is about files and directories, this same concept could apply to any text inputs, like data out of a Rails app.
- radanskoric 1y agoThat was my misunderstanding as well. That's why I wrote the article. Btw, it's not even about the RubyLLM gem. The gem abstracts away the calling of various LLM providers and gives a very clean and easy to use interface. But it's not what gives the "agentic magic". The magic is pretty much all in the underlying LLMs. Seeing all the claims made by some closed source agent products (remember the "world's first AI software engineer"?) I thought that a fair amount of AI innovation is in the agent tool itself. So I was surprised when I realised that almost all of the "magic" parts are coming from the underlying LLM. It's also kind of nice because it means that if you wanted to work on an agent product you can do that even if you're not an AI specialised engineer (like I am not).