5 ms·
Autolith: A programming agent with a live runtime
- rufius 2mo agoThis is an interesting idea. I was thinking about something similar in Smalltalk or Erlang the other day. Mostly building on the actor/object+mailbox concept. I didn’t actually do anything with that idea yet but may look at the idea in Elixir this weekend.
- magnusi 2mo agoHi! I was thinking about Smalltalk as well before I made Autolith. I ended up going with Common Lisp because I know Lisp much better (last time I used smalltalk was like, whew, 2014 or so) and because it has better platform support and ecosystem (at least in my experience). I think Elixir could be great, I knew a guy who was trying to do an agent in Elixir, but sadly didn't get far. Keep me posted if you get anywhere! And if you'd like to try Autolith, I am happy to help with issues/questions on our Zulip, haha
- yogthos 2mo agoAmusingly, I started building something very similar in Jolt today :) https://github.com/yogthos/samizdat/ https://github.com/yogthos/samizdat/
- magnusi 2mo agoHi, I'm the author of Autolith! Never heard about Jolt, and I love Chez Scheme, it was my first Lisp! Would you like to come to our Zulip at https://zulip.lambda-symbolics.com https://zulip.lambda-symbolics.com? We can exchange ideas for our harnesses
- yogthos 2mo agoOh yeah sure thing, would be fun to chat and swap ideas. :)
- sroerick 2mo agoHi, thank you both for this. I built my own, worse, harness in my own, worse, lisp and I will also be joining the chat
- yogthos 2mo agoIt's a fun thing to do. I'm really intrigued by the idea of a harness that can tune itself to a particular project. As I've been hacking on samizdat, I realized that you want to have a set of canned workflows as a starter pack. Then the tool makes a copy of that in each project, and keeps adjusting the workflows based on where it gets stuck, or when it thinks of ways to do stuff better. So, each project can evolve in its own way. The key trick is that workflows are represented as state machines which are just data structures. So, the LLM can easily inspect and change them to add or remove behaviors as it sees fit. I have a specific supervisor role whose sole job is to watch how implementer agents are doing and whether they're making progress. When it sees them get stuck, its job is to unblock them.
- magnusi 2mo agoGlad to have you!
- a2ff6eeb0 2mo agoGiven that the most important feature for agent performance is the popularity of the language, ie, the amount of training data, (https://danluu.com/pl-tokens/ https://danluu.com/pl-tokens/), why would you cause problems for yourself by using Lisp rather than Python/Javascript if you care mainly about results fast, or C/C++/Rust if you care about performance too?
- jbott 2mo ago> the most important feature for agent performance is the popularity of the language This is explicitly called out as only weakly supported in that blog post: - You should use a popular language - There's weak support for this statement
- a2ff6eeb0 2mo agoFeel free to do your own analysis -- my informal experiments backs this up, though. I see worse results when I try to do anything in an unpopular language. It makes sense, needing to train the model on things that aren't already in its weighs takes up valuable context. Until we have models that update their weights based on what they've seen in their recent sessions and learn like people, this will be a problem. For now, though, between the results I'm seeing here, and the lack of need to look at code, I think this kills off any reason for me to use less popular languages.
- jauntywundrkind 2mo agoYour attempt was probably pretty, ahem, weak. How much support would you say you gave your goes, before you threw in the towel? I don't think there's really ever a downside to leaning in and making use of a language or system that works for you. Trying to tell people they should just use the popular thing is, imo, bad advice to turn hackers and experimenters into boring people.
- a2ff6eeb0 2mo agoA day or so for each of the oddball languages; again, I'm still waiting for an argument on why there's any value here, since the entire point of an agentic system like this is that I don't have to read the code. Experiment with the AI, sure, but you've got a pretty high burden of proof to show that AI is going to pick it up without a high per-prompt token cost. AI changes the constraints here for now, since it can't permanently learn things. I'm waiting until that changes, but right now it's better to use what it knows out of the box if you want good results. A better language doesn't buy me anything other than performance; the reason to stick an AI in here is to remove interactions with the code. I don't care what the AI chooses to use, as long as it gets results.
- ggangsir 2mo agoHow do differentiate with cursor ?
- magnusi 2mo agoHi, I am the author of Autolith. What do you mean by that? I'd love to give you a good answer, haha
- vivzkestrel 2mo agostupid question: why are so many people obsessed with LISP?
- magnusi 2mo agoIt's the most flexible language there is, while also fairly performant and having great features for extensibility and debugging
- magnusi 2mo agoHello guys, I am the author of Autolith (github.com/luciusmagn), I am pretty new to Hacker News. I will respond to your posts as soon as I can :)
- cc_ashby 2mo agoThis is really interesting. I have felt for the past couple years that the moldability of Common Lisp lends agents the appropriate affordances to “do the right thing” via experimentations. I see there is a section on RLMs; have you ran Autolith via agentic benchmarks? I would love to see comparisons with Prime Agent.
- magnusi 2mo agoWe have not done that yet, it's one of the major priorities. Autolith implements RLM a bit differently than Prime Agent (although it both comes from the same paper): In Autolith, the top level agent is traditional, but has RLM tools which it can use for the things RLM is good at, namely exploratory work, processing a lot of files at once, backward context research and so on.
- eeyzs1 2mo ago[flagged]
- NeutralForest 2mo agovery cool, thanks for making this!
- magnusi 2mo agoThanks for liking it!
- iovrthoughtthis 2mo agoThis webpage is beautiful.
- hsaliak 2mo agoEver since coding agents came out, It felt a little like we were implementing the self improving lisp programs of old. This idea almost gets us there. Could the next step be to make it the program itself?
- magnusi 2mo agoWell, I've had success loading Lisp programs into Autolith and letting it poke at them from the inside, haha!
- rtpg 2mo agoI'm having a bit of trouble reading this over understanding the case where I would rather the agent update itself, vs having the agent write out tools and then call those tools. I rarely find myself thinking "ah I need the agent interface to change". And the vague generality of how agents work play into making it fairly easy to "just" have it rely on some external tooling to do anything special. Maybe this is just a counterargument to the lisp philosophy as a whole but... well... I use Emacs for example and am fine with a model of "an agent can look at my Emacs config" rather than "my agent _is running my Emacs session_".
- magnusi 2mo ago(disclaimer: I made Autolith) Well, having the agent write out tools and call those tools is something I consider to be under self-modification also. But there are other things, when it's useful: - You want the agent to update without restarting a session and e.g. killing child LISP REPLs and long-running sub-agents - There is a bug in the harness that bothers you (Autolith records papercuts and can generally solve them via self-modification) - You want to temporarily or permanently hook into literally any part of the agent lifecycle - You are of a ricing persuassion and want to change how the agent looks (very surface level, but I have seen people do it) - Related to the hooking point, you want to integrate Autolith with something else or make it emit something. We cannot predict all the knobs where you might do this, or how selectively you might do it, but self-modification lets you do it - You want to add support for yourself for non-standard or proprietary/secret providers That's off the top of my head. The secondary benefit is that this is great at developing the agent itself as a project. It can try/triage the changes it's working on, try its own tests, probe at things, and so on. This is why by far, the changes submitted to Autolith developed by Autolith are by far the highest quality out of all clanker-submitted changes.
- mathisfun123 2mo ago> a ricing persuassion wtf does this mean lol
- reikonomusha 2mo ago
- bbor 2mo agoWell I'll be damned if that isn't one of the best sites I've ever seen, with a great ethos. And in Lisp! Inspiring stuff. Godspeed, Lucius :)