11 ms·

Go is a good fit for agents

236 points by abelanger 1y ago

the_arun 1y ago
I have same bunch of reasoning for Java. The concept of Notebooks won the hearts of developers I guess.
behnamoh 1y ago
> concurrency by that logic Elixir is even better for agents. also the link at the bottom of the page is pretty much why I ditched Go: https://go.dev/blog/error-syntax https://go.dev/blog/error-syntax The AI landscape moves so fast, and this conservative, backwards looking mindset of the new Go dev team doesn't match the forward looking LLM engineering mindset.
- jclulow 1y ago
  I don't use either Go or LLMs, but isn't the point of LLMs that they write the tedious boilerplate for you? What's the value in a small syntactic improvement if the computer is generating it all anyway?
  - behnamoh 1y ago
    Elixir's concurrency model is fundamentally different than Go's; it's not just syntax difference.
- catlover76 1y ago
  [dead]
- kamikaz1k 1y ago
  by your logic, only considering part of the argument is as good as considering the entire argument.
- guywithahat 1y ago
  Well elixir doesn't produce goroutines (managed threads), they produce "lightweight processes" which have isolated memory. These are more expensive to grow and aren't as easy to share data between one another, although they're much more fault tolerant as a result. It could be better however the underlying concurrency model in Elixir is relatively unique
  - regularfry 1y ago
    It's been a while since I was in the weeds on this, but if I remember correctly they're strictly speaking mostly isolated. Binaries above a certain size share storage between processes, so moving big blobs between processes is cheap.
    - guywithahat 1y ago
      They also have their own system to share data between processes, although I haven't used it. Generally though it's a unique tool that's not always interchangeable with Go
- bheadmaster 1y ago
  > also the link at the bottom of the page is pretty much why I ditched Go: https://go.dev/blog/error-syntax https://go.dev/blog/error-syntax Funnily, it's also one of the reasons I stay with Go. Error handling is the most contraversial Go topic, with half the people saying it's terrible and needs a new syntax, and half saying it's perfect and adding any more syntax will ruin it.
- sorentwo 1y ago
  Absolutely! Elixir's lightweight processes and distribution story make it ideal for orchestration, and that includes orchestrating LLMs. Shameless plug, but that's what many people have been using Oban Pro's Workflows for recently, and something we demonstrated in our "Cascading Workflows" article: https://oban.pro/articles/weaving-stories-with-cascading-workflows https://oban.pro/articles/weaving-stories-with-cascading-wor... Unlike hatchet, it actually runs locally, in your own application as well.
- jerf 1y ago
  If it had a larger learning base, quite possibly. Erlang possibly even more so. The argument that pure code is generally safer to vibe code is compelling to me. (Elixir's purity is rather complicated to describe, Erlang's much more obvious and clear.) It's easier to analyze that this bit of code doesn't reach out and break something else along the way. Though it would be nice to have a language popular enough for the LLMs to work well on, that was pure, but that was also fast. At the moment writing in pure code means taking a fairly substantial performance hit, and I'm not talking about the O(n log n) algorithm slowdowns, I mean just normal performance.
- tptacek 1y ago
  Elixir is great for agents.
- achileas 1y ago
  This makes me want to build agents in Elixir now
tolerance 1y ago
As a functionally-code-illiterate-vibe-coder, I can confirm that LLMs are good at writing Go code.
- dpe82 1y ago
  As a highly-literate developer with almost 30 years of experience, I can also confirm that LLMs are very good at writing Go.
  - crabmusket 1y ago
    "The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt." – Rob Pike This fits LLMs pretty well too it seems!
    - dpe82 1y ago
      That's my assumption too: it's a simple language with very good tooling that's used by reasonably serious projects with good code quality. As a result the LLMs have both high quality training data and the output is easier to get right.
odyssey7 1y ago
AI engineers will literally invent a new universe before they touch JavaScript. The death knell for variety in AI languages was when Google rug-pulled TensorFlow for Swift.
- dpe82 1y ago
  Avoiding JavaScript like the plague that it is, is not unique to AI engineers. -Someone who has written a ton of JS over the past... almost 30 years now.
  - dpkirchner 1y ago
    Choosing Python over JavaScript is one of the more perplexing decisions I've seen.
    - jpk 1y ago
      It's not so perplexing when you understand that Python has long had the best ecosystem of libraries for data science and ML, from which the current wave of AI stuff was born. There are plenty of reasons to dunk on Python, but the reality is lots of people were getting real work done with it in the run up to where we are today.
      - odyssey7 1y ago
        There are choices at multiple levels. Yes, today’s ML engineer has practically no choice but to use Python, in a variety of settings, if they want to be able to work with others, access the labor market without it being an uphill battle, and most especially if they want to study AI / ML at a university. But there were also the choices to initially build out that ecosystem in Python and to always teach AI / ML in Python. They made sense logistically, since universities largely only teach Python, so it was a lowest-common-denominator language that allowed the universities to give AI / ML research opportunities to everyone, with absolutely no gatekeeping and with a steadfast spirit of friendly inclusion (sorry, couldn’t resist the sarcastic tangent). I can’t blame them for working with what they had. But now that the techniques have grown up and graduated to form multibillion-dollar companies, I’m hopeful that industry will take up the mantle to develop an ecosystem that’s better suited for production and for modern software engineering.
guywithahat 1y ago
I wish we had better concurrency models in the ML world. I tried doing some ML in Go a few months back and it's basically impossible; there's just no library support and doing anything requires a gRPC call or a wrapper. Python has limitations and C++ has a tendency to make everything too verbose.
kamikaz1k 1y ago
> High concurrency > Share memory by communicating > Centralized cancellation mechanism with context.Context > Expansive standard library > Profiling > Bonus: LLMs are good at writing Go code I think profiling is probably the lowest value good here, but would be willing to hear out stories of AI middleware applications that found value in that. Cancelling tasks is probably the highest value good here, but I think the contending runtimes (TS/Python) all prefer using 3P libraries to handle this kind of stuff, so probably not the biggest deal. Being able to write good Go code is pretty cool though; I don't write enough to make a judgement there.
- eikenberry 1y ago
  > Bonus: LLMs are good at writing Go code Good at writing bad code. But most of the code in the wild is written by mid-level devs, without guidance and on short timelines.. i.e bad code. But this is a problem with all languages, not just Go.
jasonthorsness 1y ago
Go is great for command-line tools because of library support and fast-starting single-binaries. While most of the benefits in the article are also shared with JavaScript, I wonder if the CLI advantage will help and whether command-line agents will become a thing ("grepllm"?) The language of agents doesn't matter much in the long run as it's just a thin shell of tool definitions and API calls to the backing LLM.
skybrian 1y ago
For long-running, expensive processes that do a lot of waiting, a downside is that if you kill the process running the goroutine, you lose all your work. It might be better to serialize state to a database while waiting? But this adds a lot of complexity and I don’t know any languages that make it easy to write this sort of checkpoint-based state machine.
- Karrot_Kream 1y ago
  Temporal is pretty decent at checkpointing long-running processes and is language agnostic.
- sorentwo 1y ago
  That's the issue with goroutines, threads, or any long running chain of processes. The tasks must be broken up into atomic chunks, and the state has to be serialized in some way. That allows failures to be retried, errors to be examined, results to be referenced later, and the whole thing to be distributed between multiple nodes. It must in my view at least, as that's how Oban (https://github.com/oban-bg/oban https://github.com/oban-bg/oban) in Elixir models this kind of problem. Full disclosure, I'm an author and maintainer of the project. It's Elixir specific, but this article emphasizes the importance of async task persistence: https://oban.pro/articles/oban-starts-where-tasks-end https://oban.pro/articles/oban-starts-where-tasks-end
- ashishb 1y ago
  > For long-running, expensive processes that do a lot of waiting, a downside is that if you kill the goroutine, you lose all your work. This is true regardless of the language. I always do a reasonable amount of work (milliseconds to up to a few seconds) worth of work in a Go routine every time. Anything more and your web service is not as stateless as it should be.
- abelanger 1y ago
  OP here - this type of "checkpoint-based state machine" is exactly what platforms which offer durable execution primitives like Hatchet (https://hatchet.run/ https://hatchet.run/) and Temporal (https://temporal.io/ https://temporal.io/) are offering. Disclaimer: am a founder of Hatchet. These platforms store an event history of the functions which have run as part of the same workflow, and automatically replay those when your function gets interrupted. I imagine synchronizing memory contents at the language level would be much more overhead than synchronizing at the output level.
awinter-py 1y ago
oh my god these things run on a gpu don't they? they have nothing to do with golang? to the extent they run on a cpu they're heavy; we're not like solving the c10k problem with agents
- achileas 1y ago
  Agents don't typically, and any LLM they're calling is likely hosted remotely.
_QrE 1y ago
I'm not sure how valid most of these points are. A lot of the latency in an agentic system is going to be the calls to the LLM(s). From the article: """ Agents typically have a number of shared characteristics when they start to scale (read: have actual users): They are long-running — anywhere from seconds to minutes to hours. Each execution is expensive — not just the LLM calls, but the nature of the agent is to replace something that would typically require a human operator. Development environments, browser infrastructure, large document processing — these all cost $$$. They often involve input from a user (or another agent!) at some point in their execution cycle. They spend a lot of time awaiting i/o or a human. """ No. 1 doesn't really point to one language over another, and all the rest show that execution speed and server-side efficiency is not very relevant. People ask agents a question and do something else while the agent works. If the agent takes a couple seconds longer because you've written it in Python, I doubt that anyone would care (in the majority of cases at least). I'd argue Python is a better fit for agents, mostly because of the mountain of AI-related libraries and support that it has. > Contrast this with Python: library developers need to think about asyncio, multithreading, multiprocessing, eventlet, gevent, and some other patterns... Agents aren't that hard to make work, and you can get most of the use (and paying users) without optimizing every last thing. And besides, the mountain of support you have for whatever workflow you're building means that someone has probably already tried building at least part of what you're working on, so you don't have to go in blind.
- tptacek 1y ago
  That's true from a performance perspective but, in building an agent in Go, I was thankful that I had extremely well-worn patterns to manage concurrency, backlogs, and backpressure given that most interactions will involve one or more transactions with a remote service that takes several seconds to respond. (I think you can effectively write an agent in any language and I think Javascript is probably the most popular choice. Now, generating code, regardless of whether it's an agent or a CLI tool or a server --- there, I think Go and LLM have a particularly nice chemistry.)
- philwelch 1y ago
  Go still has a much better concurrency story. It’s also much less of a headache to deploy since all you need to deploy is a static binary and not a whole bespoke Python runtime with every pip dependency.
danenania 1y ago
I built Plandex[1] (open source CLI coding agent focused on large projects and tasks) in Go and I’ve been very happy with that decision. Beneath all the jargon, it’s good to remember that an “agent” is ultimately just a bunch of http requests and streams that need to be coordinated—some serially and some concurrently. And while that sounds pretty simple at a high level, there are many subtle details to pay attention to if you want to make this kind of system robust and scalable. Timeouts, retries, cancellation, error handling, thread pools, thread safety, and so on. This stuff is Go’s bread and butter. It’s exactly what it was designed for. It’s not going to get you an MVP quite as fast as node or python, but as the codebase grows and edge cases accumulate, the advantages of Go become more and more noticeable. 1 - https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex
deleted 1y ago
[deleted]
npalli 1y ago
Agents to do what? Take ML/AI, all the infra and tools are Python/C++ so what exactly is the Agent going to help you with Go? Many such domains- gaming, HFT, HPC, Scientific Computing, Systems, UX, Enterprise etc. etc. Seems it really helps Go's sweet spot - CLI's and Networking services.
- tptacek 1y ago
  Agents mostly don't run ML/AI code; they're a structured loop around LLM calls and exist mostly to give an LLM access to local tools in some application domain; think "reading my email for me" rather than "driving ML systems".
  - achileas 1y ago
    That doesn't negate what OP was saying at all, the better support in Python isn't for "running ML/AI code" but in things like agent frameworks, observability tools, SDKs, etc. None of which directly run AI code but are still helpful/necessary and for the most part better represented (and supported) in the Python world, although that seems like it's slowly changing.
    - tptacek 1y ago
      There are agent frameworks in most languages at this point so the question just comes down to "can you invoke tools for the problem you want to solve in that language". Yes, Python is really great at that. So is Go and Javascript. I think I'd condense this out to "this is not a really important deciding factor in what language you choose for your agent". If you know you need something you can only get in Python, you'll write the agent in Python.
InTheArena 1y ago
I don't think I agree with the Go is good in LLMs. But outside of that - ML in go is basically impossible. Trying to integrate with the outside ecosystem of go is really difficult - and my experience has been that Claude Code is far less effective with Go then it is with Python, or even Swift. I ditched a project I was writing in Go and replaced it with Swift (this was mostly prompt based anyways). It was remarkably how much better the first pass of the code generation was.
- hoppp 1y ago
  What if... hear me out... You learn to write code instead of generating it...there is a drastic improvement in code quality if you can actually write it.
pantsforbirds 1y ago
I've been messing around with an Elixir + BEAM based agent framework. I think a mixture of BEAM + SQLite is about as good as you can get for agents right now. You can safely swap out agents without redeploying the application, the concurrency is way below the scale BEAM was built for, and creating stateful or ephemeral agents is incredibly easy. My plan is to set up a base agent in Python, Typescript, and Rust using MCP servers to allow users to write more complex agents in their preferred programming language too.
- nilslice 1y ago
  you should check out the Extism[0] project and the Elixir SDK[1]. This would allow you to write the core services, routing, message passing, etc in Elixir, and leverage all the BEAM/OTP have to offer, and then embed "agents" written in other languages which are small Wasm modules that act like in-process plugins. [0]: https://github.com/extism/extism https://github.com/extism/extism [1]: https://github.com/extism/elixir-sdk https://github.com/extism/elixir-sdk
  - pantsforbirds 1y ago
    That's a really interesting idea. My original thought was to use MCP as the way to define other agents, but I'll have to do some more research into extism!
- alberth 1y ago
  Any reason for SQLite use, instead of the BEAMs built-in mnesia data store? https://www.erlang.org/doc/apps/mnesia/mnesia.html https://www.erlang.org/doc/apps/mnesia/mnesia.html
  - pantsforbirds 1y ago
    I'm still in the exploration/experimentation stage of the project, but I'm currently using a mixture of SQLite, PostgreSQL, S3, and DuckDB. My original thought was to spin up SQLite databases as needed because they are super lightweight, well-tested, and supported by almost every programming language. If you want to set up an agent in another programming language via MCP, but you still want to be able to access the agent memory directly, you can use the same schema in a SQLite database. I may end up using mnesia for more metadata or system-oriented data storage though. It's very well designed imo. But one of the biggest reasons has just been the really nice integration with DuckDB. I can query all of the SQLite databases persisted in a directory and aggregate some metadata really easily.
segmondy 1y ago
go is terrible for agents IMO great for agents? lisp and prolog.
- esafak 1y ago
  Not without a supporting ecosystem!
hoppp 1y ago
Go is a good fit for many use-cases.
rpep 1y ago
In practice the library ecosystem is just way behind Python. Maybe after you’re trying to optimise once you’ve worked out how to do stuff, but even the Langchain Go port is wayyyyy behind.
carsoon 1y ago
I wrote the start to an agent library in Go. Its quite rough as most of it was implemented through using AI but I had a lot of ideas through planning/building it. 1. If you make your agents/workflows serializable you can run/load them from a config file or add/remove them from a decoupled frontend. You can also hash them to make versioning easy to track/immutable. 2. If you decouple the stateful object from the agent/workflow object you can just store that through sufficient logging then you can rebuild any flow at any state and have branching by allowing traces to build on one another. You can also restart/rerun a flow starting at any location. 3. You can allow for serializable tools by having a standard HttpRequestTool then setup cloudflare workers/any external endpoints for the actual toolcall logic. Removing primary server load and making it possible to add/remove tools without rebuilding/restarting. Given this system in golang you can have a single server which supports tens of thousands of concurrent agent workflows. The biggest problem is there isn't that many people who are working on it. So even if you can make agents 100x more efficient by running in Go it doesn't really matter if cost isn't the biggest factor for the final implementations. The actual compute/server/running costs for big AI Agent implementation contracts is <1%, so making it 100x more efficient doesn't really matter.
vergessenmir 1y ago
Go is great for concurrency. Not quite there for agent support. The problem isn't performance or message passing it's the agent middleware i.e logging, tracing, retries, configuration You need a DSL either supported in the language or through configuration. These are features you get for free in python and secondly JavaScript. You have to write most of this yourself in go
prats226 1y ago
So far bigger bottleneck I have found in writing agents is in scaling integrations and not the for loop for agent. Lack of libraries for go is a really big challenge.
huqedato 1y ago
Following the article's logic, Elixir is a better fit for agents. Ideal I would say.
crawshaw 1y ago
We have been having good luck writing Go with an agent. Sketch is mostly written with itself. (There is special Go handling built into the agent, e.g. automatically running gofmt/goimports after files change.) https://github.com/boldsoftware/sketch https://github.com/boldsoftware/sketch
deleted 1y ago
[deleted]
paxys 1y ago
Every single feature of an "agent" they have described is just...generic software development. Writing loops. if/else statements to branch execution paths. Waiting on input. Spawning child processes and communicating with them. Running CPU-bound operations (like parsing). So every discussion about the "best" programming language is really you telling the world about your favorite language. Use Go. Use Python. Use JavaScript. Use whatever the hell else you want. They are all good enough for the job. If you are held back it won't be because of the language itself.
- abelanger 1y ago
  For an agent that executes locally, or an agent that doesn't execute very often, I'd agree it's arbitrary. But programming languages make tradeoffs on those very paths (particularly spawning child processes and communicating with them, how underlying memory is accessed and modified, garbage collection). Agents often involve a specific architecture that's useful for a language with powerful concurrency features. These features differentiate the language as you hit scale. Not every language is equally suited to every task.
gyudin 1y ago
It is not. Human coding languages and paradigms revolve around solving problems related to issues that human struggle with. We need AI coding languages that are easy to read and verify by humans, but should solve problems that AI agents struggle with.
arthurcolle 1y ago
Erlang is a way better fit for a distributed agent orchestration layer. You have a ton of dependencies, over network and maybe in userspace, you have a lot of inter-operability and reliability constraints, you want to hotswap code and capabilities at runtime, without degrading the overall system performance. And you get networking/distribution/async message passing for free https://github.com/arthurcolle/agents.erl https://github.com/arthurcolle/agents.erl I consider myself an expert in this relatively niche domain and welcome follow up, critiques, and even your most challenging problems. I love this area and I think distributed systems are coming back in a big way in this new era!
- hosh 1y ago
  I came here to say that too. It already does well coordinating IoT networks. It's probably one of the most underestimated systems. The Elixir community has been working hard to be able to run models directly within BEAM, and recently, have added the capability for running Python directly.
  - arthurcolle 1y ago
    Haha I wrote https://github.com/stochastic-thread/snek.ex https://github.com/stochastic-thread/snek.ex 10 yrs ago What are they doing with Python on the BEAM these days? I'm OOTL
    - throwawaymaths 1y ago
      https://github.com/livebook-dev/pythonx https://github.com/livebook-dev/pythonx
- guywithahat 1y ago
  If you don't mind me asking why would one use Erlang over Elixir?
  - eru 1y ago
    Erlang has better syntax than Elixir. But otherwise they are mostly the same: Elixir is just an Erlang reskin. So pretty much wherever you can use one, you can use the other.
debarshri 1y ago
I agree.
mountainriver 1y ago
Rust is a way better option, not sure why it isn't being mentioned. The issue with Go, is as soon as you need to do actual machine learning it falls down. The issue with Python is that you often want concurrency in agents. Although this may be solved with Pythons new threading. Why is Rust great? It interops very well with Python, so you can write any concurrent pieces into that and simply import it into py, without needing to sacrifice any ML work. I'll be honest Go is a bit of an odd fit in the world of AI, and if thats the future I'm not sure Go has a big part to play outside of some infra stuff.
- rednafi 1y ago
  Not every post about Go needs to mention Rust. Rust has its niche and so does Go. Both kind of suck at AI. LLM researchers care about neither since Rust comes with its own headache: learning curve, slow compilation, weak stdlib, and Go’s FFI story is just sad. It’s still Python or GTFO. That said, Go is great to whip up “agents” since it’s a nicer language to write networking and glue code, which is what agents are. Other than a few niche groups, I’ve seen a lot more agents written in Go than in Rust.
  - mountainriver 1y ago
    There is wayyyyy more ML activity in Rust than Go. Rust is actually becoming quite viable in the ML space. Agents that don’t do machine learning rarely ever work, that’s the sad truth of the ecosystem.
lunarcave 1y ago
Agents easily spend >90% of their time waiting for LLMs to reply and optionally executing API calls in other services (HTTP APIs and DBs). In my experience the performance of the language runtime rarely matters. If there ever was a language feature that matters for agent performance and scale, it's actually the performance of JSON serialization and deserialization.
- fritzo 1y ago
  In my experience, the 2nd most costly function in agents (after LLM calls) is diffing/patching/merging asynchronous edits to resolve conflicts. Those conflict resolution operations can call out to low-level libraries, but they are still quite expensive optimization problems, compared to serialization etc.
  - autogn0me 1y ago
    can you be more specific about this?
    - fritzo 1y ago
      1. follow rich hickey's advice and orchestrate all llms to mutate a single shared state 2. let those llms operate asynchronously in parallel 3. when an llm wants to mutate the global state but the state has changed since it's checkout, try to safely merge changes using an expensive diff algorithm (which is still cheaper than the llm); on failure retry
  - energy123 1y ago
    What diffing/patching/merging library are you working with? Or are you building your own?
    - fritzo 1y ago
      I've used google's old diff-match-patch, a faster python binding of that C++ library fast-diff-match-patch, and biopython (which amazingly supports unicode!)
solomatov 1y ago
Is anyone aware of a good llm orchestration libraries for go like langchain for Python and Typescript?
- myzie 1y ago
  https://github.com/diveagents/dive https://github.com/diveagents/dive Dive orchestrates multi agent workflows in Go. Take a look and let me know what you think.
kristopolous 1y ago
gleam is the best. go check it out.
wwarner 1y ago
I mean why not cpp? With AI support it’s much easier to write a safe cpp17 program.
bewestphal 1y ago
If you’re not using Python or Typescripts ecosystem then you spend a lot of time as a framework dev. This has a high opportunity cost when you can easily slap together agents and have products quickly nowadays.
myzie 1y ago
+1 to the article. Along these lines, I'm building Dive: https://github.com/diveagents/dive https://github.com/diveagents/dive When building a SaaS with a Go backend, it's nice to be able to have the option of the agents and workflows being in the same process. And being confident in the ability of that to scale well. While it's true that Go lacks good ML libraries, for some this isn't too consequential if your app is primarily using Anthropic or OpenAI and a database that offers semantic or hybrid search for RAG. The ML is done elsewhere. Plus it could be that you can leverage MCP servers and at that point you're language agnostic. Regarding the concurrency model approach with Go and agents, I initially baked a message based approach (a la the Actor model, with one goroutine per agent) into Dive Agents, but eventually found that this would be better implemented as another layer. So currently in Dive it's the user's choice on how to implement concurrency and whether to use messaging. But I anticipate building that back in as an optional layer.
Aperocky 1y ago
I tend to agree, however, be very careful about proliferating channels since it's about as easy to write them as Java devs write Factories and Managers.
jillesvangurp 1y ago
Go isn't horrible for this stuff. But I don't think it's notably better than a lot of other languages either. Frankly, anything that has a compiler and supports doing asynchronous stuff decently probably does the job. Which of course describes a wide range of languages. And since agents inherently involve a lot (some would say mostly) prompt engineering, it helps if the language is good at things like multi line strings, templated strings, and just generally manipulating strings. As for the async stuff, it's nice if a language can do async things. But is that enough? Agentic systems essentially reach out to other systems over the network. Some of the tasks may be long lived. Minutes, hours, or even days. A lot can happen in such long time. IMHO the model of some system keeping all that state in a long running process is probably not ideal. We might want something more robust and long running and less dependent on a some stateful process running somewhere for days on end. There is an argument to be made for externalizing related state from the language and maybe using some middleware optimized for this sort of thing. I've seen a few things that go in that direction but not a lot yet. It seems that people are still busy reinventing wheels and not fully realizing yet that a lot of those wheels don't need reinventing. There's a lot of middleware out there that is really great at async job scheduling, processing, fan out, and all the other stuff that people eventually will figure out is needed here.
flanked-evergl 1y ago
Go's quite horrendous and limited type system makes it a poor fit for everything. The worst thing about Go is, in fact, the language. Everything except the language redeems it.
- giik 1y ago
  Can you elaborate a bit on how does "Go's quite horrendous and limited type system" get in the way of crafting agents? Honest question, I am genuinely interested in what cannot be done easily or at all due to limitations of the Go type system.
  - zveyaeyv3sfye 1y ago
    It's just a uninformed hivemind comment written by someone lacking original thought. If you are interested in the merits of golang, you should listen to someone who uses it.
    - flanked-evergl 1y ago
      I used it for years.
  - williamdclt 1y ago
    I think the point was that "Go's quite horrendous and limited type system" gets in the way of everything (programming in general), nothing specific to crafting agents. There's a lot of discussions on the internet about the bad design decisions of Golang (for example around channels, enums, error handling, redeclarations, interfaces, zero values, nilability... at least generics aren't so much a subject anymore)
  - Luker88 1y ago
    If you want to know only about the type system, nowadays it's mostly the lack of basic enums, a clear divide in basic features of the language and of the libraries (and modern generics support) leading to things like `len(..)` vs `.Len()`. Those actually end up playing a bigger role than it seems imho, but even just the rest is death by a thousand cuts. You can find many articles on the internet about it, but in my experience I would summarize it in: It looks like it's made to have a simple compiler, not to simplify the programmer's life. Initially its simplicity is wonderful. Then you start to notice how verbose things are. Channels are another looks-nice-but-maybe-don't feature. nil vs nil-interface. Lack of proper enums is hurting so much I can't describe it. I personally hate automatic type conversions, and there are so many inconsistencies in the standard and most used libraries that you really start to wonder why some things where even done. validators that validate nothing, half-done tagging systems for structs, tons of similar-but-not-quite interfaces and methods. It's like the language has learning wheels that you can't shake off or work around. You end up wanting to leave for a better one. People had to beg for years for basic generics and small features. If google is not interested in it, you'd better not be interested in it and it shows after a while. Companies started to use it as an alternative to C and C++, while in reality it's an alternative to python. Just like in python a lot of the work and warnings are tied into the linter as a clear workaround. Our linter config has something like 70+ linters classes enabled, and we are a very small team. C can be described as a relatively simple language (with caveats), C++ has grown to a blob that does and has everything, and while they have lots of footguns I did not find the same level of frustration as with go. You always end up fighting a lot of corner cases everywhere. Wanted to say even more, but I think I ranted enough.
jeswin 1y ago
Go has few advantages for this kind of workload - most of the time it'll just be waiting on io. And you suffer from the language itself; many type system features that you get for free in modern langauges require workarounds in Go. I've found that TypeScript is an excellent glue language for all kinds of AI. Python, followed by TS enjoy broad library support from vendors. I personally prefer it over Python because the type system is much more expressive and mature. Python is rapidly improving though. > It turns out, cancelling long-running work in Node.js and Python is incredibly difficult for multiple reasons: Evidence is lacking for this claim. Almost all tools out there support cancellations, and they're mostly either Python or JS.
- pjmlp 1y ago
  Plus if one really needs more performance than V8 can delivery, I rather write a native module in C++/Rust than reach out to Go.
dragochat 1y ago
...could we just get Go's GREAT concurrency model and decent standard lib, but in a language that is less horrible than Go (like with decent type system, enums, expressions based grammar, pattern matching etc etc)? pretty please :P we all yearn for a good static language, and most of us would kill for "something like Rust (good type system, syntax, tools) but without ownership / linear-typing - just a good GC, all-on-the-heap and a dash of nice immutable datastructs"...
- Bgd1 1y ago
  Scala?
  - dragochat 1y ago
    good point, might be worth a revisit
TeeWEE 1y ago
The main thing that lacks in go is auto OpenAOI generation from a golang func. You at least need reflection. It can be done. But not as easy as in python
bschmidt5000 1y ago
[flagged]
rgavuliak 1y ago
This reminds me of a talk named - Modern Data Science in Go. That line of thinking went nowhere and many arguments in that talk were misleading.
prats226 1y ago
Reminded my of this article: https://ampcode.com/how-to-build-an-agent https://ampcode.com/how-to-build-an-agent Some of the things are just more natural in python being a dynamic language. Eg decorator to quickly convert methods into tool calls, iterating over tool functions to create list of tools, packages to quickly convert them into json schema etc. Consuming many incoming triggers, eg from user input as well as incoming emails from gmail, or messages from slack which would trigger new agent run was lot more natural in go with channels and switch for loop vs in python where had to create many queues and threading etc

236 points by abelanger 1y ago

I have same bunch of reasoning for Java. The concept of Notebooks won the hearts of developers I guess.

> concurrency by that logic Elixir is even better for agents. also the link at the bottom of the page is pretty much why I ditched Go: https://go.dev/blog/error-syntax https://go.dev/blog/error-syntax The AI landscape moves so fast, and this conservative, backwards looking mindset of the new Go dev team doesn't match the forward looking LLM engineering mindset.

jclulow 1y ago
I don't use either Go or LLMs, but isn't the point of LLMs that they write the tedious boilerplate for you? What's the value in a small syntactic improvement if the computer is generating it all anyway?
- behnamoh 1y ago
  Elixir's concurrency model is fundamentally different than Go's; it's not just syntax difference.
catlover76 1y ago
[dead]
kamikaz1k 1y ago
by your logic, only considering part of the argument is as good as considering the entire argument.
guywithahat 1y ago
Well elixir doesn't produce goroutines (managed threads), they produce "lightweight processes" which have isolated memory. These are more expensive to grow and aren't as easy to share data between one another, although they're much more fault tolerant as a result. It could be better however the underlying concurrency model in Elixir is relatively unique
- regularfry 1y ago
  It's been a while since I was in the weeds on this, but if I remember correctly they're strictly speaking mostly isolated. Binaries above a certain size share storage between processes, so moving big blobs between processes is cheap.
  - guywithahat 1y ago
    They also have their own system to share data between processes, although I haven't used it. Generally though it's a unique tool that's not always interchangeable with Go
bheadmaster 1y ago
> also the link at the bottom of the page is pretty much why I ditched Go: https://go.dev/blog/error-syntax https://go.dev/blog/error-syntax Funnily, it's also one of the reasons I stay with Go. Error handling is the most contraversial Go topic, with half the people saying it's terrible and needs a new syntax, and half saying it's perfect and adding any more syntax will ruin it.
sorentwo 1y ago
Absolutely! Elixir's lightweight processes and distribution story make it ideal for orchestration, and that includes orchestrating LLMs. Shameless plug, but that's what many people have been using Oban Pro's Workflows for recently, and something we demonstrated in our "Cascading Workflows" article: https://oban.pro/articles/weaving-stories-with-cascading-workflows https://oban.pro/articles/weaving-stories-with-cascading-wor... Unlike hatchet, it actually runs locally, in your own application as well.
jerf 1y ago
If it had a larger learning base, quite possibly. Erlang possibly even more so. The argument that pure code is generally safer to vibe code is compelling to me. (Elixir's purity is rather complicated to describe, Erlang's much more obvious and clear.) It's easier to analyze that this bit of code doesn't reach out and break something else along the way. Though it would be nice to have a language popular enough for the LLMs to work well on, that was pure, but that was also fast. At the moment writing in pure code means taking a fairly substantial performance hit, and I'm not talking about the O(n log n) algorithm slowdowns, I mean just normal performance.
tptacek 1y ago
Elixir is great for agents.
achileas 1y ago
This makes me want to build agents in Elixir now

tolerance 1y ago

As a functionally-code-illiterate-vibe-coder, I can confirm that LLMs are good at writing Go code.

dpe82 1y ago
As a highly-literate developer with almost 30 years of experience, I can also confirm that LLMs are very good at writing Go.
- crabmusket 1y ago
  "The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt." – Rob Pike This fits LLMs pretty well too it seems!
  - dpe82 1y ago
    That's my assumption too: it's a simple language with very good tooling that's used by reasonably serious projects with good code quality. As a result the LLMs have both high quality training data and the output is easier to get right.

odyssey7 1y ago

AI engineers will literally invent a new universe before they touch JavaScript. The death knell for variety in AI languages was when Google rug-pulled TensorFlow for Swift.

dpe82 1y ago
Avoiding JavaScript like the plague that it is, is not unique to AI engineers. -Someone who has written a ton of JS over the past... almost 30 years now.
- dpkirchner 1y ago
  Choosing Python over JavaScript is one of the more perplexing decisions I've seen.
  - jpk 1y ago
    It's not so perplexing when you understand that Python has long had the best ecosystem of libraries for data science and ML, from which the current wave of AI stuff was born. There are plenty of reasons to dunk on Python, but the reality is lots of people were getting real work done with it in the run up to where we are today.
    - odyssey7 1y ago
      There are choices at multiple levels. Yes, today’s ML engineer has practically no choice but to use Python, in a variety of settings, if they want to be able to work with others, access the labor market without it being an uphill battle, and most especially if they want to study AI / ML at a university. But there were also the choices to initially build out that ecosystem in Python and to always teach AI / ML in Python. They made sense logistically, since universities largely only teach Python, so it was a lowest-common-denominator language that allowed the universities to give AI / ML research opportunities to everyone, with absolutely no gatekeeping and with a steadfast spirit of friendly inclusion (sorry, couldn’t resist the sarcastic tangent). I can’t blame them for working with what they had. But now that the techniques have grown up and graduated to form multibillion-dollar companies, I’m hopeful that industry will take up the mantle to develop an ecosystem that’s better suited for production and for modern software engineering.

guywithahat 1y ago

I wish we had better concurrency models in the ML world. I tried doing some ML in Go a few months back and it's basically impossible; there's just no library support and doing anything requires a gRPC call or a wrapper. Python has limitations and C++ has a tendency to make everything too verbose.

kamikaz1k 1y ago

> High concurrency > Share memory by communicating > Centralized cancellation mechanism with context.Context > Expansive standard library > Profiling > Bonus: LLMs are good at writing Go code I think profiling is probably the lowest value good here, but would be willing to hear out stories of AI middleware applications that found value in that. Cancelling tasks is probably the highest value good here, but I think the contending runtimes (TS/Python) all prefer using 3P libraries to handle this kind of stuff, so probably not the biggest deal. Being able to write good Go code is pretty cool though; I don't write enough to make a judgement there.

eikenberry 1y ago
> Bonus: LLMs are good at writing Go code Good at writing bad code. But most of the code in the wild is written by mid-level devs, without guidance and on short timelines.. i.e bad code. But this is a problem with all languages, not just Go.

jasonthorsness 1y ago

Go is great for command-line tools because of library support and fast-starting single-binaries. While most of the benefits in the article are also shared with JavaScript, I wonder if the CLI advantage will help and whether command-line agents will become a thing ("grepllm"?) The language of agents doesn't matter much in the long run as it's just a thin shell of tool definitions and API calls to the backing LLM.

skybrian 1y ago

For long-running, expensive processes that do a lot of waiting, a downside is that if you kill the process running the goroutine, you lose all your work. It might be better to serialize state to a database while waiting? But this adds a lot of complexity and I don’t know any languages that make it easy to write this sort of checkpoint-based state machine.

Karrot_Kream 1y ago
Temporal is pretty decent at checkpointing long-running processes and is language agnostic.
sorentwo 1y ago
That's the issue with goroutines, threads, or any long running chain of processes. The tasks must be broken up into atomic chunks, and the state has to be serialized in some way. That allows failures to be retried, errors to be examined, results to be referenced later, and the whole thing to be distributed between multiple nodes. It must in my view at least, as that's how Oban (https://github.com/oban-bg/oban https://github.com/oban-bg/oban) in Elixir models this kind of problem. Full disclosure, I'm an author and maintainer of the project. It's Elixir specific, but this article emphasizes the importance of async task persistence: https://oban.pro/articles/oban-starts-where-tasks-end https://oban.pro/articles/oban-starts-where-tasks-end
ashishb 1y ago
> For long-running, expensive processes that do a lot of waiting, a downside is that if you kill the goroutine, you lose all your work. This is true regardless of the language. I always do a reasonable amount of work (milliseconds to up to a few seconds) worth of work in a Go routine every time. Anything more and your web service is not as stateless as it should be.
abelanger 1y ago
OP here - this type of "checkpoint-based state machine" is exactly what platforms which offer durable execution primitives like Hatchet (https://hatchet.run/ https://hatchet.run/) and Temporal (https://temporal.io/ https://temporal.io/) are offering. Disclaimer: am a founder of Hatchet. These platforms store an event history of the functions which have run as part of the same workflow, and automatically replay those when your function gets interrupted. I imagine synchronizing memory contents at the language level would be much more overhead than synchronizing at the output level.

awinter-py 1y ago

oh my god these things run on a gpu don't they? they have nothing to do with golang? to the extent they run on a cpu they're heavy; we're not like solving the c10k problem with agents

achileas 1y ago
Agents don't typically, and any LLM they're calling is likely hosted remotely.

_QrE 1y ago

I'm not sure how valid most of these points are. A lot of the latency in an agentic system is going to be the calls to the LLM(s). From the article: """ Agents typically have a number of shared characteristics when they start to scale (read: have actual users): They are long-running — anywhere from seconds to minutes to hours. Each execution is expensive — not just the LLM calls, but the nature of the agent is to replace something that would typically require a human operator. Development environments, browser infrastructure, large document processing — these all cost $$$. They often involve input from a user (or another agent!) at some point in their execution cycle. They spend a lot of time awaiting i/o or a human. """ No. 1 doesn't really point to one language over another, and all the rest show that execution speed and server-side efficiency is not very relevant. People ask agents a question and do something else while the agent works. If the agent takes a couple seconds longer because you've written it in Python, I doubt that anyone would care (in the majority of cases at least). I'd argue Python is a better fit for agents, mostly because of the mountain of AI-related libraries and support that it has. > Contrast this with Python: library developers need to think about asyncio, multithreading, multiprocessing, eventlet, gevent, and some other patterns... Agents aren't that hard to make work, and you can get most of the use (and paying users) without optimizing every last thing. And besides, the mountain of support you have for whatever workflow you're building means that someone has probably already tried building at least part of what you're working on, so you don't have to go in blind.

tptacek 1y ago
That's true from a performance perspective but, in building an agent in Go, I was thankful that I had extremely well-worn patterns to manage concurrency, backlogs, and backpressure given that most interactions will involve one or more transactions with a remote service that takes several seconds to respond. (I think you can effectively write an agent in any language and I think Javascript is probably the most popular choice. Now, generating code, regardless of whether it's an agent or a CLI tool or a server --- there, I think Go and LLM have a particularly nice chemistry.)
philwelch 1y ago
Go still has a much better concurrency story. It’s also much less of a headache to deploy since all you need to deploy is a static binary and not a whole bespoke Python runtime with every pip dependency.

danenania 1y ago

I built Plandex[1] (open source CLI coding agent focused on large projects and tasks) in Go and I’ve been very happy with that decision. Beneath all the jargon, it’s good to remember that an “agent” is ultimately just a bunch of http requests and streams that need to be coordinated—some serially and some concurrently. And while that sounds pretty simple at a high level, there are many subtle details to pay attention to if you want to make this kind of system robust and scalable. Timeouts, retries, cancellation, error handling, thread pools, thread safety, and so on. This stuff is Go’s bread and butter. It’s exactly what it was designed for. It’s not going to get you an MVP quite as fast as node or python, but as the codebase grows and edge cases accumulate, the advantages of Go become more and more noticeable. 1 - https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex

deleted 1y ago

[deleted]

npalli 1y ago

Agents to do what? Take ML/AI, all the infra and tools are Python/C++ so what exactly is the Agent going to help you with Go? Many such domains- gaming, HFT, HPC, Scientific Computing, Systems, UX, Enterprise etc. etc. Seems it really helps Go's sweet spot - CLI's and Networking services.

tptacek 1y ago
Agents mostly don't run ML/AI code; they're a structured loop around LLM calls and exist mostly to give an LLM access to local tools in some application domain; think "reading my email for me" rather than "driving ML systems".
- achileas 1y ago
  That doesn't negate what OP was saying at all, the better support in Python isn't for "running ML/AI code" but in things like agent frameworks, observability tools, SDKs, etc. None of which directly run AI code but are still helpful/necessary and for the most part better represented (and supported) in the Python world, although that seems like it's slowly changing.
  - tptacek 1y ago
    There are agent frameworks in most languages at this point so the question just comes down to "can you invoke tools for the problem you want to solve in that language". Yes, Python is really great at that. So is Go and Javascript. I think I'd condense this out to "this is not a really important deciding factor in what language you choose for your agent". If you know you need something you can only get in Python, you'll write the agent in Python.

InTheArena 1y ago

I don't think I agree with the Go is good in LLMs. But outside of that - ML in go is basically impossible. Trying to integrate with the outside ecosystem of go is really difficult - and my experience has been that Claude Code is far less effective with Go then it is with Python, or even Swift. I ditched a project I was writing in Go and replaced it with Swift (this was mostly prompt based anyways). It was remarkably how much better the first pass of the code generation was.

hoppp 1y ago
What if... hear me out... You learn to write code instead of generating it...there is a drastic improvement in code quality if you can actually write it.

pantsforbirds 1y ago

I've been messing around with an Elixir + BEAM based agent framework. I think a mixture of BEAM + SQLite is about as good as you can get for agents right now. You can safely swap out agents without redeploying the application, the concurrency is way below the scale BEAM was built for, and creating stateful or ephemeral agents is incredibly easy. My plan is to set up a base agent in Python, Typescript, and Rust using MCP servers to allow users to write more complex agents in their preferred programming language too.

nilslice 1y ago
you should check out the Extism[0] project and the Elixir SDK[1]. This would allow you to write the core services, routing, message passing, etc in Elixir, and leverage all the BEAM/OTP have to offer, and then embed "agents" written in other languages which are small Wasm modules that act like in-process plugins. [0]: https://github.com/extism/extism https://github.com/extism/extism [1]: https://github.com/extism/elixir-sdk https://github.com/extism/elixir-sdk
- pantsforbirds 1y ago
  That's a really interesting idea. My original thought was to use MCP as the way to define other agents, but I'll have to do some more research into extism!
alberth 1y ago
Any reason for SQLite use, instead of the BEAMs built-in mnesia data store? https://www.erlang.org/doc/apps/mnesia/mnesia.html https://www.erlang.org/doc/apps/mnesia/mnesia.html
- pantsforbirds 1y ago
  I'm still in the exploration/experimentation stage of the project, but I'm currently using a mixture of SQLite, PostgreSQL, S3, and DuckDB. My original thought was to spin up SQLite databases as needed because they are super lightweight, well-tested, and supported by almost every programming language. If you want to set up an agent in another programming language via MCP, but you still want to be able to access the agent memory directly, you can use the same schema in a SQLite database. I may end up using mnesia for more metadata or system-oriented data storage though. It's very well designed imo. But one of the biggest reasons has just been the really nice integration with DuckDB. I can query all of the SQLite databases persisted in a directory and aggregate some metadata really easily.

segmondy 1y ago

go is terrible for agents IMO great for agents? lisp and prolog.

esafak 1y ago
Not without a supporting ecosystem!

hoppp 1y ago

Go is a good fit for many use-cases.

rpep 1y ago

In practice the library ecosystem is just way behind Python. Maybe after you’re trying to optimise once you’ve worked out how to do stuff, but even the Langchain Go port is wayyyyy behind.

carsoon 1y ago

I wrote the start to an agent library in Go. Its quite rough as most of it was implemented through using AI but I had a lot of ideas through planning/building it. 1. If you make your agents/workflows serializable you can run/load them from a config file or add/remove them from a decoupled frontend. You can also hash them to make versioning easy to track/immutable. 2. If you decouple the stateful object from the agent/workflow object you can just store that through sufficient logging then you can rebuild any flow at any state and have branching by allowing traces to build on one another. You can also restart/rerun a flow starting at any location. 3. You can allow for serializable tools by having a standard HttpRequestTool then setup cloudflare workers/any external endpoints for the actual toolcall logic. Removing primary server load and making it possible to add/remove tools without rebuilding/restarting. Given this system in golang you can have a single server which supports tens of thousands of concurrent agent workflows. The biggest problem is there isn't that many people who are working on it. So even if you can make agents 100x more efficient by running in Go it doesn't really matter if cost isn't the biggest factor for the final implementations. The actual compute/server/running costs for big AI Agent implementation contracts is <1%, so making it 100x more efficient doesn't really matter.

vergessenmir 1y ago

Go is great for concurrency. Not quite there for agent support. The problem isn't performance or message passing it's the agent middleware i.e logging, tracing, retries, configuration You need a DSL either supported in the language or through configuration. These are features you get for free in python and secondly JavaScript. You have to write most of this yourself in go

prats226 1y ago

So far bigger bottleneck I have found in writing agents is in scaling integrations and not the for loop for agent. Lack of libraries for go is a really big challenge.

huqedato 1y ago

Following the article's logic, Elixir is a better fit for agents. Ideal I would say.

crawshaw 1y ago

We have been having good luck writing Go with an agent. Sketch is mostly written with itself. (There is special Go handling built into the agent, e.g. automatically running gofmt/goimports after files change.) https://github.com/boldsoftware/sketch https://github.com/boldsoftware/sketch

deleted 1y ago

[deleted]

paxys 1y ago

Every single feature of an "agent" they have described is just...generic software development. Writing loops. if/else statements to branch execution paths. Waiting on input. Spawning child processes and communicating with them. Running CPU-bound operations (like parsing). So every discussion about the "best" programming language is really you telling the world about your favorite language. Use Go. Use Python. Use JavaScript. Use whatever the hell else you want. They are all good enough for the job. If you are held back it won't be because of the language itself.

abelanger 1y ago
For an agent that executes locally, or an agent that doesn't execute very often, I'd agree it's arbitrary. But programming languages make tradeoffs on those very paths (particularly spawning child processes and communicating with them, how underlying memory is accessed and modified, garbage collection). Agents often involve a specific architecture that's useful for a language with powerful concurrency features. These features differentiate the language as you hit scale. Not every language is equally suited to every task.

gyudin 1y ago

It is not. Human coding languages and paradigms revolve around solving problems related to issues that human struggle with. We need AI coding languages that are easy to read and verify by humans, but should solve problems that AI agents struggle with.

arthurcolle 1y ago

Erlang is a way better fit for a distributed agent orchestration layer. You have a ton of dependencies, over network and maybe in userspace, you have a lot of inter-operability and reliability constraints, you want to hotswap code and capabilities at runtime, without degrading the overall system performance. And you get networking/distribution/async message passing for free https://github.com/arthurcolle/agents.erl https://github.com/arthurcolle/agents.erl I consider myself an expert in this relatively niche domain and welcome follow up, critiques, and even your most challenging problems. I love this area and I think distributed systems are coming back in a big way in this new era!

hosh 1y ago
I came here to say that too. It already does well coordinating IoT networks. It's probably one of the most underestimated systems. The Elixir community has been working hard to be able to run models directly within BEAM, and recently, have added the capability for running Python directly.
- arthurcolle 1y ago
  Haha I wrote https://github.com/stochastic-thread/snek.ex https://github.com/stochastic-thread/snek.ex 10 yrs ago What are they doing with Python on the BEAM these days? I'm OOTL
  - throwawaymaths 1y ago
    https://github.com/livebook-dev/pythonx https://github.com/livebook-dev/pythonx
guywithahat 1y ago
If you don't mind me asking why would one use Erlang over Elixir?
- eru 1y ago
  Erlang has better syntax than Elixir. But otherwise they are mostly the same: Elixir is just an Erlang reskin. So pretty much wherever you can use one, you can use the other.

debarshri 1y ago

I agree.

mountainriver 1y ago

Rust is a way better option, not sure why it isn't being mentioned. The issue with Go, is as soon as you need to do actual machine learning it falls down. The issue with Python is that you often want concurrency in agents. Although this may be solved with Pythons new threading. Why is Rust great? It interops very well with Python, so you can write any concurrent pieces into that and simply import it into py, without needing to sacrifice any ML work. I'll be honest Go is a bit of an odd fit in the world of AI, and if thats the future I'm not sure Go has a big part to play outside of some infra stuff.

rednafi 1y ago
Not every post about Go needs to mention Rust. Rust has its niche and so does Go. Both kind of suck at AI. LLM researchers care about neither since Rust comes with its own headache: learning curve, slow compilation, weak stdlib, and Go’s FFI story is just sad. It’s still Python or GTFO. That said, Go is great to whip up “agents” since it’s a nicer language to write networking and glue code, which is what agents are. Other than a few niche groups, I’ve seen a lot more agents written in Go than in Rust.
- mountainriver 1y ago
  There is wayyyyy more ML activity in Rust than Go. Rust is actually becoming quite viable in the ML space. Agents that don’t do machine learning rarely ever work, that’s the sad truth of the ecosystem.

lunarcave 1y ago

Agents easily spend >90% of their time waiting for LLMs to reply and optionally executing API calls in other services (HTTP APIs and DBs). In my experience the performance of the language runtime rarely matters. If there ever was a language feature that matters for agent performance and scale, it's actually the performance of JSON serialization and deserialization.

fritzo 1y ago
In my experience, the 2nd most costly function in agents (after LLM calls) is diffing/patching/merging asynchronous edits to resolve conflicts. Those conflict resolution operations can call out to low-level libraries, but they are still quite expensive optimization problems, compared to serialization etc.
- autogn0me 1y ago
  can you be more specific about this?
  - fritzo 1y ago
    1. follow rich hickey's advice and orchestrate all llms to mutate a single shared state 2. let those llms operate asynchronously in parallel 3. when an llm wants to mutate the global state but the state has changed since it's checkout, try to safely merge changes using an expensive diff algorithm (which is still cheaper than the llm); on failure retry
- energy123 1y ago
  What diffing/patching/merging library are you working with? Or are you building your own?
  - fritzo 1y ago
    I've used google's old diff-match-patch, a faster python binding of that C++ library fast-diff-match-patch, and biopython (which amazingly supports unicode!)

solomatov 1y ago

Is anyone aware of a good llm orchestration libraries for go like langchain for Python and Typescript?

myzie 1y ago
https://github.com/diveagents/dive https://github.com/diveagents/dive Dive orchestrates multi agent workflows in Go. Take a look and let me know what you think.

kristopolous 1y ago

gleam is the best. go check it out.

wwarner 1y ago

I mean why not cpp? With AI support it’s much easier to write a safe cpp17 program.

bewestphal 1y ago

If you’re not using Python or Typescripts ecosystem then you spend a lot of time as a framework dev. This has a high opportunity cost when you can easily slap together agents and have products quickly nowadays.

myzie 1y ago

+1 to the article. Along these lines, I'm building Dive: https://github.com/diveagents/dive https://github.com/diveagents/dive When building a SaaS with a Go backend, it's nice to be able to have the option of the agents and workflows being in the same process. And being confident in the ability of that to scale well. While it's true that Go lacks good ML libraries, for some this isn't too consequential if your app is primarily using Anthropic or OpenAI and a database that offers semantic or hybrid search for RAG. The ML is done elsewhere. Plus it could be that you can leverage MCP servers and at that point you're language agnostic. Regarding the concurrency model approach with Go and agents, I initially baked a message based approach (a la the Actor model, with one goroutine per agent) into Dive Agents, but eventually found that this would be better implemented as another layer. So currently in Dive it's the user's choice on how to implement concurrency and whether to use messaging. But I anticipate building that back in as an optional layer.

Aperocky 1y ago

I tend to agree, however, be very careful about proliferating channels since it's about as easy to write them as Java devs write Factories and Managers.

jillesvangurp 1y ago

Go isn't horrible for this stuff. But I don't think it's notably better than a lot of other languages either. Frankly, anything that has a compiler and supports doing asynchronous stuff decently probably does the job. Which of course describes a wide range of languages. And since agents inherently involve a lot (some would say mostly) prompt engineering, it helps if the language is good at things like multi line strings, templated strings, and just generally manipulating strings. As for the async stuff, it's nice if a language can do async things. But is that enough? Agentic systems essentially reach out to other systems over the network. Some of the tasks may be long lived. Minutes, hours, or even days. A lot can happen in such long time. IMHO the model of some system keeping all that state in a long running process is probably not ideal. We might want something more robust and long running and less dependent on a some stateful process running somewhere for days on end. There is an argument to be made for externalizing related state from the language and maybe using some middleware optimized for this sort of thing. I've seen a few things that go in that direction but not a lot yet. It seems that people are still busy reinventing wheels and not fully realizing yet that a lot of those wheels don't need reinventing. There's a lot of middleware out there that is really great at async job scheduling, processing, fan out, and all the other stuff that people eventually will figure out is needed here.

flanked-evergl 1y ago

Go's quite horrendous and limited type system makes it a poor fit for everything. The worst thing about Go is, in fact, the language. Everything except the language redeems it.

giik 1y ago
Can you elaborate a bit on how does "Go's quite horrendous and limited type system" get in the way of crafting agents? Honest question, I am genuinely interested in what cannot be done easily or at all due to limitations of the Go type system.
- zveyaeyv3sfye 1y ago
  It's just a uninformed hivemind comment written by someone lacking original thought. If you are interested in the merits of golang, you should listen to someone who uses it.
  - flanked-evergl 1y ago
    I used it for years.
- williamdclt 1y ago
  I think the point was that "Go's quite horrendous and limited type system" gets in the way of everything (programming in general), nothing specific to crafting agents. There's a lot of discussions on the internet about the bad design decisions of Golang (for example around channels, enums, error handling, redeclarations, interfaces, zero values, nilability... at least generics aren't so much a subject anymore)
- Luker88 1y ago
  If you want to know only about the type system, nowadays it's mostly the lack of basic enums, a clear divide in basic features of the language and of the libraries (and modern generics support) leading to things like `len(..)` vs `.Len()`. Those actually end up playing a bigger role than it seems imho, but even just the rest is death by a thousand cuts. You can find many articles on the internet about it, but in my experience I would summarize it in: It looks like it's made to have a simple compiler, not to simplify the programmer's life. Initially its simplicity is wonderful. Then you start to notice how verbose things are. Channels are another looks-nice-but-maybe-don't feature. nil vs nil-interface. Lack of proper enums is hurting so much I can't describe it. I personally hate automatic type conversions, and there are so many inconsistencies in the standard and most used libraries that you really start to wonder why some things where even done. validators that validate nothing, half-done tagging systems for structs, tons of similar-but-not-quite interfaces and methods. It's like the language has learning wheels that you can't shake off or work around. You end up wanting to leave for a better one. People had to beg for years for basic generics and small features. If google is not interested in it, you'd better not be interested in it and it shows after a while. Companies started to use it as an alternative to C and C++, while in reality it's an alternative to python. Just like in python a lot of the work and warnings are tied into the linter as a clear workaround. Our linter config has something like 70+ linters classes enabled, and we are a very small team. C can be described as a relatively simple language (with caveats), C++ has grown to a blob that does and has everything, and while they have lots of footguns I did not find the same level of frustration as with go. You always end up fighting a lot of corner cases everywhere. Wanted to say even more, but I think I ranted enough.

jeswin 1y ago

Go has few advantages for this kind of workload - most of the time it'll just be waiting on io. And you suffer from the language itself; many type system features that you get for free in modern langauges require workarounds in Go. I've found that TypeScript is an excellent glue language for all kinds of AI. Python, followed by TS enjoy broad library support from vendors. I personally prefer it over Python because the type system is much more expressive and mature. Python is rapidly improving though. > It turns out, cancelling long-running work in Node.js and Python is incredibly difficult for multiple reasons: Evidence is lacking for this claim. Almost all tools out there support cancellations, and they're mostly either Python or JS.

pjmlp 1y ago
Plus if one really needs more performance than V8 can delivery, I rather write a native module in C++/Rust than reach out to Go.

dragochat 1y ago

...could we just get Go's GREAT concurrency model and decent standard lib, but in a language that is less horrible than Go (like with decent type system, enums, expressions based grammar, pattern matching etc etc)? pretty please :P we all yearn for a good static language, and most of us would kill for "something like Rust (good type system, syntax, tools) but without ownership / linear-typing - just a good GC, all-on-the-heap and a dash of nice immutable datastructs"...

Bgd1 1y ago
Scala?
- dragochat 1y ago
  good point, might be worth a revisit

TeeWEE 1y ago

The main thing that lacks in go is auto OpenAOI generation from a golang func. You at least need reflection. It can be done. But not as easy as in python

bschmidt5000 1y ago

[flagged]

rgavuliak 1y ago

This reminds me of a talk named - Modern Data Science in Go. That line of thinking went nowhere and many arguments in that talk were misleading.

prats226 1y ago

Reminded my of this article: https://ampcode.com/how-to-build-an-agent https://ampcode.com/how-to-build-an-agent Some of the things are just more natural in python being a dynamic language. Eg decorator to quickly convert methods into tool calls, iterating over tool functions to create list of tools, packages to quickly convert them into json schema etc. Consuming many incoming triggers, eg from user input as well as incoming emails from gmail, or messages from slack which would trigger new agent run was lot more natural in go with channels and switch for loop vs in python where had to create many queues and threading etc