4 ms·
I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
by rullopat 5mo ago
I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
- iddan 5mo agoRunning many of those in scale.
- tornikeo 5mo agoSimplest explanation I could come up with: Just for hype and fun. Rewriting things in rust is "cool". Bun did it, other projects did it. Therefore, writing a coding agent in one should be cool too. And apparently enough HN crowd agrees with it to take the #1 spot on the board.
- GodelNumbering 5mo agoFor the most part, doing things right in the given language matters more than change of language. A lot of refactors in Rust (in the coding agent space) I see jump straight to Rust without considering what inefficiencies can be addressed before changing the language. Having said that, I considered a Go/Rust rewrite of Dirac (https://github.com/dirac-run/dirac https://github.com/dirac-run/dirac) for some modules to support cases when someone wants to run like 30 agents, but it quickly became obvious that, a) while the node event loop is a bottleneck, it is not the sole bottleneck and b) if you have a VSCode extension, you can't totally get rid of TypeScript, so it just becomes the case of bi-lingual project and the maintenance burden that comes with it
- flossly 5mo agoRust is just another language. Sure it's cooler than some langs, to some ppl. Sure. The author made the choice. Open sourced it (thanks!). So now we all enjoy more options. Saying author did so because "cool" does not sit well with me. It's feels like you get a no-strings attached gift of significant value and then going saying the giver gave it to be seen as cool.
- wint3rmute 5mo agoEven a simple coding agent TUI should work instantenously, which I sadly cannot say is true about typescript-based applications like Claude Code or Gemini. After switching away from GNOME Terminal + Zsh to Ghostty + Nushell, I started to appreciate how instant everything feels. Why not make everything just as fast?
- itsdavesanders 5mo agoI have to say this is one of my favorite things about local Qwen and Qwen code, it seems a heck of a lot faster that Claude and feels better to work with. Problem is it is nowhere near as smart, so what speed I get in conversation gets killed by iteration.
- mapcars 5mo agoI see spreading Rust as an overall good thing, because it changes benchmark on how software should feel in terms of performance, stability, memory footprint. So even if it doesn't create tangible advantage in a particular use case - its still good for the whole industry.
- amelius 5mo agoNo because it means people will use Rust for the wrong reasons. Systems programming is only a tiny fraction of code out there. Approaching every problem as a systems programming problem is a massive waste of resources and intellect.
- angusturner 5mo agoFor small to medium projects, an LLM can write functional (if not well crafted) Rust. Considering how easy this is now, why choose a heavier, slower and less typesafe language?
- singpolyma3 5mo agoCould choose a similar weight, similar speed, equal or more typesafe language though :)
- galangalalgol 5mo agoAda? Other than c and c++ everything else benchmarks 2-4 times slower than rust for compute bound tasks, even after jit warmup. I'm up for ada though, especially with an llm where I don't have to type all that verbose syntax.
- singpolyma3 5mo agoOCaml? Haskell? Idris? Lots of options with no jit or warmup
- throwa356262 5mo agoWhile we are not there yet, people are looking into running agents in esp32 and alike. See projects such as picoclaw, nullclaw and more. https://github.com/sipeed/picoclaw https://github.com/sipeed/picoclaw https://github.com/nullclaw/nullclaw https://github.com/nullclaw/nullclaw
- joelthelion 5mo agoOpencode can be surprisingly hard on the CPU (could be an issue when coding on battery or a weak remote VM), and uses a lot of RAM. A little competition is always welcome.
- jwxz 5mo agoI didn't see anyone mention this, but I think having a single binary is much nicer than having a JS (or Python) program sprawled all over your system.
- ink-splatters 5mo agoHaving single binary output is completely different problem and is solved for both Python and typescript (bun supports the later).
- crabmusket 5mo agoNode and Deno can also bundle apps into a single executable.
- jwxz 5mo agoThat's true, but it's not quite the same thing. The single binary you're referring to is the interpreter and source code packaged together (at least for TS/JS). If you install too many of these "single binaries" then at some point you would be better off just having a single interpreter and using npm/pip. By contrast the Rust binary only contains the machine code for this program and can be directly executed.
- tjoff 5mo agoBefore I tried coding agents my guess would have been: none. But seeing how slow claude code and copilot cli are and how much ram they use I'm flabbergasted. If you have long running sessions they can both take tens pf gigabytes of ram and feel quite sluggish.
- i_am_a_peasant 5mo agohuh. my evidence with codex hasn’t been so bad. and tbh why would i discourage anyone from coding. hack away mr hacker. your solution will either sink or swim
- krzyk 5mo agocodex is in rust and not in power and memory hungry js/ts.
- i_am_a_peasant 5mo agooh sweet I had no idea. funny that i mostly use it to write rust
- manmal 5mo agoCheck out its app-server, IMO it’s a decent foundation to the codex clients.
- dorian-graph 5mo agoIt was previously JS/TS, but they rewrote it in Rust, sometime in the past 12 months.
- adabsurdo 5mo ago[dead]
- Mjarvis 5mo agoYes...exactly. Its frustrating and inefficient.
- tcfhgj 5mo ago- Reduce the footprint on the planet - prolonged life of hardware - less electricity - less expensive hardware
- sdevonoes 5mo agoCompared to what LLMs actually consume, your agent makes zero difference
- tcfhgj 5mo agovery wrong - especially on the local machine, see https://news.ycombinator.com/item?id=48164613 https://news.ycombinator.com/item?id=48164613
- krzyk 5mo agoWhy would anyone compare a cloud LLMs power usage when one doesn't pay for it? Local power consumption is important for those.
- afavour 5mo agoOP specifically cited “reduce the footprint on the planet”
- phplovesong 5mo agoI recall back in the mid 2000s when i saw many "rewrite in rails" apps. Its just hype, and it will die out in a few years when something new comes out.
- cpa 5mo ago[dead]
- krzyk 5mo agoe.g. opencode right now uses ~80% of my CPU. At first I also thought that it would be just call and wait, but a lot of work is done locally (any tool calls).
- tacone 5mo agoIt's also dealing with memory issues (see: Memory Megathread https://github.com/anomalyco/opencode/issues/20695 https://github.com/anomalyco/opencode/issues/20695). And in my experience is not that much faster to start than more complex software like Visual Studio Code.
- flossly 5mo agoOver time software grows. Once big rewriting it in another language is hard and gets harder as the project grows in size. Starting with a resource-saving attitude may be a very good long term strategy. Also: with Rust there are many features of high-level, modern, type-safe, FP-inspired languages that you do not have to miss.
- faangguyindia 5mo agoIf you write in Go, you get faster compile time, more likely your code will compile fine after long time.
- rbalicki 5mo agoThat's exactly the tradeoff I made with Barnum (https://barnum-circus.github.io/ https://barnum-circus.github.io/). It's just not important to optimize the performance of the rust side for the reason you stated. So instead, all focus goes into making it easy for an LLM to build a reliable pipeline (from which LLMs are invoked).