4 ms·

> I could also see it slipping into obscurity as LLMs get faster and compile times become a more and more obvious bottleneck on iteration speeds. I’ve worked o

> I could also see it slipping into obscurity as LLMs get faster and compile times become a more and more obvious bottleneck on iteration speeds. I’ve worked on some very large Rust projects. The incremental compile times are nowhere near the same order of magnitude of a bottleneck as an LLM turn.

apitman 3mo ago
I don't doubt this is true for many projects currently (though it's not for a bevy project I'm working on). Have you tried Cerebras, Groq, Taalas, et al? It was a paradigm shift for me.
- Aurornis 3mo ago
  I have, but even with the high token generation speed, incremental Rust compile times were not a bottleneck. I was disagreeing with the concept that incremental compile times could be a bottleneck. LLMs are even better at doing large swaths of work at once and having it compile first or second try than a human.
  - apitman 3mo ago
    Oh are you saying the human remains the bottleneck?
  - nullsanity 3mo ago
    [dead]
rr808 3mo ago
How long does a compile take? Because this was the biggest issue for me. I dont really like Python but the ability to make some changes then run it instantly is wild for me.
- woodruffw 3mo ago
  You probably already know this, but I figure it bears repeating: most people should be running `cargo check` during development, not `cargo build`. The latter is only necessary when you actually need the built binary; the former is sufficient for type- and borrow-checking. (On my local machine, `cargo check` is roughly 2x faster than `cargo build`. It's still slower than hot-reloaded Python, but it's rarely my development bottleneck.)
  - pjmlp 3mo ago
    A non starter for anything related to graphics or UI development, which is kind of most Rust stuff are CLIs or TUIs, like being back on curses heyday, Turbo Vision and Clipper.
    - Klonoar 3mo ago
      Various Rust UI projects have been working towards hot-reloadable capabilities, based on the work from the Dioxus team. IME it's just not really that big of a deal at this point. YMMV, etc.
      - pjmlp 3mo ago
        I am aware, still far away from what something like C++ Builder was already offering in the 1990's, let alone other more modern alternatives.
    - woodruffw 3mo ago
      Is it? An immense amount of the world’s graphical software is written for native graphical targets like SwiftUI, where the iteration cycle is similarly bound to build times. It’s certainly painful, but I don’t think it’s a non-starter.
      - pjmlp 3mo ago
        SwiftUI has an incremental compiler with interactive workflows.
- Aurornis 3mo ago
  Initial compilation and incremental compilation are very different. Large Rust projects are broken up into modules. If you change one module you don’t have recompile the whole project. Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn.
  - majormajor 3mo ago
    > Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn. This entirely depends on (a) what a step/prompt/turn is trying to do and (b) tooling and rate limiting and various other aspects of your model and/or access to it. If you have high rate limits and aren't using an Opus-or-larger sized model you can get a LOT of changes done in 10 seconds. And this is especially true for "oh not all the tests are passing yet, let me change..." iterations where the change attempts are often low-single-digit seconds.
- embedding-shape 3mo ago
  > I dont really like Python but the ability to make some changes then run it instantly is wild for me. You'll love Clojure, gets you even closer; you can make changes while your program is running, by changing source code then applying just that function you changed, while your editor is connected to your program and can show what the new results are.
  - 7734128 3mo ago
    I'm quite often working in a Jupyter notebook where the imported modules are reloaded. I can have x in memory, change method y of X and then call x.y() and get the new definition. It feels almost criminal that it works, but it's quite nice.
insanitybit 3mo ago
I use LLMs and rust and compile times are absolutely a significant area of degradation. I love rust, I think it's the best language for LLMs, but the biggest win Rust could get for agentic development is to speed up the compiler.
- embedding-shape 3mo ago
  Neither of you talk about what sort of machine you're sitting on. Back when I used a netbook, I'd agree with you, for developing quick off programs, the compile times are horrible. But as someone on a workstation now, compilation times are the least of my problems, and disk space is more of a concern for me. You also don't share what LLM you use, some of them reason a lot, some of them nothing, some a bit. Again, personally I use LLMs with their maximum reasoning always, trading quality for speed/waiting every single time, and even compiling the Linux kernel would be faster than most LLM responses I get nowadays, for me, on a workstation.
  - insanitybit 3mo ago
    I'm on an M4 Macbook. I use Codex, gpt, and Claude in various configurations, including Fable, xhigh, medium, high, ultracode, etc.
- dnautics 3mo ago
  > I think it's the best language for LLMs because of the type system? i work in ~dynamically typed language and the llms almost never make a type error.
  - insanitybit 3mo ago
    The type system is great and gives extremely fast feedback. The performance wins are also incredible, it just feels like you're throwing 100s of megabytes of RAM away and 10x latency when you don't choose rust now, and for little reason. The testing tooling is really solid - property testing, fuzzing, mutation testing, etc, is all easy to use. Code structure with nice abstractions that don't become spaghetti is nice (traits, enums, etc). Static binaries as a production artifact is really nice. Supply chain story is pretty solid with cargo-vet. It's hard to come up with anything where I think Rust isn't "best in class" other than compile times.

apitman 3mo ago

I don't doubt this is true for many projects currently (though it's not for a bevy project I'm working on). Have you tried Cerebras, Groq, Taalas, et al? It was a paradigm shift for me.

Aurornis 3mo ago
I have, but even with the high token generation speed, incremental Rust compile times were not a bottleneck. I was disagreeing with the concept that incremental compile times could be a bottleneck. LLMs are even better at doing large swaths of work at once and having it compile first or second try than a human.
- apitman 3mo ago
  Oh are you saying the human remains the bottleneck?
- nullsanity 3mo ago
  [dead]

rr808 3mo ago

How long does a compile take? Because this was the biggest issue for me. I dont really like Python but the ability to make some changes then run it instantly is wild for me.

woodruffw 3mo ago
You probably already know this, but I figure it bears repeating: most people should be running `cargo check` during development, not `cargo build`. The latter is only necessary when you actually need the built binary; the former is sufficient for type- and borrow-checking. (On my local machine, `cargo check` is roughly 2x faster than `cargo build`. It's still slower than hot-reloaded Python, but it's rarely my development bottleneck.)
- pjmlp 3mo ago
  A non starter for anything related to graphics or UI development, which is kind of most Rust stuff are CLIs or TUIs, like being back on curses heyday, Turbo Vision and Clipper.
  - Klonoar 3mo ago
    Various Rust UI projects have been working towards hot-reloadable capabilities, based on the work from the Dioxus team. IME it's just not really that big of a deal at this point. YMMV, etc.
    - pjmlp 3mo ago
      I am aware, still far away from what something like C++ Builder was already offering in the 1990's, let alone other more modern alternatives.
  - woodruffw 3mo ago
    Is it? An immense amount of the world’s graphical software is written for native graphical targets like SwiftUI, where the iteration cycle is similarly bound to build times. It’s certainly painful, but I don’t think it’s a non-starter.
    - pjmlp 3mo ago
      SwiftUI has an incremental compiler with interactive workflows.
Aurornis 3mo ago
Initial compilation and incremental compilation are very different. Large Rust projects are broken up into modules. If you change one module you don’t have recompile the whole project. Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn.
- majormajor 3mo ago
  > Remember we’re talking about compensation time versus LLM turns in this thread. Even a 10 second incremental compile is orders of magnitude faster than an LLM turn. This entirely depends on (a) what a step/prompt/turn is trying to do and (b) tooling and rate limiting and various other aspects of your model and/or access to it. If you have high rate limits and aren't using an Opus-or-larger sized model you can get a LOT of changes done in 10 seconds. And this is especially true for "oh not all the tests are passing yet, let me change..." iterations where the change attempts are often low-single-digit seconds.
embedding-shape 3mo ago
> I dont really like Python but the ability to make some changes then run it instantly is wild for me. You'll love Clojure, gets you even closer; you can make changes while your program is running, by changing source code then applying just that function you changed, while your editor is connected to your program and can show what the new results are.
- 7734128 3mo ago
  I'm quite often working in a Jupyter notebook where the imported modules are reloaded. I can have x in memory, change method y of X and then call x.y() and get the new definition. It feels almost criminal that it works, but it's quite nice.

insanitybit 3mo ago

I use LLMs and rust and compile times are absolutely a significant area of degradation. I love rust, I think it's the best language for LLMs, but the biggest win Rust could get for agentic development is to speed up the compiler.

embedding-shape 3mo ago
Neither of you talk about what sort of machine you're sitting on. Back when I used a netbook, I'd agree with you, for developing quick off programs, the compile times are horrible. But as someone on a workstation now, compilation times are the least of my problems, and disk space is more of a concern for me. You also don't share what LLM you use, some of them reason a lot, some of them nothing, some a bit. Again, personally I use LLMs with their maximum reasoning always, trading quality for speed/waiting every single time, and even compiling the Linux kernel would be faster than most LLM responses I get nowadays, for me, on a workstation.
- insanitybit 3mo ago
  I'm on an M4 Macbook. I use Codex, gpt, and Claude in various configurations, including Fable, xhigh, medium, high, ultracode, etc.
dnautics 3mo ago
> I think it's the best language for LLMs because of the type system? i work in ~dynamically typed language and the llms almost never make a type error.
- insanitybit 3mo ago
  The type system is great and gives extremely fast feedback. The performance wins are also incredible, it just feels like you're throwing 100s of megabytes of RAM away and 10x latency when you don't choose rust now, and for little reason. The testing tooling is really solid - property testing, fuzzing, mutation testing, etc, is all easy to use. Code structure with nice abstractions that don't become spaghetti is nice (traits, enums, etc). Static binaries as a production artifact is really nice. Supply chain story is pretty solid with cargo-vet. It's hard to come up with anything where I think Rust isn't "best in class" other than compile times.