4 ms·

For one, Java programs always take too long to start, and that's not something you want in a program that's supposed to be interactive.

jeffbee 1y ago
See, that's exactly the type of uncritical statement that will force me to discount the rest of your statements on a whole range of topics. bazelisk help 0.01s user 0.00s system 26% cpu 0.053 total
- lostdog 1y ago
  Takes 1.97s on my machine, which is a lot of time for just showing a help message. Maybe you have the setup where Bazel is staying alive in the background or something?
  - jeffbee 1y ago
    Literally everyone who uses bazel uses it client-server as a persistent daemon.
    - lostdog 1y ago
      And part of why it's a persistent daemon is because Java is slow to start up. I'll note that bazelisk is written in Go.
      - jeffbee 1y ago
        Exactly. And a Go program that forks a Java program that makes an RPC is still faster than `cargo help`
    - sunshowers 1y ago
      Demonization introduces a range of potential cache invalidation issues. The issues are solvable, but whose KPIs depend on getting to the bottom of them?
      - joshuamorton 1y ago
        Do you have specific examples in the context of blaze/bazel here? I think "the set of cache invalidation issues" you're describing are basically "the set of cache invalidation issues blaze/bazel intends to solve", so the answer to "whose KPIs" is "the blaze team".
        sunshowers 1y ago
        I haven't used Bazel but I have used buck1 extensively, and the daemonized mode was quite buggy and often required the process to be killed. Quite frequently the process was just wedged, and even when it wasn't, memory leaks were quite common. Standard black box debuggers like strace and DTrace also become harder to use (e.g. you need to attribute a particular set of syscalls to a client, and a mindless "strace buck build ..." doesn't do what you want). Daemonization is sometimes necessary, but it introduces lifecycle management problems that get in the way of robustness. Daemonization simply because your choice of programming language has bad startup times doesn't seem like a great idea to me. I think on-disk cache invalidation and in-memory cache invalidation are distinctly different in practice.
        jeffbee 1y ago
        > I haven't used Bazel That was the perfect place to end the comment.
        sunshowers 1y ago
        Are you saying that daemonization doesn't make black box debugging harder?
        joshuamorton 1y ago
        I asked this because I've used bazel some, and blaze fairly extensively, and despite having done some deeply cursed things to blaze at times, I've never had issues with it as a daemon, to the point where I'd never consider needing to run it in non-daemonized mode as part of a debug process. It just works. Second, "startup-time" is probably the least relevant complaint here in terms of why to daemonize. Running `help` without a daemon set up does take a few seconds (and yeah that's gross), but constructing the action graph cold for $arbitrary_expensive_thing takes a minute or two the first time I do it, and then around 1 second the next time. Caching the build graph across builds is valuable, and persisting it in memory makes a lot more sense than on disk, in this case. The article we're discussing even argues in favor of making this even more extreme and moving action graph calculation entirely into a service on another machine because it prevents the situation where the daemon dies and you lose your (extremely valuable!) cached analysis graph: > Bonanza performs analysis remotely. When traditional Bazel is configured to execute all actions remotely, the Bazel server process is essentially a driver that constructs and walks a graph of nodes. This in-memory graph is known as Skyframe and is used to represent and execute a Bazel build. Bonanza lifts the same graph theory from the Bazel server process, puts it into a remote cluster, and relies on a distributed persistent cache to store the graph’s nodes. The consequence of storing the graph in a distributed storage system is that, all of a sudden, all builds become incremental. There is no more “cold build” effect like the one you see with Bazel when you lose the analysis cache. If you're worried about cache invalidation and correctness issues due to daemonization, I think you'd want to be even more concerned about moving them entirely off machine. (I'm also not sure how their proposal intends to manage e.g. me and Phil on another machine trying to build conflicting changes that both significantly impact how the analysis graph is calculated and thrash each other, either you have to namespace so they don't thrash and then you've moved the daemon to the cloud, or you do some very very fancy partial graph invalidation approach, but that isn't discussed and it feels like it would be white paper worthy)
aseipp 1y ago
I don't disagree in general but in Bazel's case this path has been heavily optimized. Maybe there are limits to it but "java startup slow" is a 101-level complaint. In fact, I don't even think the client program for Bazel is written in Java, but C++, and the Java daemon it talks to is started when you first attach to a Bazel workspace and it persists, so subsequent interactions are very fast. Just running `bazel` randomly is not truly indicative of what using it feels like, because that's not what people actually use it for. People use it inside a build workspace, that is the case that matters. Beyond that, other build systems like Buck2 (Rust) also use the client-daemon architecture for a number of other reasons, including the fact that actually-really-large build graphs are far too large to rebuild on every invocation, so the daemon is necessary to keep the build graph and interactively analyze and incrementally invalidate it on demand. Doesn't matter if it's Rust or Java, these things are architectural. That's also one of the points of the article, of course, and why they're theorizing about server-side analysis caches and graphs, etc. This all indicates to me that the people designing these systems actually do care about interactivity of their tool. It is not a matter of "java startup slow" to use a client-server design, though it certainly is known to help.

jeffbee 1y ago

See, that's exactly the type of uncritical statement that will force me to discount the rest of your statements on a whole range of topics. bazelisk help 0.01s user 0.00s system 26% cpu 0.053 total

lostdog 1y ago
Takes 1.97s on my machine, which is a lot of time for just showing a help message. Maybe you have the setup where Bazel is staying alive in the background or something?
- jeffbee 1y ago
  Literally everyone who uses bazel uses it client-server as a persistent daemon.
  - lostdog 1y ago
    And part of why it's a persistent daemon is because Java is slow to start up. I'll note that bazelisk is written in Go.
    - jeffbee 1y ago
      Exactly. And a Go program that forks a Java program that makes an RPC is still faster than `cargo help`
  - sunshowers 1y ago
    Demonization introduces a range of potential cache invalidation issues. The issues are solvable, but whose KPIs depend on getting to the bottom of them?
    - joshuamorton 1y ago
      Do you have specific examples in the context of blaze/bazel here? I think "the set of cache invalidation issues" you're describing are basically "the set of cache invalidation issues blaze/bazel intends to solve", so the answer to "whose KPIs" is "the blaze team".
      - sunshowers 1y ago
        I haven't used Bazel but I have used buck1 extensively, and the daemonized mode was quite buggy and often required the process to be killed. Quite frequently the process was just wedged, and even when it wasn't, memory leaks were quite common. Standard black box debuggers like strace and DTrace also become harder to use (e.g. you need to attribute a particular set of syscalls to a client, and a mindless "strace buck build ..." doesn't do what you want). Daemonization is sometimes necessary, but it introduces lifecycle management problems that get in the way of robustness. Daemonization simply because your choice of programming language has bad startup times doesn't seem like a great idea to me. I think on-disk cache invalidation and in-memory cache invalidation are distinctly different in practice.
        jeffbee 1y ago
        > I haven't used Bazel That was the perfect place to end the comment.
        sunshowers 1y ago
        Are you saying that daemonization doesn't make black box debugging harder?
        joshuamorton 1y ago
        I asked this because I've used bazel some, and blaze fairly extensively, and despite having done some deeply cursed things to blaze at times, I've never had issues with it as a daemon, to the point where I'd never consider needing to run it in non-daemonized mode as part of a debug process. It just works. Second, "startup-time" is probably the least relevant complaint here in terms of why to daemonize. Running `help` without a daemon set up does take a few seconds (and yeah that's gross), but constructing the action graph cold for $arbitrary_expensive_thing takes a minute or two the first time I do it, and then around 1 second the next time. Caching the build graph across builds is valuable, and persisting it in memory makes a lot more sense than on disk, in this case. The article we're discussing even argues in favor of making this even more extreme and moving action graph calculation entirely into a service on another machine because it prevents the situation where the daemon dies and you lose your (extremely valuable!) cached analysis graph: > Bonanza performs analysis remotely. When traditional Bazel is configured to execute all actions remotely, the Bazel server process is essentially a driver that constructs and walks a graph of nodes. This in-memory graph is known as Skyframe and is used to represent and execute a Bazel build. Bonanza lifts the same graph theory from the Bazel server process, puts it into a remote cluster, and relies on a distributed persistent cache to store the graph’s nodes. The consequence of storing the graph in a distributed storage system is that, all of a sudden, all builds become incremental. There is no more “cold build” effect like the one you see with Bazel when you lose the analysis cache. If you're worried about cache invalidation and correctness issues due to daemonization, I think you'd want to be even more concerned about moving them entirely off machine. (I'm also not sure how their proposal intends to manage e.g. me and Phil on another machine trying to build conflicting changes that both significantly impact how the analysis graph is calculated and thrash each other, either you have to namespace so they don't thrash and then you've moved the daemon to the cloud, or you do some very very fancy partial graph invalidation approach, but that isn't discussed and it feels like it would be white paper worthy)

aseipp 1y ago

I don't disagree in general but in Bazel's case this path has been heavily optimized. Maybe there are limits to it but "java startup slow" is a 101-level complaint. In fact, I don't even think the client program for Bazel is written in Java, but C++, and the Java daemon it talks to is started when you first attach to a Bazel workspace and it persists, so subsequent interactions are very fast. Just running `bazel` randomly is not truly indicative of what using it feels like, because that's not what people actually use it for. People use it inside a build workspace, that is the case that matters. Beyond that, other build systems like Buck2 (Rust) also use the client-daemon architecture for a number of other reasons, including the fact that actually-really-large build graphs are far too large to rebuild on every invocation, so the daemon is necessary to keep the build graph and interactively analyze and incrementally invalidate it on demand. Doesn't matter if it's Rust or Java, these things are architectural. That's also one of the points of the article, of course, and why they're theorizing about server-side analysis caches and graphs, etc. This all indicates to me that the people designing these systems actually do care about interactivity of their tool. It is not a matter of "java startup slow" to use a client-server design, though it certainly is known to help.