5 ms·

Is the trend mainly due to ZK being written in Java?

tylerhannan 3y ago
TBH, I don't think so. I mean, I have worked and, and been guilty of tooling driven development (RiiR anyone?) . But, also, in a comment below Alexey shares many of the reasons other than language. I think Oxia does a good job of sharing their approach in - https://github.com/streamnative/oxia/blob/main/docs/design-goals.md https://github.com/streamnative/oxia/blob/main/docs/design-g... (Alexey's comment, FYI, https://news.ycombinator.com/item?id=37677324 https://news.ycombinator.com/item?id=37677324)
Xeoncross 3y ago
It's not the written in Java part, it's the running in the JVM that's the issue. Memory hungry is what I think of when I think Java apps. I'd much rather have a low memory Go or Rust service.
- adra 3y ago
  I mean java native compilation is a thing. I imagine you'd get the majority of the lift via that route instead of reinventing the wheel, but it's possible there are caveats to that route for these impls. Even java native/golang have GC heaps, but in general ZK is really low GC churn so I don't see that being the impetus to rewrite. My guess if anything is that people always complain about ZK being annoying as a second piece of infra to distribute for simple setups, so my guess this is just a prelude to them embedding their keeper into the DB deployment itself which is the same general strategy Kafka is taking (a verrrry loong time) to rollout.
- dathinab 3y ago
  but if you use ZK for what it was designed for (basic cluster role/naming cordination, distributing configs to cluster nodes and similar) then this kind doesn't matter I mean in a typical use case of it you would - run it on long running nodes (e.g. not lambda or spot instances) - run more or less exactly 3 nodes up to quite a cluster size, I guess some use-cases which involve a lot of serverless might need more - configs tend to not change "that" much nor are they "that" big what this means is - java needing time to run-hot (JIT optimize) is not an issue for it - GC isn't an issue and if you look at how much memory (RAM) typical minimal nodes in the cloud have it in context of typical config sizes and cluster sizes is also not an issue through I guess depending what you want to do there could be issues if you use it - for squeezing through analytics or similar - setups with very very very large constantly changing clusters, e.g. in some serverless context with ton of ad-hoc spawned up instances, maybe using WASM and certain snapshot tricks allowing insane fast startup time - you want to bundle it directly into other applications, running on the same hardware and that applications need more memory but all of it are cases it wasn't designed for so I wouldn't call it an "alternative" but a ZooKeeper like service for different use-cases, I guess
  - StevePerkins 3y ago
    Long-running server processes are not only "not an issue" for the JVM, they're the main use case in which the JVM is superior to AOT compilation! Same is true for C# and the .NET CLR, by the way. If you're running a Lambda function in which startup time is extremely important, or an embedded application where size and resources are paramount, or even just a short-lived process where you don't care either way... then AOT makes a lot of sense. But for long-running server processes, just-in-time compilation almost always results is better performance than AOT compilation that cannot optimize at runtime based on what's actually happening. HN should be full of people who know better, but these discussions feel like piping information into /dev/null. Web devs, students and hobbyists, and other low-information voters just have it in their heads that AOT is always a superior model, and JIT always an inferior fallback, and there's nothing you can say to break through that. There aren't enough people from the business server side world who spend enough time in online discussion forums to correct the narrative.
    - neonsunset 3y ago
      I think there's an important distinction to be made: Theoretically speaking, given that JIT compilers generally care much more about compilation speed than AOT ones, it is not unreasonable to assume that AOT compilers ought to produce more optimized code. With that said, this is not the case for both JVM and .NET. Stepping away from ought to is, both have JIT compilers which produce better optimized code than their AOT counterparts, due to a variety of reasons including R&D effort done for JIT throughout their history and JIT allowing to dynamically profile and recompile the code according to its execution characteristics (.NET's Tier 1 PGO Optimized and HotSpot JVM's C2).
      - GeneralMayhem 3y ago
        Theoretically speaking, JITs have access to strictly more information than AOT, so they ought to be better once you amortize out the timing issues. A really good profiling JIT will do a fast-compile pass the first time through, and then progressively re-compile with knowledge gained from profiling during runtime.
- thinkmassive 3y ago
  If you need a Zookeeper drop-in replacement then I suppose there’s limited options, but many of the same needs could be met by etcd.

TBH, I don't think so. I mean, I have worked and, and been guilty of tooling driven development (RiiR anyone?) . But, also, in a comment below Alexey shares many of the reasons other than language. I think Oxia does a good job of sharing their approach in - https://github.com/streamnative/oxia/blob/main/docs/design-goals.md https://github.com/streamnative/oxia/blob/main/docs/design-g... (Alexey's comment, FYI, https://news.ycombinator.com/item?id=37677324 https://news.ycombinator.com/item?id=37677324)

Xeoncross 3y ago

It's not the written in Java part, it's the running in the JVM that's the issue. Memory hungry is what I think of when I think Java apps. I'd much rather have a low memory Go or Rust service.

adra 3y ago
I mean java native compilation is a thing. I imagine you'd get the majority of the lift via that route instead of reinventing the wheel, but it's possible there are caveats to that route for these impls. Even java native/golang have GC heaps, but in general ZK is really low GC churn so I don't see that being the impetus to rewrite. My guess if anything is that people always complain about ZK being annoying as a second piece of infra to distribute for simple setups, so my guess this is just a prelude to them embedding their keeper into the DB deployment itself which is the same general strategy Kafka is taking (a verrrry loong time) to rollout.
dathinab 3y ago
but if you use ZK for what it was designed for (basic cluster role/naming cordination, distributing configs to cluster nodes and similar) then this kind doesn't matter I mean in a typical use case of it you would - run it on long running nodes (e.g. not lambda or spot instances) - run more or less exactly 3 nodes up to quite a cluster size, I guess some use-cases which involve a lot of serverless might need more - configs tend to not change "that" much nor are they "that" big what this means is - java needing time to run-hot (JIT optimize) is not an issue for it - GC isn't an issue and if you look at how much memory (RAM) typical minimal nodes in the cloud have it in context of typical config sizes and cluster sizes is also not an issue through I guess depending what you want to do there could be issues if you use it - for squeezing through analytics or similar - setups with very very very large constantly changing clusters, e.g. in some serverless context with ton of ad-hoc spawned up instances, maybe using WASM and certain snapshot tricks allowing insane fast startup time - you want to bundle it directly into other applications, running on the same hardware and that applications need more memory but all of it are cases it wasn't designed for so I wouldn't call it an "alternative" but a ZooKeeper like service for different use-cases, I guess
- StevePerkins 3y ago
  Long-running server processes are not only "not an issue" for the JVM, they're the main use case in which the JVM is superior to AOT compilation! Same is true for C# and the .NET CLR, by the way. If you're running a Lambda function in which startup time is extremely important, or an embedded application where size and resources are paramount, or even just a short-lived process where you don't care either way... then AOT makes a lot of sense. But for long-running server processes, just-in-time compilation almost always results is better performance than AOT compilation that cannot optimize at runtime based on what's actually happening. HN should be full of people who know better, but these discussions feel like piping information into /dev/null. Web devs, students and hobbyists, and other low-information voters just have it in their heads that AOT is always a superior model, and JIT always an inferior fallback, and there's nothing you can say to break through that. There aren't enough people from the business server side world who spend enough time in online discussion forums to correct the narrative.
  - neonsunset 3y ago
    I think there's an important distinction to be made: Theoretically speaking, given that JIT compilers generally care much more about compilation speed than AOT ones, it is not unreasonable to assume that AOT compilers ought to produce more optimized code. With that said, this is not the case for both JVM and .NET. Stepping away from ought to is, both have JIT compilers which produce better optimized code than their AOT counterparts, due to a variety of reasons including R&D effort done for JIT throughout their history and JIT allowing to dynamically profile and recompile the code according to its execution characteristics (.NET's Tier 1 PGO Optimized and HotSpot JVM's C2).
    - GeneralMayhem 3y ago
      Theoretically speaking, JITs have access to strictly more information than AOT, so they ought to be better once you amortize out the timing issues. A really good profiling JIT will do a fast-compile pass the first time through, and then progressively re-compile with knowledge gained from profiling during runtime.
thinkmassive 3y ago
If you need a Zookeeper drop-in replacement then I suppose there’s limited options, but many of the same needs could be met by etcd.