4 ms·

The lack of proper non-Java support in the Hadoop world is maddening, and Zookeeper is a prime example of this. It's a heterogenous world, people! How many peop

by shykes 15y ago

The lack of proper non-Java support in the Hadoop world is maddening, and Zookeeper is a prime example of this. It's a heterogenous world, people! How many people are missing out on Zookeeper because it lacks proper Python or PHP support?

jswank 15y ago
Nothing to say about Hadoop/Java homogeneity, but a Zookeeper alternative I've been looking at for the same reasons is Dozerd from Heroku (https://github.com/ha https://github.com/ha).
riobard 15y ago
The thinking might be along the line: if you need to use Hadoop at all, you are probably already using Java anyway to maximize single machine performance.
- true_religion 15y ago
  Why Java? Why not C? Personally, I've never looked to Java as a top performer. It's a middle language---not as expressive as Python, not as fast as C. Even compared to C#, I wouldn't naturally choose Java but I've been forced to because C# wasn't well supported away from a windows' box. Now adays, I just skip the Java step for CPU intensive work and go to straight C integrated with Python.
  - jbellis 15y ago
    Your information is out of date. http://www.azulsystems.com/blog/cliff/2009-09-06-java-vs-c-performanceagain http://www.azulsystems.com/blog/cliff/2009-09-06-java-vs-c-p...
tikhonj 15y ago
Can't you use Hadoop with other JVM languages like Clojure or Scala? That was my impression, but I haven't used Hadoop for anything nontrivial or in any language but Java, so I don't really know.
csears 15y ago
Java is certainly the most common language ZK seems to be used with, but there are client bindings in Python, Perl and quite a few other languages: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClientBindings https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClie...
- nknight 15y ago
  The Python bindings are not very robust, don't fit the language well, and have a build process alien to the Python ecosystem. If it's representative of the other non-Java bindings, it's small wonder it doesn't get used by non-Java projects.
  - shykes 15y ago
    I second that. At dotCloud we had to pull away from Zookeeper because of the poor quality of the Python (and therefore C) bindings.
- pquerna 15y ago
  Yes, they are binding to a C client, include the existing node-zookeeper: https://github.com/yfinkelstein/node-zookeeper https://github.com/yfinkelstein/node-zookeeper The C client isn't what I would call great, and the API surface is fairly complicated for languages to bind to -- you see many reports about the bindings crashing their respective languages. I'm currently hacking on a pure-Node.js implementation of the Apache Jute RecrodIO and Apache Zookeeper network protocol: https://github.com/racker/node-zk https://github.com/racker/node-zk The core problem is that Zookeeper is using a deprecated serialization layer as its network protocol. Jute predates Apache Thrift or Apache Avro, and it only has code-gen for C and Java. Ideally Zookeeper would add a second network protocol, one which was either a spec for a nice binary protocol, or a more modern serialization framework like Thrift or Avro, which support many more languages. The ticket to change out Jute is here: https://issues.apache.org/jira/browse/ZOOKEEPER-102 https://issues.apache.org/jira/browse/ZOOKEEPER-102 But no one seems to actively be working on it.

by shykes 15y ago

jswank 15y ago

Nothing to say about Hadoop/Java homogeneity, but a Zookeeper alternative I've been looking at for the same reasons is Dozerd from Heroku (https://github.com/ha https://github.com/ha).

riobard 15y ago

The thinking might be along the line: if you need to use Hadoop at all, you are probably already using Java anyway to maximize single machine performance.

true_religion 15y ago
Why Java? Why not C? Personally, I've never looked to Java as a top performer. It's a middle language---not as expressive as Python, not as fast as C. Even compared to C#, I wouldn't naturally choose Java but I've been forced to because C# wasn't well supported away from a windows' box. Now adays, I just skip the Java step for CPU intensive work and go to straight C integrated with Python.
- jbellis 15y ago
  Your information is out of date. http://www.azulsystems.com/blog/cliff/2009-09-06-java-vs-c-performanceagain http://www.azulsystems.com/blog/cliff/2009-09-06-java-vs-c-p...

tikhonj 15y ago

Can't you use Hadoop with other JVM languages like Clojure or Scala? That was my impression, but I haven't used Hadoop for anything nontrivial or in any language but Java, so I don't really know.

csears 15y ago

Java is certainly the most common language ZK seems to be used with, but there are client bindings in Python, Perl and quite a few other languages: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClientBindings https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClie...

nknight 15y ago
The Python bindings are not very robust, don't fit the language well, and have a build process alien to the Python ecosystem. If it's representative of the other non-Java bindings, it's small wonder it doesn't get used by non-Java projects.
- shykes 15y ago
  I second that. At dotCloud we had to pull away from Zookeeper because of the poor quality of the Python (and therefore C) bindings.
pquerna 15y ago
Yes, they are binding to a C client, include the existing node-zookeeper: https://github.com/yfinkelstein/node-zookeeper https://github.com/yfinkelstein/node-zookeeper The C client isn't what I would call great, and the API surface is fairly complicated for languages to bind to -- you see many reports about the bindings crashing their respective languages. I'm currently hacking on a pure-Node.js implementation of the Apache Jute RecrodIO and Apache Zookeeper network protocol: https://github.com/racker/node-zk https://github.com/racker/node-zk The core problem is that Zookeeper is using a deprecated serialization layer as its network protocol. Jute predates Apache Thrift or Apache Avro, and it only has code-gen for C and Java. Ideally Zookeeper would add a second network protocol, one which was either a spec for a nice binary protocol, or a more modern serialization framework like Thrift or Avro, which support many more languages. The ticket to change out Jute is here: https://issues.apache.org/jira/browse/ZOOKEEPER-102 https://issues.apache.org/jira/browse/ZOOKEEPER-102 But no one seems to actively be working on it.