4 ms·

It would have been worthwhile for them to mention why not use Java. I mean, grep for Java and JVM and that short blog lights up! It was good that they did stres

It would have been worthwhile for them to mention why not use Java. I mean, grep for Java and JVM and that short blog lights up! It was good that they did stress the (very) small team size for the product as well. This was an aesthetic choice and I wish he had stressed that. Java remains an excellent choice for concurrent systems, with a commonly accepted static type system, and all "java.util.concurrent" and the rest of JVM are native and the language and tools do scale to even monstrous sized "teams".

aphyr 3y ago
I've been a Java programmer for ~18 years! A nontrivial portion of Jepsen is written in Java too. :-) Many of the points I touched on in the post are direct critiques of Java, but maybe I should be explicit. An obvious factor is size--when I've written the same project in both languages, Clojure usually winds up 5-10 times smaller. That's definitely improving as Java adds streams, lambda expressions, and so on, but it's still a good deal more verbose. That gets in the way of the kind of exploratory programming I do, especially at the REPL, and it's harder for me to read and maintain a giant codebase. I appreciate having fewer nouns, in general: Clojure emphasizes a uniform way of traversing and transforming data structures, and in Java every single class has its own way of representing its data, and many of those ways are bad. The standard collections library is full of mutability and thread safety pitfalls. Printed representations of datatypes are verbose. It's not particularly good at dealing with highly polymorphic data structure transformation, and it's not a particularly good static type system either--a good part of my Java Brain (TM) is devoted to knowing the mechanisms of erasure, memory layout, and when unsafe casts are actually the right choice. Functions are sort of becoming first-class with method refs and functional interfaces, but it's nowhere near the convenience and flexibility of a Lisp-1. Accessing the compiler at runtime is a pain in the ass. No hygenic macro system makes things like custom iteration or compositional error handling a pain. There's no analogue to Clojure's protocols, which are a fantastic tool for "Hey, compiler, I want a monomorphic-cached type-dispatch polymorphic call site here for a type hierarchy I don't control". Etc etc. There are things I love about Java. Automated refactoring is easier, and I generally like the level of IDE support. Nearly anything involving primitives, I drop to Java. Ditto, APIs that require annotations. Interface specification is more rigorous. Etc etc. That's why I write in both languages. But most of Jepsen is in Clojure for good reasons. :-)
- Capricorn2481 3y ago
  > Nearly anything involving primitives, I drop to Java Can you describe what you mean by this? Does this just mean you're using native Java data types sometimes when speed is a concern? Is there an example somewhere in Jepsen?
  - aphyr 3y ago
    Speaking very loosely, primitives on the JVM are values which are represented directly in memory, instead of as pointers to objects on the heap. Clojure (again, very loosely) generally treats everything as a pointer to a heap object. There is no specialized equivalent for, say, a vector of shorts, or a map where values are floats. The compiler can emit specialized function signatures for... IIRC longs and doubles, but other types (e.g. byte, float) aren't directly accessible--they go through widening conversion. It's also easy for the compiler to quietly fail to recognize it can preserve primitives in some kinds of loops, so you wind up with what Java calls "autoboxing": wrapping a primitive in a corresponding Object type. Here's a recent example of some code in a hot path inside Elle, one of Jepsen's safety checkers. It does a lot in primitives, using packed structures and bitmasks to avoid pointer chasing. https://github.com/jepsen-io/elle/blob/main/src/elle/BFSPath.java https://github.com/jepsen-io/elle/blob/main/src/elle/BFSPath... There was actually a Clojure version of this earlier that got pretty close perf-wise, but I wound up dropping to Java for it instead: https://github.com/jepsen-io/elle/blob/913cbff5ebb19ba850c0a06c7897b5ff0a3cd624/src/elle/bfs.clj#L38-L252 https://github.com/jepsen-io/elle/blob/913cbff5ebb19ba850c0a...
    - Capricorn2481 3y ago
      How often is this necessary? I haven't been able to make an example of Java code performing faster than Clojure. I tried to make the java equivalent of this in Clojure ``` (defn sum-clojure [size] (reduce + 0 (range 0 size))) (sum-clojure 100000000) ``` Despite the fact that Clojure primitives are boxed, a manually constructed long array that was summed together in Java was much slower. Why is that?
      - 3y ago

aphyr 3y ago

I've been a Java programmer for ~18 years! A nontrivial portion of Jepsen is written in Java too. :-) Many of the points I touched on in the post are direct critiques of Java, but maybe I should be explicit. An obvious factor is size--when I've written the same project in both languages, Clojure usually winds up 5-10 times smaller. That's definitely improving as Java adds streams, lambda expressions, and so on, but it's still a good deal more verbose. That gets in the way of the kind of exploratory programming I do, especially at the REPL, and it's harder for me to read and maintain a giant codebase. I appreciate having fewer nouns, in general: Clojure emphasizes a uniform way of traversing and transforming data structures, and in Java every single class has its own way of representing its data, and many of those ways are bad. The standard collections library is full of mutability and thread safety pitfalls. Printed representations of datatypes are verbose. It's not particularly good at dealing with highly polymorphic data structure transformation, and it's not a particularly good static type system either--a good part of my Java Brain (TM) is devoted to knowing the mechanisms of erasure, memory layout, and when unsafe casts are actually the right choice. Functions are sort of becoming first-class with method refs and functional interfaces, but it's nowhere near the convenience and flexibility of a Lisp-1. Accessing the compiler at runtime is a pain in the ass. No hygenic macro system makes things like custom iteration or compositional error handling a pain. There's no analogue to Clojure's protocols, which are a fantastic tool for "Hey, compiler, I want a monomorphic-cached type-dispatch polymorphic call site here for a type hierarchy I don't control". Etc etc. There are things I love about Java. Automated refactoring is easier, and I generally like the level of IDE support. Nearly anything involving primitives, I drop to Java. Ditto, APIs that require annotations. Interface specification is more rigorous. Etc etc. That's why I write in both languages. But most of Jepsen is in Clojure for good reasons. :-)

Capricorn2481 3y ago
> Nearly anything involving primitives, I drop to Java Can you describe what you mean by this? Does this just mean you're using native Java data types sometimes when speed is a concern? Is there an example somewhere in Jepsen?
- aphyr 3y ago
  Speaking very loosely, primitives on the JVM are values which are represented directly in memory, instead of as pointers to objects on the heap. Clojure (again, very loosely) generally treats everything as a pointer to a heap object. There is no specialized equivalent for, say, a vector of shorts, or a map where values are floats. The compiler can emit specialized function signatures for... IIRC longs and doubles, but other types (e.g. byte, float) aren't directly accessible--they go through widening conversion. It's also easy for the compiler to quietly fail to recognize it can preserve primitives in some kinds of loops, so you wind up with what Java calls "autoboxing": wrapping a primitive in a corresponding Object type. Here's a recent example of some code in a hot path inside Elle, one of Jepsen's safety checkers. It does a lot in primitives, using packed structures and bitmasks to avoid pointer chasing. https://github.com/jepsen-io/elle/blob/main/src/elle/BFSPath.java https://github.com/jepsen-io/elle/blob/main/src/elle/BFSPath... There was actually a Clojure version of this earlier that got pretty close perf-wise, but I wound up dropping to Java for it instead: https://github.com/jepsen-io/elle/blob/913cbff5ebb19ba850c0a06c7897b5ff0a3cd624/src/elle/bfs.clj#L38-L252 https://github.com/jepsen-io/elle/blob/913cbff5ebb19ba850c0a...
  - Capricorn2481 3y ago
    How often is this necessary? I haven't been able to make an example of Java code performing faster than Clojure. I tried to make the java equivalent of this in Clojure ``` (defn sum-clojure [size] (reduce + 0 (range 0 size))) (sum-clojure 100000000) ``` Despite the fact that Clojure primitives are boxed, a manually constructed long array that was summed together in Java was much slower. Why is that?
    - 3y ago