8 ms·

Clojure Protocols can extend final Java classes (like java.lang.String)

76 points by swannodette 17y ago

swannodette 17y ago
As simple as this: (defprotocol Foo (bar-ify [this])) (extend-type java.lang.String Foo (bar-ify [this] (.concat this "Bar"))) (bar-ify "Hello world!") Everything can be redefined interactively at the REPL. Also due to Rich Hickey's mad skills you get pretty much Java method dispatch performance. EDIT: There is a gross misunderstanding about how this works. The following is invalid and will throw an exception, please take the time to understand this: user> (ns baz) baz> (bar-ify "foo") Your changes will not affect use of java.lang.String in other libs.
- chousuke 17y ago
  I'd like to clarify that Protocols just generate a bunch of functions and a Protocol object (and an interface, but that's an implementation/interop detail). The functions are the interesting things in most cases. However, they're also namespaced like everything else. You can do the following though: user> (defprotocol Foo (describe [object])) user> (extend-type java.lang.String Foo (describe [string] "a String")) (ns bar) bar> (user/describe "String") -> "a String" bar> (extend-type java.lang.Object user/Foo (describe [obj] "an Object")) bar> (in-ns 'user) user> (describe 3) -> "an Object" user> (describe "Hello") -> "a String" So there is potential for two libraries extending the same things and causing a conflict if the semantics differ, but I'm not too concerned about that. It would be rather rude to extend someone else's protocol to cover someone else's java class without documenting that you do so.
zachbeane 17y ago
Reminds me of jwz's Java doc on the topic of final variables, and how the access is restricted by compiler convention, not in the JVM: System.in, out and err (the stdio streams) are all final variables. They didn't used to be, but some clever applet-writer realized that you could change them and start intercepting all output and do all sorts of nasty stuff. So, the whip-smart folks at Sun went and made them final. But hey! Sometimes it's okay to change them! So, they also added System.setIn, setOut, and setErr methods to change them! ``Change a final variable?!'' I hear you cry. Yep. They sneak in through native code and change finals now. You might think it'd give 'em pause to think and realize that other people might also want to have public read-only yet privately writable variables, but no. Oh, but it gets even better: it turns out they didn't really have to sneak in through native code anyway, at least as far as the JVM is concerned, since the JVM treats final variables as always writable to the class they're defined in! There's no special case for constructors: they're just always writable. The javac compiler, on the other hand, pretends that they're only assignable once, either in static init code for static finals or once per constructor for instance variables. It also will optimize access to finals, despite the fact that it's actually unsafe to do so.
- bobbyi 17y ago
  > some clever applet-writer realized that you could change them and start intercepting all output and do all sorts of nasty stuff. Can someone explain how this worked? What does he mean by "intercepting all output"?
  - btilly 17y ago
    Replace System.out with an object that responds to all of the same methods, but which does something clever with them. For instance one that adds timestamps on output, logs stuff somewhere, filters, etc. This is a very powerful trick. However the downside is that it is in the category of things where only one person gets to be clever. Because the second person to replace that object breaks the first. So they decided to discourage that.
mquander 17y ago
Why's this worth noting? There's no reason you shouldn't be able to extend a type with methods to support a new interface, no matter what that type's author intended. It's not in any sense "dangerous" to existing APIs, as changing final variables could potentially be. It seems like completely natural behavior.
- KirinDave 17y ago
  Maybe so, but it's not trivial to do in java. It's noteworthy because it gives clojure a lot more power when dealing with native libraries.
  - lukev 17y ago
    libs you mean. Native libs. :p
    - KirinDave 17y ago
      iPad sad trombone moment. ;)
- pohl 17y ago
  It's not in any sense "dangerous" to existing APIs If your job is to deliver and maintain such an API, then by allowing subclassing you have a larger contract you have to maintain when you want to make changes to the underlying class: rather than just the public interface, you also need to worry about the protected interface. If your API is new, you may want to shepherd your users (by marking a class as final) to favor composition over inheritance to maximize your ability to evolve the implementation in ways that don't break.
  - mquander 17y ago
    Ah, that's true, if extend can see protected members. I didn't realize that was the case. EDIT: I'm not sure if it is the case or not; I can't find definitive documentation and I don't have a REPL available. Anyone know for sure?
xtho 17y ago
Does is actually extend the class or simply wrap it with a decorator? I think Groovy takes this approach but I could be wrong.
- chousuke 17y ago
  It does neither. The dispatch logic is on the protocol side, which can be modified. When a class is extended, the protocol function is changed to have an extra instance check. (I don't understand the details in full, though. You'll need to ask Rich Hickey) AFAIK there is also some kind of caching involved to improve performance.

76 points by swannodette 17y ago

As simple as this: (defprotocol Foo (bar-ify [this])) (extend-type java.lang.String Foo (bar-ify [this] (.concat this "Bar"))) (bar-ify "Hello world!") Everything can be redefined interactively at the REPL. Also due to Rich Hickey's mad skills you get pretty much Java method dispatch performance. EDIT: There is a gross misunderstanding about how this works. The following is invalid and will throw an exception, please take the time to understand this: user> (ns baz) baz> (bar-ify "foo") Your changes will not affect use of java.lang.String in other libs.

chousuke 17y ago
I'd like to clarify that Protocols just generate a bunch of functions and a Protocol object (and an interface, but that's an implementation/interop detail). The functions are the interesting things in most cases. However, they're also namespaced like everything else. You can do the following though: user> (defprotocol Foo (describe [object])) user> (extend-type java.lang.String Foo (describe [string] "a String")) (ns bar) bar> (user/describe "String") -> "a String" bar> (extend-type java.lang.Object user/Foo (describe [obj] "an Object")) bar> (in-ns 'user) user> (describe 3) -> "an Object" user> (describe "Hello") -> "a String" So there is potential for two libraries extending the same things and causing a conflict if the semantics differ, but I'm not too concerned about that. It would be rather rude to extend someone else's protocol to cover someone else's java class without documenting that you do so.

zachbeane 17y ago

Reminds me of jwz's Java doc on the topic of final variables, and how the access is restricted by compiler convention, not in the JVM: System.in, out and err (the stdio streams) are all final variables. They didn't used to be, but some clever applet-writer realized that you could change them and start intercepting all output and do all sorts of nasty stuff. So, the whip-smart folks at Sun went and made them final. But hey! Sometimes it's okay to change them! So, they also added System.setIn, setOut, and setErr methods to change them! ``Change a final variable?!'' I hear you cry. Yep. They sneak in through native code and change finals now. You might think it'd give 'em pause to think and realize that other people might also want to have public read-only yet privately writable variables, but no. Oh, but it gets even better: it turns out they didn't really have to sneak in through native code anyway, at least as far as the JVM is concerned, since the JVM treats final variables as always writable to the class they're defined in! There's no special case for constructors: they're just always writable. The javac compiler, on the other hand, pretends that they're only assignable once, either in static init code for static finals or once per constructor for instance variables. It also will optimize access to finals, despite the fact that it's actually unsafe to do so.

bobbyi 17y ago
> some clever applet-writer realized that you could change them and start intercepting all output and do all sorts of nasty stuff. Can someone explain how this worked? What does he mean by "intercepting all output"?
- btilly 17y ago
  Replace System.out with an object that responds to all of the same methods, but which does something clever with them. For instance one that adds timestamps on output, logs stuff somewhere, filters, etc. This is a very powerful trick. However the downside is that it is in the category of things where only one person gets to be clever. Because the second person to replace that object breaks the first. So they decided to discourage that.

mquander 17y ago

Why's this worth noting? There's no reason you shouldn't be able to extend a type with methods to support a new interface, no matter what that type's author intended. It's not in any sense "dangerous" to existing APIs, as changing final variables could potentially be. It seems like completely natural behavior.

KirinDave 17y ago
Maybe so, but it's not trivial to do in java. It's noteworthy because it gives clojure a lot more power when dealing with native libraries.
- lukev 17y ago
  libs you mean. Native libs. :p
  - KirinDave 17y ago
    iPad sad trombone moment. ;)
pohl 17y ago
It's not in any sense "dangerous" to existing APIs If your job is to deliver and maintain such an API, then by allowing subclassing you have a larger contract you have to maintain when you want to make changes to the underlying class: rather than just the public interface, you also need to worry about the protected interface. If your API is new, you may want to shepherd your users (by marking a class as final) to favor composition over inheritance to maximize your ability to evolve the implementation in ways that don't break.
- mquander 17y ago
  Ah, that's true, if extend can see protected members. I didn't realize that was the case. EDIT: I'm not sure if it is the case or not; I can't find definitive documentation and I don't have a REPL available. Anyone know for sure?

xtho 17y ago

Does is actually extend the class or simply wrap it with a decorator? I think Groovy takes this approach but I could be wrong.

chousuke 17y ago
It does neither. The dispatch logic is on the protocol side, which can be modified. When a class is extended, the protocol function is changed to have an extra instance check. (I don't understand the details in full, though. You'll need to ask Rich Hickey) AFAIK there is also some kind of caching involved to improve performance.