8 ms·

Nuances of overloading and overriding in Java

87 points by whack 3y ago

lern_too_spel 3y ago
None of this is surprising. Just think about how the compiler is going to use the object's vtable. There is no magic going on.
- alserio 3y ago
  I believe that OP is confused about when the resolution happens. Overloading is resolved by the compiler, inheritance at runtime
  - marginalia_nu 3y ago
    Kind of a blurry distinction with a JIT-compiled languague tbh.
    - alserio 3y ago
      why? the JIT works on bytecode and JVM bytecode has no concept of overloading. Maybe I should have specified that by "compiler" i meant the first compilation step, Java source to bytecode.
      - hinkley 3y ago
        The JVM bytecode is not the JVM. There’s some late binding allowed in the JVM to deal with API changes. So the caller can call a method signature that doesn’t exist in the target (but either does in a later version or did in a previous one), and the JVM has to resolve it at first invocation, rewriting the bytecode in the process. The compile time resolution ends up being a hint that usually works, but doesn’t always. And if memory serves a number of languages that run on top of the JVM have leveraged this fact, including the original third party implementation of generics.
        alserio 3y ago
        still the lookup is based on the symbol, no? so the JVM does not resolve overload even in that case, i believe
        hinkley 3y ago
        Read what I said again. Step 1 is a lookup by symbol. You're ignoring the vast majority of the logic. There are hints about what I'm talking about in this conversation about invokespecial: https://stackoverflow.com/questions/13764238/why-invokespecial-is-needed-when-invokevirtual-exists https://stackoverflow.com/questions/13764238/why-invokespeci... This trickiness is not called out in the bytecode specification, but in the JVM spec. They get a little closer to the mark here: https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-6.html#jvms-6.5.invokespecial https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-6.ht... "Is Java Statically Typed or Dynamically Typed?" is basically an interview gotcha question. Most people should answer the former, but if you are hiring people to create a JVM or a language that translates to the JVM, you're looking for a much longer answer that circles around gradations of the latter, because the compiler is strongly typed but the JVM has weaker type guarantees that are sorted out at first invocation, and which look like variance (predating most literature on contra and covariance, I might add). Prior to invokedynamic people used this and other facts to trick the JVM into doing things that are illegal in Java. The Hotspot people learned a whole lot about how the JVM actually functions from interactions with these sorts of people.
        alserio 3y ago
        i now get what you mean, thanks
        hinkley 3y ago
        If I wasn't already clear, this isn't something that normal devs should have or want to think about. But it's hard to avoid if you end up dealing with separate compilation units that are run on a different build schedule. Sooner or later your refactoring will either blow up with an obscure error, or work even though if you really think about it, the compiler would say it shouldn't. And of course if you actually dig you'll find it out as well. If you want sophisticated dispatch you want to use invokedynamic, but that was an RFE when I was learning about this stuff.
- jchw 3y ago
  I think the main confusion is that when you override a method in Java, similar to other languages like C++, it considers not only the name but actually the entire function signature. That's in fact why overloading works to begin with, because of the fact that the identity of a method is not just its name but in fact it's signature.
- Jenk 3y ago
  I rather suspect that "magic" and "vtable" are very similar concepts to many developers.
yotamoron 3y ago
I hate inheritance.
- nforgerit 3y ago
  Some months ago, I stumbled into an OOP inheritance rabbithole which got me thinking the same: - Isn't the (biological) concept of inheritance built at it's core around the idea of "generations"? How does that make any sense in OOP? - Why do Java beginner courses still teach Inheritance like it was 1995 with all the bells and whistles and dogs and cats and mammals? - Why do they teach Inheritance and later introduce "favor composition to inheritance"? Doesn't this just confuse everyone? - Why does Inheritance have first-class syntax support in an OOP language like Java ("extends") whereas Composition usually needs to be "engineered" using some more complex patterns, e.g. DI? Most of it is probably for historical reasons, i guess.
  - psychoslave 3y ago
    I don't know for Java currently but for example in Ruby composition is simply a matter of putting your code in a module rather than a class. Then you can extend any class with this module. You can even extend instances. That is 3.extends(Some_module).method_from_some_module is perfectly valid. In PHP, surely you can use traits.
    - emodendroket 3y ago
      Of the many models of “composition” that are possible I think Ruby’s free-for-all blend of mix-ins and monkey patches is the only one that can drive a maintenance programmer more insane than a deep hierarchy of inheritance of Java classes.
      - shagie 3y ago
        irb(main):001:0> "bar".foo Traceback (most recent call last): 4: from /usr/bin/irb:23:in `<main>' 3: from /usr/bin/irb:23:in `load' 2: from /Library/Ruby/Gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>' 1: from (irb):1 NoMethodError (undefined method `foo' for "bar":String) Did you mean? for irb(main):002:0> class String irb(main):003:1> def foo irb(main):004:2> "foobar!" irb(main):005:2> end irb(main):006:1> end => :foo irb(main):007:0> "bar".foo => "foobar!" irb(main):008:0> Yes! Let's modify the core library String class on the fly to add new functions to it. There are things about ruby that truly scare me if my goal was to write secure and reasonable code.
bvanderveen 3y ago
Just say no.
invalidname 3y ago
Are these things HN readers didn't know? I didn't know you can do operator overloading in Java (with Manifold). That would be interesting: https://www.youtube.com/watch?v=pwQs-308OdY https://www.youtube.com/watch?v=pwQs-308OdY
- nilptr 3y ago
  I would be terrified on introducing that into a project and worrying about its long-term support...
  - Twisol 3y ago
    A team I was on decided against Manifold precisely because we wanted to keep up with Java's twice-yearly version updates, and we didn't want a bespoke compiler plugin to lock us to an older version. I think Manifold is extremely cool, and it does look like they're keeping up with the latest -- but the deep compiler magic it relies on really makes me nervous.
    - zeedude 3y ago
      They are good about supporting latest JDK release and all LTS releases, going back to 8. It’s Oracle’s saber rattling lately concerning plugins like Lombok and Manifold that make me more disappointed than nervous.
      - whartung 3y ago
        What rattling is that?
      - Twisol 3y ago
        Yes, exactly -- the responsibility to keep up with Java versions does not, unfortunately, lie with Manifold alone. The deep magic they rely on also needs to remain accessible.
  - invalidname 3y ago
    It is interesting through ;-) Personally, I will probably introduce some of it to a project I'm working on. Here's why: * It's relatively conservative - almost of all its work is done in compile time. Once you have a binary and it went through testing then Manifold is irrelevant. * It's open source - I know it's not the same as "support" but the project has been around for a while and the author is active. * Worst case scenario I can remove it - it would suck since I will need to rewrite code. But the compiler will point me at the exact lines of code I need to fix.
mrkeen 3y ago
I'm disappointed that > The method that gets invoked depends on the “actual” instance type, not the “declared” instance type. doesn't apply more broadly. If you have a call(fruit) method and a call(apple) method, when you supply a fruit, it will resolve to call(fruit), even if the fruit instance is actually an apple.
- ackfoobar 3y ago
  Some languages do it. https://en.wikipedia.org/wiki/Multiple_dispatch https://en.wikipedia.org/wiki/Multiple_dispatch
- deleted 3y ago
  [deleted]
cutler 3y ago
A good reason, if any were needed, to escape from the Java OOP maze once and for all and never return. Honestly, talk about a mountain of incidental complexity. People, wake up! It doesn't have to be this way. You've been sold Java's absurd contortions dressed in the emperor's new clothes as the default by universities and big corps for decades. There's a whole other world out there where data is .... just ..... data.
- seadan83 3y ago
  Most of these exact same issues would be the same in C++, for sure similarly contrived examples could be contrived. > People, wake up! It doesn't have to be this way. You've been sold Java's absurd contortions dressed in the emperor's new clothes as the default by universities and big corps for decades. I think this is a very incorrect characterization. Effective Java was required reading for me in the very late 90s (decades ago). Back then it was even argued favor composition over inheritance [1]. AFAIK you'd have to go back to the 70s or 80s for inheritance to still have a good reputation (and which at the time was certainly better than a GOTO statement). The incidental complexity was later revealed to be quite profound, and a large chunk of "Effective Java" is dedicated to explaining these complexities and why and how to avoid these. [1] https://blogs.oracle.com/javamagazine/post/java-inheritance-composition https://blogs.oracle.com/javamagazine/post/java-inheritance-...
  - brazzy 3y ago
    Inheritance is still absolutely a good thing to leverage polymorphism - but for that all you need are simple, shallow inheritance hierarchies. What has a deservedly gotten a bad reputation are ivory tower architectures with deep inheritance hierarchies that lead to things like the Abstract Factory Pattern.
    - paulddraper 3y ago
      Abstract factory pattern isn't related to deep inheritance hierarchies.
- rootlocus 3y ago
adelarsq 3y ago
These are one of the things that shiver me in production legacy code.
agumonkey 3y ago
Makes you appreciate the complexity of haskell.
shagie 3y ago
Varargs methods in Java was one that bit me with an overload in the past (my tale of is told in more detail at http://the-whiteboard.github.io/java/debugging/warstory/2016/07/27/dont-overload-varargs.html http://the-whiteboard.github.io/java/debugging/warstory/2016... ) The issue was that there was a framework class with the method signature of execute(Object ... inParams). The class had overloaded the method signature to be execute(String foo, String bar). If you called it with two args, it got the overloaded call and there were a half dozen calls to this method. At some point, someone added another argument to this so that it was execute(String foo, String bar, String qux) and updated the half dozen less one methods to use the third parameter. However, the two args call in some other code still compiled correctly because it now went back to calling execute(Object ... inParams). Though, the parameters were completely wrong for that call structure and the application suddenly sprouted runtime errors.
- int_19h 3y ago
  This kind of thing is exactly why in C++, when a derived class declares a method, it doesn't just overload any methods with the same name inherited from the bases, but hides them. If you actually want overloading across the hierarchy, the derived class needs to explicitly opt into it by "using" the base methods it needs.
- tpoacher 3y ago
  In fairness, it was pretty reckless to just "update" the existing method without providing a dummy "throw an informative error if anything attempts to call the old method" fallback method
  - shagie 3y ago
    There were all sorts of "wrong" with that attempted refactoring that goes back to the original "subclass the StoredProcedure class and overload the execute(Object ...) method rather than creating a new one... and name it callProc (for example). Note that the execute methods ( https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/jdbc/object/StoredProcedure.html https://docs.spring.io/spring-framework/docs/current/javadoc... ) all are suggested for subclass invocation rather than external. I'd have to spin up a test to verify, but I believe that if Spring had made it a protected method so that execute could only be called by a subclass rather than public where it can be called from other methods in other classes it would have mitigated the issue that allowed for the original developer to overload it. That would have made it so when the arguments changed it now was a compile error since the execute(Object...) form of the call would only have been accessible from the subclass.
exabrial 3y ago
Most of these rules are pretty straigthtforward. I don't really find any surprises in the Java method dispatch system, it just sort of works without a lot of thinking about it. The one area I think leaves some question marks for me is the "new" default methods in interfaces, aka "public defender methods". They were introduced awhile back, but they've mostly been avoided until recent years. As they become more common, a thorough understand of how Java deals with double-diamond needs to be understood by Java developers.
martincmartin 3y ago
Many of these things are at the core of the book Java Puzzlers. http://www.javapuzzlers.com/ http://www.javapuzzlers.com/
tmtvl 3y ago
Reminds me of CLOS and the versatility of defgeneric and defmethod.
vanillax 3y ago
"half a decade" isn't a brag. Its overselling the fact that you coded for 5 years. Big whoop. Stop being a wordcel and making it seem like you are more qualified than others in the space because you used the term "half a decade" like its some large quantity of time.
aymar_99 3y ago
Thanks for this curation, useful for me as a beginner. All the comments belittling these puzzles kinda feel rude.

87 points by whack 3y ago

None of this is surprising. Just think about how the compiler is going to use the object's vtable. There is no magic going on.

alserio 3y ago
I believe that OP is confused about when the resolution happens. Overloading is resolved by the compiler, inheritance at runtime
- marginalia_nu 3y ago
  Kind of a blurry distinction with a JIT-compiled languague tbh.
  - alserio 3y ago
    why? the JIT works on bytecode and JVM bytecode has no concept of overloading. Maybe I should have specified that by "compiler" i meant the first compilation step, Java source to bytecode.
    - hinkley 3y ago
      The JVM bytecode is not the JVM. There’s some late binding allowed in the JVM to deal with API changes. So the caller can call a method signature that doesn’t exist in the target (but either does in a later version or did in a previous one), and the JVM has to resolve it at first invocation, rewriting the bytecode in the process. The compile time resolution ends up being a hint that usually works, but doesn’t always. And if memory serves a number of languages that run on top of the JVM have leveraged this fact, including the original third party implementation of generics.
      - alserio 3y ago
        still the lookup is based on the symbol, no? so the JVM does not resolve overload even in that case, i believe
        hinkley 3y ago
        Read what I said again. Step 1 is a lookup by symbol. You're ignoring the vast majority of the logic. There are hints about what I'm talking about in this conversation about invokespecial: https://stackoverflow.com/questions/13764238/why-invokespecial-is-needed-when-invokevirtual-exists https://stackoverflow.com/questions/13764238/why-invokespeci... This trickiness is not called out in the bytecode specification, but in the JVM spec. They get a little closer to the mark here: https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-6.html#jvms-6.5.invokespecial https://docs.oracle.com/javase/specs/jvms/se9/html/jvms-6.ht... "Is Java Statically Typed or Dynamically Typed?" is basically an interview gotcha question. Most people should answer the former, but if you are hiring people to create a JVM or a language that translates to the JVM, you're looking for a much longer answer that circles around gradations of the latter, because the compiler is strongly typed but the JVM has weaker type guarantees that are sorted out at first invocation, and which look like variance (predating most literature on contra and covariance, I might add). Prior to invokedynamic people used this and other facts to trick the JVM into doing things that are illegal in Java. The Hotspot people learned a whole lot about how the JVM actually functions from interactions with these sorts of people.
        alserio 3y ago
        i now get what you mean, thanks
        hinkley 3y ago
        If I wasn't already clear, this isn't something that normal devs should have or want to think about. But it's hard to avoid if you end up dealing with separate compilation units that are run on a different build schedule. Sooner or later your refactoring will either blow up with an obscure error, or work even though if you really think about it, the compiler would say it shouldn't. And of course if you actually dig you'll find it out as well. If you want sophisticated dispatch you want to use invokedynamic, but that was an RFE when I was learning about this stuff.
jchw 3y ago
I think the main confusion is that when you override a method in Java, similar to other languages like C++, it considers not only the name but actually the entire function signature. That's in fact why overloading works to begin with, because of the fact that the identity of a method is not just its name but in fact it's signature.
Jenk 3y ago
I rather suspect that "magic" and "vtable" are very similar concepts to many developers.

yotamoron 3y ago

I hate inheritance.

nforgerit 3y ago
Some months ago, I stumbled into an OOP inheritance rabbithole which got me thinking the same: - Isn't the (biological) concept of inheritance built at it's core around the idea of "generations"? How does that make any sense in OOP? - Why do Java beginner courses still teach Inheritance like it was 1995 with all the bells and whistles and dogs and cats and mammals? - Why do they teach Inheritance and later introduce "favor composition to inheritance"? Doesn't this just confuse everyone? - Why does Inheritance have first-class syntax support in an OOP language like Java ("extends") whereas Composition usually needs to be "engineered" using some more complex patterns, e.g. DI? Most of it is probably for historical reasons, i guess.
- psychoslave 3y ago
  I don't know for Java currently but for example in Ruby composition is simply a matter of putting your code in a module rather than a class. Then you can extend any class with this module. You can even extend instances. That is 3.extends(Some_module).method_from_some_module is perfectly valid. In PHP, surely you can use traits.
  - emodendroket 3y ago
    Of the many models of “composition” that are possible I think Ruby’s free-for-all blend of mix-ins and monkey patches is the only one that can drive a maintenance programmer more insane than a deep hierarchy of inheritance of Java classes.
    - shagie 3y ago
      irb(main):001:0> "bar".foo Traceback (most recent call last): 4: from /usr/bin/irb:23:in `<main>' 3: from /usr/bin/irb:23:in `load' 2: from /Library/Ruby/Gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>' 1: from (irb):1 NoMethodError (undefined method `foo' for "bar":String) Did you mean? for irb(main):002:0> class String irb(main):003:1> def foo irb(main):004:2> "foobar!" irb(main):005:2> end irb(main):006:1> end => :foo irb(main):007:0> "bar".foo => "foobar!" irb(main):008:0> Yes! Let's modify the core library String class on the fly to add new functions to it. There are things about ruby that truly scare me if my goal was to write secure and reasonable code.

bvanderveen 3y ago

Just say no.

invalidname 3y ago

Are these things HN readers didn't know? I didn't know you can do operator overloading in Java (with Manifold). That would be interesting: https://www.youtube.com/watch?v=pwQs-308OdY https://www.youtube.com/watch?v=pwQs-308OdY

nilptr 3y ago
I would be terrified on introducing that into a project and worrying about its long-term support...
- Twisol 3y ago
  A team I was on decided against Manifold precisely because we wanted to keep up with Java's twice-yearly version updates, and we didn't want a bespoke compiler plugin to lock us to an older version. I think Manifold is extremely cool, and it does look like they're keeping up with the latest -- but the deep compiler magic it relies on really makes me nervous.
  - zeedude 3y ago
    They are good about supporting latest JDK release and all LTS releases, going back to 8. It’s Oracle’s saber rattling lately concerning plugins like Lombok and Manifold that make me more disappointed than nervous.
    - whartung 3y ago
      What rattling is that?
    - Twisol 3y ago
      Yes, exactly -- the responsibility to keep up with Java versions does not, unfortunately, lie with Manifold alone. The deep magic they rely on also needs to remain accessible.
- invalidname 3y ago
  It is interesting through ;-) Personally, I will probably introduce some of it to a project I'm working on. Here's why: * It's relatively conservative - almost of all its work is done in compile time. Once you have a binary and it went through testing then Manifold is irrelevant. * It's open source - I know it's not the same as "support" but the project has been around for a while and the author is active. * Worst case scenario I can remove it - it would suck since I will need to rewrite code. But the compiler will point me at the exact lines of code I need to fix.

mrkeen 3y ago

I'm disappointed that > The method that gets invoked depends on the “actual” instance type, not the “declared” instance type. doesn't apply more broadly. If you have a call(fruit) method and a call(apple) method, when you supply a fruit, it will resolve to call(fruit), even if the fruit instance is actually an apple.

ackfoobar 3y ago
Some languages do it. https://en.wikipedia.org/wiki/Multiple_dispatch https://en.wikipedia.org/wiki/Multiple_dispatch
deleted 3y ago
[deleted]

cutler 3y ago

A good reason, if any were needed, to escape from the Java OOP maze once and for all and never return. Honestly, talk about a mountain of incidental complexity. People, wake up! It doesn't have to be this way. You've been sold Java's absurd contortions dressed in the emperor's new clothes as the default by universities and big corps for decades. There's a whole other world out there where data is .... just ..... data.

seadan83 3y ago
Most of these exact same issues would be the same in C++, for sure similarly contrived examples could be contrived. > People, wake up! It doesn't have to be this way. You've been sold Java's absurd contortions dressed in the emperor's new clothes as the default by universities and big corps for decades. I think this is a very incorrect characterization. Effective Java was required reading for me in the very late 90s (decades ago). Back then it was even argued favor composition over inheritance [1]. AFAIK you'd have to go back to the 70s or 80s for inheritance to still have a good reputation (and which at the time was certainly better than a GOTO statement). The incidental complexity was later revealed to be quite profound, and a large chunk of "Effective Java" is dedicated to explaining these complexities and why and how to avoid these. [1] https://blogs.oracle.com/javamagazine/post/java-inheritance-composition https://blogs.oracle.com/javamagazine/post/java-inheritance-...
- brazzy 3y ago
  Inheritance is still absolutely a good thing to leverage polymorphism - but for that all you need are simple, shallow inheritance hierarchies. What has a deservedly gotten a bad reputation are ivory tower architectures with deep inheritance hierarchies that lead to things like the Abstract Factory Pattern.
  - paulddraper 3y ago
    Abstract factory pattern isn't related to deep inheritance hierarchies.
rootlocus 3y ago

adelarsq 3y ago

These are one of the things that shiver me in production legacy code.

agumonkey 3y ago

Makes you appreciate the complexity of haskell.

shagie 3y ago

Varargs methods in Java was one that bit me with an overload in the past (my tale of is told in more detail at http://the-whiteboard.github.io/java/debugging/warstory/2016/07/27/dont-overload-varargs.html http://the-whiteboard.github.io/java/debugging/warstory/2016... ) The issue was that there was a framework class with the method signature of execute(Object ... inParams). The class had overloaded the method signature to be execute(String foo, String bar). If you called it with two args, it got the overloaded call and there were a half dozen calls to this method. At some point, someone added another argument to this so that it was execute(String foo, String bar, String qux) and updated the half dozen less one methods to use the third parameter. However, the two args call in some other code still compiled correctly because it now went back to calling execute(Object ... inParams). Though, the parameters were completely wrong for that call structure and the application suddenly sprouted runtime errors.

int_19h 3y ago
This kind of thing is exactly why in C++, when a derived class declares a method, it doesn't just overload any methods with the same name inherited from the bases, but hides them. If you actually want overloading across the hierarchy, the derived class needs to explicitly opt into it by "using" the base methods it needs.
tpoacher 3y ago
In fairness, it was pretty reckless to just "update" the existing method without providing a dummy "throw an informative error if anything attempts to call the old method" fallback method
- shagie 3y ago
  There were all sorts of "wrong" with that attempted refactoring that goes back to the original "subclass the StoredProcedure class and overload the execute(Object ...) method rather than creating a new one... and name it callProc (for example). Note that the execute methods ( https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/jdbc/object/StoredProcedure.html https://docs.spring.io/spring-framework/docs/current/javadoc... ) all are suggested for subclass invocation rather than external. I'd have to spin up a test to verify, but I believe that if Spring had made it a protected method so that execute could only be called by a subclass rather than public where it can be called from other methods in other classes it would have mitigated the issue that allowed for the original developer to overload it. That would have made it so when the arguments changed it now was a compile error since the execute(Object...) form of the call would only have been accessible from the subclass.

exabrial 3y ago

Most of these rules are pretty straigthtforward. I don't really find any surprises in the Java method dispatch system, it just sort of works without a lot of thinking about it. The one area I think leaves some question marks for me is the "new" default methods in interfaces, aka "public defender methods". They were introduced awhile back, but they've mostly been avoided until recent years. As they become more common, a thorough understand of how Java deals with double-diamond needs to be understood by Java developers.

martincmartin 3y ago

Many of these things are at the core of the book Java Puzzlers. http://www.javapuzzlers.com/ http://www.javapuzzlers.com/

tmtvl 3y ago

Reminds me of CLOS and the versatility of defgeneric and defmethod.

vanillax 3y ago

"half a decade" isn't a brag. Its overselling the fact that you coded for 5 years. Big whoop. Stop being a wordcel and making it seem like you are more qualified than others in the space because you used the term "half a decade" like its some large quantity of time.

aymar_99 3y ago

Thanks for this curation, useful for me as a beginner. All the comments belittling these puzzles kinda feel rude.