3 ms·

Since being created, the Java pitch was "don't worry about low-level stuff like value/reference classes, a Sufficiently Smart Compiler will automatically pick t

Since being created, the Java pitch was "don't worry about low-level stuff like value/reference classes, a Sufficiently Smart Compiler will automatically pick the best option given your code and runtime profiling". What changed, why suddenly they adopt C++ features they explicitly excluded? https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler

tancop 2mo ago
Java is so dynamic that it's impossible to prove a class will only be used in "value friendly" ways. When objects have no identity the meaning of `==` is different, and the compiler would have to do some kind of whole program analysis to find out if there is any way an instance of the class could ever be checked for equality. That's hard when you have type erasure and polymorphism and runtime class loading. And even if they pulled it off it would blow up compile times and be programmer unfriendly because adding one line could deoptimize an important class defined in another package. So they decided to bite the bullet and add a way to statically opt in for faster but incompatible behavior.
- noduerme 2mo ago
  This is really interesting. I've worked and lived alongside Java since, like, becoming extremely proficient in AS3/ECMA5(ish) and understanding the low-level quirks of that VM, but I never dealt in Java. What's funny to me is that by your description, AS3 started almost as dynamic as the objective mess you're describing, and somewhat correctly headed down a path of compile-time type safety along its trajectory, which kind of gave good guardrails for those of us who had to switch to a wild west nonsense of JavaScript tempered with some hints from typescript. I wasn't aware that you could `==` two objects in Java and that it would, like, deconstruct them somehow and see if their contents matched rather than just telling you whether they referenced the same object. I'm not even sure if that is what you're saying, because that's wild and insane and it's the whole reason for observable classes in other languages (which are sort of hackish). But after so many years of like, figuring out the quickest ways to diff similar objects, relying on the VM to do it would feel like never the best solution to any given problem, and more of a footgun than a feature...?
  - benmmurphy 2mo ago
    `==` is reference identity in java and that is the problem. so you can't treat an arbitrary object as a value type because somewhere in the program `==` might be called on it.
  - tsimionescu 2mo ago
    I think there is some major misunderstanding here. The point the OP was making is that a Sufficiently Smart Compiler should be able tell if a class is a reference class (and must be allocated on the heap and referenced by pointer) or a value class (that can be copied around and stored in-line in arrays or other classes). Specifically, a class can only be a Value Class if two instances of this class are never compared using reference comparison, ==. If the program never uses this operation on two instances of this class, that means that the program would never be affected if copies are passed around instead of the original object being referenced around. The GP was pointing out that this verification is not actually feasible, as it requires access to the entire source of the entire program, it can't be decided locally.
- pjmlp 2mo ago
  Many still don't get that while Java and C# (due to how it came to be after J++ lawsuit) are inspired by C++ for the syntax, it is mostly a mix of Smalltalk and Objective-C semantics in what is achievable, hence why a Smalltalk inspired JIT design was such a great addition early on.
mcculley 2mo ago
It was not at all “suddenly”. The discussions about explicitly defining value types because escape analysis is insufficient have been going on for at least a decade.
pestatije 2mo ago
I don't think that was ever the case...value(primitive) types were there from day 1 and the given reason that it is too much overhead to use objects
debugnik 2mo ago
Even a smart compiler can't break the program semantics, and without a closed world assumption it simply can't assume that an entirely different part of the program doesn't expect to observe object identity for a type. Java is adding small, orthogonal features that amount to the same feature set as value types in other languages, but can be cherry-picked into existing code for partial advantages without significant changes. These value classes are still nullable, lack a guaranteed layout, and can't be observed torn, unlike in C++/C#/Go.
- pjmlp 2mo ago
  Nullability is coming later, and already with Panama you have plenty of room to do C like stuff. Go isn't much better.
  - tsimionescu 2mo ago
    You probably mean non-nullability. And the big limitation is the rule that object writes can't tear - while this remains in place, it means that only tiny value classes will get any of the performance advantages being discussed, on regular processors. Specifically, the largest guaranteed atomic norma read/write in x86-64 is 64 bits, so any class that is larger than that (say, a pair of longs, or even a pair of ints until we get non-nullability) will not be compactible. An array of 1M (long, long) pairs will hold 1M pointers to (long, long) pairs allocated in the GC heap, forever. An array of 1M (int, int) pairs will as well, but in some future release when non-nullability makes it in, it will actually work as hoped.
    - debugnik 2mo ago
      Torn reads are probably coming as a separate attribute to opt-in. It's technically already there but not stable, and I don't think it does anything yet to unbox values on fields/arrays.
  - debugnik 2mo ago
    Right, I didn't mean those are bad things! Just that value classes don't change the semantics of regular objects as much as value semantics do in other language. I like that the remaining features will be orthogonal and opt-in. Go isn't even memory-safe under data races, because they don't want to pick between slower loads (like .NET's Memory<T>.Span) or removing fat pointers. Meanwhile the JVM never had value types until now so they'd silently break a lot of code if a single keyword applied willy-nilly introduced torn reads like struct does in C#. It's coming but as a separate opt-in.
DarkNova6 2mo ago
Yes, this was the idiom in the 90s, but Escape Analysis has not proven to be powerful enough to optimize away identity. And looking at all the edge-cases and possible data-races via tearing, declaring something as a value must be an explicit design decision that cannot be inferred by a compiler or optimizer alone.
- tsimionescu 2mo ago
  Note that tearing is NOT a risk for value classes in Java - the JIT compiler is not allowed to optimize the layout of any value class that can tear on the current architecture (so, for any value class larger than 64 bits on x86-64).
  - DarkNova6 2mo ago
    The tradeoff being that tearing is what enables flattening. So if you truly want the best performance (cache lines etc.) you must enable it. But whether your data’s integrity can be broken by such must be decided by the class author.
    - tsimionescu 2mo ago
      > The tradeoff being that tearing is what enables flattening. Yes, exactly. > So if you truly want the best performance (cache lines etc.) you must enable it. Someone was saying that this is planned as an option in some future version of Java. As it stands, this is just not possible, and to me this suggests that the performance gains from Project Valhalla in this Java will be minimal.
      - DarkNova6 2mo ago
        Project Valhalla is not a JEP, it's a collection of JEPs and this is just the very first one. Flattening has been the goal from the start, but it's simply not something that's delivered in this JEP. But even so, people should not overestimate the effect of values and flattening. The biggest advantage will come from specialized libraries (stuff like OpenCV), and overall library enhancements (Tuples not requiring a dedicated indirection).
jasode 2mo ago
>What changed, why suddenly they adopt C++ features they explicitly excluded? Project Valhalla, which includes the effort to add value types was announced in 2014. They've been working on it for a while. As for "what changed" ... Back in 1990s when Java was conceived, there was an idea that CPUs in desktops had plenty of extra cycles that were being wasted and could therefore be used to reduce mental load on developers. It was the same "cpus are cheaper than developers" idea that is repeated today with "tokens/cpu are cheaper than developers". With that philosophy, James Gosling talked about Java's "everything-is-an-object" as a mental simplification for developers. All the extra indirections of pointer-chasing to box unbox primitives and/or iterate through arrays of objects wasn't seen as a penalty (again, "CPUs are cheap; devs are expensive"). However, the later evolution in 2000s of CPU hardware vs RAM hardware changed that performance tradeoff thesis: https://en.wikipedia.org/wiki/Random-access_memory#Memory_wall https://en.wikipedia.org/wiki/Random-access_memory#Memory_wa... Now having value types that are contiguous in RAM is a big deal for performance. Avoid a bunch of pointer chasing. Even C++ best practices were affected. E.g. the traditional tradeoffs you learned from from classroom textbooks of linked-lists being faster than arrays for middle-of-list insertions was no longer always correct in the new world where CPUs are caching adjacent RAM areas to try to reduce the memory wall issue. So O(n) could be faster than O(log n) depending on the size of the data structure and interactions with RAM pre-fetch, etc.
- SkiFire13 2mo ago
  > So O(n) could be faster than O(log n) depending on the size of the data structure and interactions with RAM pre-fetch, etc. This has always been the case. The RAM effects only changed at which point the O(n) stops being faster than the O(log n) solution.
  - inigyou 2mo ago
    Apparently linear search now beats hashmap if you have less than TWO HUNDRED elements. Crazy!
    - afdbcreid 2mo ago
SkiFire13 2mo ago
> don't worry about low-level stuff like value/reference classes Part of the issue is that this was never low-level stuff. When you're selecting between the two you're making a semantic choice, and that's not something a compiler can do for you.
marginalia_nu 2mo ago
This was pretty much always wrong. Nobody writing performance critical code in Java trusts the compiler to magically figure things out, and the sort of code you arrive at if you want Java to go fast is generally unidiomatic.
- bjoli 2mo ago
  that of course depends on what you mean by fast. Fast can mean "know how to make the job of the JIT easy" (which it does in many cases), but it can also mean "write C in Java" which is, arguably, worse than writing C in C.
xxs 2mo ago
The object header overhead was always present and expensive in massive arrays - the classic example would be the Point class that has two "double x, y". Realistically the value classes make sense most (only) if they are placed in arrays. In order to use tons of points you'd end up having two arrays, double[] x, double[] y... or go even with direct buffers. Personally, I don't care about the tiny optimizations possible in cases of just using few of them, e.g. Integer, as int[] is an option, and even writing custom maps where the keys are placed the said array ain't difficult. As for performance, often times I had to PrintAssembly (the article mentions that at the bottom) to ensure the compiler did its bests, e.g. optimizing away boundary checks, inlining calls, etc.
- tsimionescu 2mo ago
  > Realistically the value classes make sense most (only) if they are placed in arrays. Note that this is only relevant with the way Project Valhalla works for tiny value classes. A Point value class with two double coordinates can not be stored in line in an array, on an x86-64 processor, even after non-nullability is added to get one extra bit: the largest class that can be in-lined in that way has to be 64 bits in total size. So, ultimately, the performance advantages are only going to materialize for []Integer and (in the future) []Long, and for some very specific byte-level processing code.
mrkeen 2mo ago
On the other hand, value semantics are just plain easier to reason about. My 2 is your 2. We don't have different 2s because it makes no sense. But sometimes you don't just want a 2, you want a pair (2,3). All of a sudden, my (2,3) isn't your (2,3), unless you call equals instead of ==, and remember to implement equals() (and that demands implementing .hashcode() too, and probably a toString() while you're at it). While it's easy to forget about if you've been using Java for a while, 2 gets passed by value, but your pair gets passed by pointer-value. So I'd come at it from the other side. Pick a value representation of (2,3), reap the simplicity, and hope the sufficiently smart compiler is smart enough to do good things with it.
pron 2mo ago
First, it's not a C++ feature. In C++ you tell the compiler how to lay out objects in memory. Here you declare what properties your class has (e.g. whether it needs identity or not), and the compiler decides how to lay out each of its instances in memory, and may automatically do it in different ways in different places. So the principle that "you tell us what semantics you want and let the compiler figure out the implementation" remains in effect. Second, the reason why the compiler cannot infer on its own that a class does or does not need identity without you declaring it is that the use of identity can be in a different module.

tancop 2mo ago

Java is so dynamic that it's impossible to prove a class will only be used in "value friendly" ways. When objects have no identity the meaning of `==` is different, and the compiler would have to do some kind of whole program analysis to find out if there is any way an instance of the class could ever be checked for equality. That's hard when you have type erasure and polymorphism and runtime class loading. And even if they pulled it off it would blow up compile times and be programmer unfriendly because adding one line could deoptimize an important class defined in another package. So they decided to bite the bullet and add a way to statically opt in for faster but incompatible behavior.

noduerme 2mo ago
This is really interesting. I've worked and lived alongside Java since, like, becoming extremely proficient in AS3/ECMA5(ish) and understanding the low-level quirks of that VM, but I never dealt in Java. What's funny to me is that by your description, AS3 started almost as dynamic as the objective mess you're describing, and somewhat correctly headed down a path of compile-time type safety along its trajectory, which kind of gave good guardrails for those of us who had to switch to a wild west nonsense of JavaScript tempered with some hints from typescript. I wasn't aware that you could `==` two objects in Java and that it would, like, deconstruct them somehow and see if their contents matched rather than just telling you whether they referenced the same object. I'm not even sure if that is what you're saying, because that's wild and insane and it's the whole reason for observable classes in other languages (which are sort of hackish). But after so many years of like, figuring out the quickest ways to diff similar objects, relying on the VM to do it would feel like never the best solution to any given problem, and more of a footgun than a feature...?
- benmmurphy 2mo ago
  `==` is reference identity in java and that is the problem. so you can't treat an arbitrary object as a value type because somewhere in the program `==` might be called on it.
- tsimionescu 2mo ago
  I think there is some major misunderstanding here. The point the OP was making is that a Sufficiently Smart Compiler should be able tell if a class is a reference class (and must be allocated on the heap and referenced by pointer) or a value class (that can be copied around and stored in-line in arrays or other classes). Specifically, a class can only be a Value Class if two instances of this class are never compared using reference comparison, ==. If the program never uses this operation on two instances of this class, that means that the program would never be affected if copies are passed around instead of the original object being referenced around. The GP was pointing out that this verification is not actually feasible, as it requires access to the entire source of the entire program, it can't be decided locally.
pjmlp 2mo ago
Many still don't get that while Java and C# (due to how it came to be after J++ lawsuit) are inspired by C++ for the syntax, it is mostly a mix of Smalltalk and Objective-C semantics in what is achievable, hence why a Smalltalk inspired JIT design was such a great addition early on.

mcculley 2mo ago

It was not at all “suddenly”. The discussions about explicitly defining value types because escape analysis is insufficient have been going on for at least a decade.

pestatije 2mo ago

I don't think that was ever the case...value(primitive) types were there from day 1 and the given reason that it is too much overhead to use objects

debugnik 2mo ago

Even a smart compiler can't break the program semantics, and without a closed world assumption it simply can't assume that an entirely different part of the program doesn't expect to observe object identity for a type. Java is adding small, orthogonal features that amount to the same feature set as value types in other languages, but can be cherry-picked into existing code for partial advantages without significant changes. These value classes are still nullable, lack a guaranteed layout, and can't be observed torn, unlike in C++/C#/Go.

pjmlp 2mo ago
Nullability is coming later, and already with Panama you have plenty of room to do C like stuff. Go isn't much better.
- tsimionescu 2mo ago
  You probably mean non-nullability. And the big limitation is the rule that object writes can't tear - while this remains in place, it means that only tiny value classes will get any of the performance advantages being discussed, on regular processors. Specifically, the largest guaranteed atomic norma read/write in x86-64 is 64 bits, so any class that is larger than that (say, a pair of longs, or even a pair of ints until we get non-nullability) will not be compactible. An array of 1M (long, long) pairs will hold 1M pointers to (long, long) pairs allocated in the GC heap, forever. An array of 1M (int, int) pairs will as well, but in some future release when non-nullability makes it in, it will actually work as hoped.
  - debugnik 2mo ago
    Torn reads are probably coming as a separate attribute to opt-in. It's technically already there but not stable, and I don't think it does anything yet to unbox values on fields/arrays.
- debugnik 2mo ago
  Right, I didn't mean those are bad things! Just that value classes don't change the semantics of regular objects as much as value semantics do in other language. I like that the remaining features will be orthogonal and opt-in. Go isn't even memory-safe under data races, because they don't want to pick between slower loads (like .NET's Memory<T>.Span) or removing fat pointers. Meanwhile the JVM never had value types until now so they'd silently break a lot of code if a single keyword applied willy-nilly introduced torn reads like struct does in C#. It's coming but as a separate opt-in.

DarkNova6 2mo ago

Yes, this was the idiom in the 90s, but Escape Analysis has not proven to be powerful enough to optimize away identity. And looking at all the edge-cases and possible data-races via tearing, declaring something as a value must be an explicit design decision that cannot be inferred by a compiler or optimizer alone.

tsimionescu 2mo ago
Note that tearing is NOT a risk for value classes in Java - the JIT compiler is not allowed to optimize the layout of any value class that can tear on the current architecture (so, for any value class larger than 64 bits on x86-64).
- DarkNova6 2mo ago
  The tradeoff being that tearing is what enables flattening. So if you truly want the best performance (cache lines etc.) you must enable it. But whether your data’s integrity can be broken by such must be decided by the class author.
  - tsimionescu 2mo ago
    > The tradeoff being that tearing is what enables flattening. Yes, exactly. > So if you truly want the best performance (cache lines etc.) you must enable it. Someone was saying that this is planned as an option in some future version of Java. As it stands, this is just not possible, and to me this suggests that the performance gains from Project Valhalla in this Java will be minimal.
    - DarkNova6 2mo ago
      Project Valhalla is not a JEP, it's a collection of JEPs and this is just the very first one. Flattening has been the goal from the start, but it's simply not something that's delivered in this JEP. But even so, people should not overestimate the effect of values and flattening. The biggest advantage will come from specialized libraries (stuff like OpenCV), and overall library enhancements (Tuples not requiring a dedicated indirection).

jasode 2mo ago

>What changed, why suddenly they adopt C++ features they explicitly excluded? Project Valhalla, which includes the effort to add value types was announced in 2014. They've been working on it for a while. As for "what changed" ... Back in 1990s when Java was conceived, there was an idea that CPUs in desktops had plenty of extra cycles that were being wasted and could therefore be used to reduce mental load on developers. It was the same "cpus are cheaper than developers" idea that is repeated today with "tokens/cpu are cheaper than developers". With that philosophy, James Gosling talked about Java's "everything-is-an-object" as a mental simplification for developers. All the extra indirections of pointer-chasing to box unbox primitives and/or iterate through arrays of objects wasn't seen as a penalty (again, "CPUs are cheap; devs are expensive"). However, the later evolution in 2000s of CPU hardware vs RAM hardware changed that performance tradeoff thesis: https://en.wikipedia.org/wiki/Random-access_memory#Memory_wall https://en.wikipedia.org/wiki/Random-access_memory#Memory_wa... Now having value types that are contiguous in RAM is a big deal for performance. Avoid a bunch of pointer chasing. Even C++ best practices were affected. E.g. the traditional tradeoffs you learned from from classroom textbooks of linked-lists being faster than arrays for middle-of-list insertions was no longer always correct in the new world where CPUs are caching adjacent RAM areas to try to reduce the memory wall issue. So O(n) could be faster than O(log n) depending on the size of the data structure and interactions with RAM pre-fetch, etc.

SkiFire13 2mo ago
> So O(n) could be faster than O(log n) depending on the size of the data structure and interactions with RAM pre-fetch, etc. This has always been the case. The RAM effects only changed at which point the O(n) stops being faster than the O(log n) solution.
- inigyou 2mo ago
  Apparently linear search now beats hashmap if you have less than TWO HUNDRED elements. Crazy!
  - afdbcreid 2mo ago

SkiFire13 2mo ago

> don't worry about low-level stuff like value/reference classes Part of the issue is that this was never low-level stuff. When you're selecting between the two you're making a semantic choice, and that's not something a compiler can do for you.

marginalia_nu 2mo ago

This was pretty much always wrong. Nobody writing performance critical code in Java trusts the compiler to magically figure things out, and the sort of code you arrive at if you want Java to go fast is generally unidiomatic.

bjoli 2mo ago
that of course depends on what you mean by fast. Fast can mean "know how to make the job of the JIT easy" (which it does in many cases), but it can also mean "write C in Java" which is, arguably, worse than writing C in C.

xxs 2mo ago

The object header overhead was always present and expensive in massive arrays - the classic example would be the Point class that has two "double x, y". Realistically the value classes make sense most (only) if they are placed in arrays. In order to use tons of points you'd end up having two arrays, double[] x, double[] y... or go even with direct buffers. Personally, I don't care about the tiny optimizations possible in cases of just using few of them, e.g. Integer, as int[] is an option, and even writing custom maps where the keys are placed the said array ain't difficult. As for performance, often times I had to PrintAssembly (the article mentions that at the bottom) to ensure the compiler did its bests, e.g. optimizing away boundary checks, inlining calls, etc.

tsimionescu 2mo ago
> Realistically the value classes make sense most (only) if they are placed in arrays. Note that this is only relevant with the way Project Valhalla works for tiny value classes. A Point value class with two double coordinates can not be stored in line in an array, on an x86-64 processor, even after non-nullability is added to get one extra bit: the largest class that can be in-lined in that way has to be 64 bits in total size. So, ultimately, the performance advantages are only going to materialize for []Integer and (in the future) []Long, and for some very specific byte-level processing code.

mrkeen 2mo ago

On the other hand, value semantics are just plain easier to reason about. My 2 is your 2. We don't have different 2s because it makes no sense. But sometimes you don't just want a 2, you want a pair (2,3). All of a sudden, my (2,3) isn't your (2,3), unless you call equals instead of ==, and remember to implement equals() (and that demands implementing .hashcode() too, and probably a toString() while you're at it). While it's easy to forget about if you've been using Java for a while, 2 gets passed by value, but your pair gets passed by pointer-value. So I'd come at it from the other side. Pick a value representation of (2,3), reap the simplicity, and hope the sufficiently smart compiler is smart enough to do good things with it.

pron 2mo ago

First, it's not a C++ feature. In C++ you tell the compiler how to lay out objects in memory. Here you declare what properties your class has (e.g. whether it needs identity or not), and the compiler decides how to lay out each of its instances in memory, and may automatically do it in different ways in different places. So the principle that "you tell us what semantics you want and let the compiler figure out the implementation" remains in effect. Second, the reason why the compiler cannot infer on its own that a class does or does not need identity without you declaring it is that the use of identity can be in a different module.