8 ms·

JVM Anatomy Quark #10: String.intern (2019)

119 points by hyperpape 4y ago

metadat 4y ago
Tl;dr: > In almost every project we were taking care of, removing String.intern() from the hotpaths, or optionally replacing it with a handrolled deduplicator, was the very profitable performance optimization. > Do not use String.intern() without thinking very hard about it, okay? Overall fantastic article covering intern() in depth! The only thing I'd like to see added would be visual graphs instead of numeric tables for the benchmark results.
abadger9 4y ago
Having worked in the JVM ecosystem building low latency systems for over a decade, I always aspire to have the minute knowledge Aleksey has. I wish there was a book/MOOC for more senior engineers, but i've found myself content and happy with the Anatomy Quark series.
- hyperpape 4y ago
  I share the feeling that I wish there were more resources out there. I've collected a motley set at https://justinblank.com/notebooks/jvmarchitecture.html https://justinblank.com/notebooks/jvmarchitecture.html, but I haven't found anything systematic.
  - abadger9 4y ago
    this is the one that got me started in low latency java a decade ago - http://blog.vanillajava.blog/ http://blog.vanillajava.blog/ - I've spoken with Peter a couple of times, he's a phenomenal technologist
    - gwittel 4y ago
      Vanilla Java is a great resource. The work that Chronicle does got me into low latency Java as well (not doing java now though). Some other great resources I used: * http://psy-lob-saw.blogspot.com/ http://psy-lob-saw.blogspot.com/ - Not updated since 2018, but still great stuff in there * https://richardstartin.github.io/ https://richardstartin.github.io/ - A few posts per year. Richard does a lot of interesting things.
  - layer8 4y ago
    https://shipilev.net/jvm-anatomy-park https://shipilev.net/jvm-anatomy-park yields 404. Maybe that should link to the TFA now?
    - hyperpape 4y ago
      Thanks, will update later. https://shipilev.net/jvm/anatomy-quarks/ https://shipilev.net/jvm/anatomy-quarks/ is the address.
- nnoitra 4y ago
  Why is Java used in low latency systems? Isn't C++ a better choice there?
  - thriftwy 4y ago
    C++ is actually an esoteric language, not unlike Haskell or Prolog. It is just a widely used one, for historical reasons. Meanwhile, you may just get things done in Java without giving much thought.
    - icedchai 4y ago
      When there are orders of magnitude more people working with C++ compare to Haskell or Prolog, I would hesitate to call it "esoteric." The problem with C++ is everyone uses their own little dialect, ranging from little more than C with classes / nicer syntax, to every new "modern" feature under the sun.
  - twhitmore 4y ago
    It works 24/7, I guess. Reliability is the key -- it stays running. It's got a vast array of third-party libraries available. It performs pretty well even in a low-latency environment. Developers can spend time solving actual business problems, rather than waiting to compile, fooling around with memory allocation, templating problems & inefficient STL data structures, or badly handrolling data structures. There is a little extra work to stay within a latency envelope, but there are definitely productivity boosts from just forgetting all the old C legacy and being able to work at a higher level of thinking. Even better, use the JVM but try Kotlin as a language.
theandrewbailey 4y ago
I've been writing Java for about 15 years. There hasn't been a single instance where I've used String.intern(). I'm starting to think that there is never an appropriate place for it. What do you get from it? Anything else other than the ability to reliably use == for String comparison instead of .equals()? Nevermind the architecture astronauts saying that the string intern pool is actually global state and global state is always problematic.
- jefffoster 4y ago
  Once upon a time I found string interning useful in conjunction with IdentityHashMap for quick lookup of lots of strings.
- GlitchMr 4y ago
  I don't think using String::intern makes sense, especially now that Java's garbage collector is capable of deduplicating strings (https://openjdk.org/jeps/192 https://openjdk.org/jeps/192). In the past it could have been used to reduce memory usage when a given string was used a lot, but now there are better ways of dealing with that issue.
  - hyperpape 4y ago
    G1's deduplication is nice, but note that G1's deduplication is a lot weaker than what String.intern does. G1 deduplicates the underlying byte array, but leaves separate strings (so s1 == s2 will evaluate false). So you still have an extra object header. If you have (like one our applications did) millions of copies of the string "USA" in memory, that's many megabytes of memory that explicit deduplication can save that the garbage collector can't. String.intern isn't the way, for all the reasons this post outlines, but just using G1 isn't the right approach either.
    - vips7L 4y ago
      IIRC all of the concurrent GCs can dedupe now. Not just G1. Hopefully soon object headers will be negligible with progress from Lilliput though.
throwaway23234 4y ago
For anyone that uses XML stuff in Java, you are probably aware of String.intern memory and performance issues. I don't know if that stuff persists today, but a ton of XML parsing goes through that method.
deleted 4y ago
[deleted]
smarks 4y ago
(Note: the article is from 2019, but it is still relevant and interesting.) If all you're doing is deduplicating string objects in order to save memory, the article makes a good case to use `HashMap` or `ConcurrentHashMap` instead of `String.intern`. The reason to use `String.intern` is in the fairly narrow case where you want to use `==` to compare strings, and some of these strings originated as string literals in Java source code. Java specifies that string literals for equal strings are always the same Java object, even if they occur in different class files that are compiled separately. This is accomplished by interning the strings, and `String.intern` provides user-level access to this same intern pool.
- chii 4y ago
  the guava library has an interner that does exactly what you'd want for a cache of strings, without blowing up the jvm's internal code cache (which iirc is what String.intern() uses). https://guava.dev/releases/22.0/api/docs/com/google/common/collect/Interners.html https://guava.dev/releases/22.0/api/docs/com/google/common/c...
- deleted 4y ago
  [deleted]
- pciexpgpu 4y ago
  Isn’t this concept originally from C#? (Where interning is the default behavior)
  - smarks 4y ago
    Unlikely, as Java’s string interning behavior was specified in the first edition of the JLS (1996). This predates C# as far as I know. https://download.oracle.com/otn-pub/jcp/jls1-spec/jls1.pdf?AuthParam=1655960553_b4987d0abe9cfd0c583f90405571146d https://download.oracle.com/otn-pub/jcp/jls1-spec/jls1.pdf?A... See section 3.10.5.
  - robertlagrant 4y ago
    The JVM interns string literals under the hood, yes.
marginalia_nu 4y ago
I've sort of felt that String.intern is sort of a primordial vestige from really old Java landscape that doesn't quite make sense anymore. Can't remove it for backwards compatibility issues, but I don't understand when or where you would want to use it in today's landscape.
brundolf 4y ago
Huh. I assumed that all languages with immutable strings did this sort of thing transparently Edit: according to another comment, it sort of can > In today's JVM, if you are using the G1 garbage collector, you can get 99% of the memory savings almost completely for free with a VM parameter: You still pay for an extra object, just pointed to the same character array underneath.
mrighele 4y ago
I would like also a comparison with the String deduplication option ( XX:+UseStringDeduplication ). From what I understand it doesn't use String.intern but makes different instances share the same underlying char array so I am wondering how it behaves. In my tests it gives nice memory savings "for free", but I never had the chance to test with a production workload
kllrnohj 4y ago
Not core to the article's ultimate point and conclusion, but in the premise it has: > The performance is at the mercy of the native HashTable implementation, which may lag behind what is available in high-performance Java world, especially under concurrent access. Does anyone have any idea what this is referring to, if anything? Java's HashMap performance is pretty mediocre and without value types seems un-improvable. Certainly no where close to the SIMD-accelerated cache-line optimized hash maps you can find in C++ like the one in abseil or F14. It's a good article otherwise, just this note comes across as wishful thinking than a premise with any truth behind it.
- smarks 4y ago
  The "native HashTable" referred to here is the string hash table that's inside the JVM. The JVM has to intern strings from class files' string literals here, so it's all implemented in native code. The comment about performance probably refers to Java constantly improving in performance because of improvements to the JIT compiler, whereas that native HashTable in the JVM is pretty static in performance unless somebody rewrites it or if the C++ optimization gets significantly better. Essentially Shipilëv is saying that Java code gets faster more quickly than the C++ code in the JVM.
  - kllrnohj 4y ago
    JITs don't improve a hashtables performance as they are dominated by memory access patterns (eg, open vs closed addressing), load factor, and hash quality. Of those 3 factors, the first is borderline impossible to optimize in Java at all. Regardless, the runtime isn't helping with any of those 3, it requires changes to the algorithms used. Which doesn't seem to be happening in Java land broadly? It's possible that the JVM just ignores its c++ code, sure, but that's not commentary on Java or the Java ecosystem. Also offline compiler optimization has so far consistently outpaced (or at least kept up with) JIT optimizations. So it's incorrect to consider the native code as static anyway unless you're just never upgrading the compiler.

119 points by hyperpape 4y ago

Tl;dr: > In almost every project we were taking care of, removing String.intern() from the hotpaths, or optionally replacing it with a handrolled deduplicator, was the very profitable performance optimization. > Do not use String.intern() without thinking very hard about it, okay? Overall fantastic article covering intern() in depth! The only thing I'd like to see added would be visual graphs instead of numeric tables for the benchmark results.

abadger9 4y ago

Having worked in the JVM ecosystem building low latency systems for over a decade, I always aspire to have the minute knowledge Aleksey has. I wish there was a book/MOOC for more senior engineers, but i've found myself content and happy with the Anatomy Quark series.

hyperpape 4y ago
I share the feeling that I wish there were more resources out there. I've collected a motley set at https://justinblank.com/notebooks/jvmarchitecture.html https://justinblank.com/notebooks/jvmarchitecture.html, but I haven't found anything systematic.
- abadger9 4y ago
  this is the one that got me started in low latency java a decade ago - http://blog.vanillajava.blog/ http://blog.vanillajava.blog/ - I've spoken with Peter a couple of times, he's a phenomenal technologist
  - gwittel 4y ago
    Vanilla Java is a great resource. The work that Chronicle does got me into low latency Java as well (not doing java now though). Some other great resources I used: * http://psy-lob-saw.blogspot.com/ http://psy-lob-saw.blogspot.com/ - Not updated since 2018, but still great stuff in there * https://richardstartin.github.io/ https://richardstartin.github.io/ - A few posts per year. Richard does a lot of interesting things.
- layer8 4y ago
  https://shipilev.net/jvm-anatomy-park https://shipilev.net/jvm-anatomy-park yields 404. Maybe that should link to the TFA now?
  - hyperpape 4y ago
    Thanks, will update later. https://shipilev.net/jvm/anatomy-quarks/ https://shipilev.net/jvm/anatomy-quarks/ is the address.
nnoitra 4y ago
Why is Java used in low latency systems? Isn't C++ a better choice there?
- thriftwy 4y ago
  C++ is actually an esoteric language, not unlike Haskell or Prolog. It is just a widely used one, for historical reasons. Meanwhile, you may just get things done in Java without giving much thought.
  - icedchai 4y ago
    When there are orders of magnitude more people working with C++ compare to Haskell or Prolog, I would hesitate to call it "esoteric." The problem with C++ is everyone uses their own little dialect, ranging from little more than C with classes / nicer syntax, to every new "modern" feature under the sun.
- twhitmore 4y ago
  It works 24/7, I guess. Reliability is the key -- it stays running. It's got a vast array of third-party libraries available. It performs pretty well even in a low-latency environment. Developers can spend time solving actual business problems, rather than waiting to compile, fooling around with memory allocation, templating problems & inefficient STL data structures, or badly handrolling data structures. There is a little extra work to stay within a latency envelope, but there are definitely productivity boosts from just forgetting all the old C legacy and being able to work at a higher level of thinking. Even better, use the JVM but try Kotlin as a language.

theandrewbailey 4y ago

I've been writing Java for about 15 years. There hasn't been a single instance where I've used String.intern(). I'm starting to think that there is never an appropriate place for it. What do you get from it? Anything else other than the ability to reliably use == for String comparison instead of .equals()? Nevermind the architecture astronauts saying that the string intern pool is actually global state and global state is always problematic.

jefffoster 4y ago
Once upon a time I found string interning useful in conjunction with IdentityHashMap for quick lookup of lots of strings.
GlitchMr 4y ago
I don't think using String::intern makes sense, especially now that Java's garbage collector is capable of deduplicating strings (https://openjdk.org/jeps/192 https://openjdk.org/jeps/192). In the past it could have been used to reduce memory usage when a given string was used a lot, but now there are better ways of dealing with that issue.
- hyperpape 4y ago
  G1's deduplication is nice, but note that G1's deduplication is a lot weaker than what String.intern does. G1 deduplicates the underlying byte array, but leaves separate strings (so s1 == s2 will evaluate false). So you still have an extra object header. If you have (like one our applications did) millions of copies of the string "USA" in memory, that's many megabytes of memory that explicit deduplication can save that the garbage collector can't. String.intern isn't the way, for all the reasons this post outlines, but just using G1 isn't the right approach either.
  - vips7L 4y ago
    IIRC all of the concurrent GCs can dedupe now. Not just G1. Hopefully soon object headers will be negligible with progress from Lilliput though.

throwaway23234 4y ago

For anyone that uses XML stuff in Java, you are probably aware of String.intern memory and performance issues. I don't know if that stuff persists today, but a ton of XML parsing goes through that method.

deleted 4y ago

[deleted]

smarks 4y ago

(Note: the article is from 2019, but it is still relevant and interesting.) If all you're doing is deduplicating string objects in order to save memory, the article makes a good case to use `HashMap` or `ConcurrentHashMap` instead of `String.intern`. The reason to use `String.intern` is in the fairly narrow case where you want to use `==` to compare strings, and some of these strings originated as string literals in Java source code. Java specifies that string literals for equal strings are always the same Java object, even if they occur in different class files that are compiled separately. This is accomplished by interning the strings, and `String.intern` provides user-level access to this same intern pool.

chii 4y ago
the guava library has an interner that does exactly what you'd want for a cache of strings, without blowing up the jvm's internal code cache (which iirc is what String.intern() uses). https://guava.dev/releases/22.0/api/docs/com/google/common/collect/Interners.html https://guava.dev/releases/22.0/api/docs/com/google/common/c...
deleted 4y ago
[deleted]
pciexpgpu 4y ago
Isn’t this concept originally from C#? (Where interning is the default behavior)
- smarks 4y ago
  Unlikely, as Java’s string interning behavior was specified in the first edition of the JLS (1996). This predates C# as far as I know. https://download.oracle.com/otn-pub/jcp/jls1-spec/jls1.pdf?AuthParam=1655960553_b4987d0abe9cfd0c583f90405571146d https://download.oracle.com/otn-pub/jcp/jls1-spec/jls1.pdf?A... See section 3.10.5.
- robertlagrant 4y ago
  The JVM interns string literals under the hood, yes.

marginalia_nu 4y ago

I've sort of felt that String.intern is sort of a primordial vestige from really old Java landscape that doesn't quite make sense anymore. Can't remove it for backwards compatibility issues, but I don't understand when or where you would want to use it in today's landscape.

brundolf 4y ago

Huh. I assumed that all languages with immutable strings did this sort of thing transparently Edit: according to another comment, it sort of can > In today's JVM, if you are using the G1 garbage collector, you can get 99% of the memory savings almost completely for free with a VM parameter: You still pay for an extra object, just pointed to the same character array underneath.

mrighele 4y ago

I would like also a comparison with the String deduplication option ( XX:+UseStringDeduplication ). From what I understand it doesn't use String.intern but makes different instances share the same underlying char array so I am wondering how it behaves. In my tests it gives nice memory savings "for free", but I never had the chance to test with a production workload

kllrnohj 4y ago

Not core to the article's ultimate point and conclusion, but in the premise it has: > The performance is at the mercy of the native HashTable implementation, which may lag behind what is available in high-performance Java world, especially under concurrent access. Does anyone have any idea what this is referring to, if anything? Java's HashMap performance is pretty mediocre and without value types seems un-improvable. Certainly no where close to the SIMD-accelerated cache-line optimized hash maps you can find in C++ like the one in abseil or F14. It's a good article otherwise, just this note comes across as wishful thinking than a premise with any truth behind it.

smarks 4y ago
The "native HashTable" referred to here is the string hash table that's inside the JVM. The JVM has to intern strings from class files' string literals here, so it's all implemented in native code. The comment about performance probably refers to Java constantly improving in performance because of improvements to the JIT compiler, whereas that native HashTable in the JVM is pretty static in performance unless somebody rewrites it or if the C++ optimization gets significantly better. Essentially Shipilëv is saying that Java code gets faster more quickly than the C++ code in the JVM.
- kllrnohj 4y ago
  JITs don't improve a hashtables performance as they are dominated by memory access patterns (eg, open vs closed addressing), load factor, and hash quality. Of those 3 factors, the first is borderline impossible to optimize in Java at all. Regardless, the runtime isn't helping with any of those 3, it requires changes to the algorithms used. Which doesn't seem to be happening in Java land broadly? It's possible that the JVM just ignores its c++ code, sure, but that's not commentary on Java or the Java ecosystem. Also offline compiler optimization has so far consistently outpaced (or at least kept up with) JIT optimizations. So it's incorrect to consider the native code as static anyway unless you're just never upgrading the compiler.