6 ms·
The sorry state of Java deserialization
- deleted 2y ago[deleted]
- pestatije 2y agoare the benchmarks done properly? whats the actual test code?
- Jaxan 2y agoI don’t think the conclusion need a lot of precision in the benchmark. When the suggested code from the standard library (or some tutorial) is two orders of magnitude slower, something is not right. The author is right that we are wasting something somewhere when we are only operating at 2% of the possible speed of the hard disk.
- antonhag 2y agoFrom the code samples it's hard to tell whether or not this has to do with de-serialization though. It would have been fun to see profiling results for tests such as these.
- marginalia_nu 2y agoAuthor here, I'm away from my computer atm, but I can cook up a repo with each test in a few hours when I get home. I designed the tests as a drag race because that mimics my real world usage.
- cowwoc 2y agoHi, Please add https://github.com/apache/fury https://github.com/apache/fury to the benchmark. It claims to be a drop-in replacement for the built-in serialization mechanism so it should be easy to try.
- marginalia_nu 2y agoWill do!
- antonhag 2y agoThat's nice - I'd encourage you to play around with attaching e.g. JMC [1] to the process to better understand why things are as they are. I tried recreating your DataInputStream + BufferedInputStream (wrote the 1brc data to separate output files, read using your code - I had to guess at ResultObserver implementation though). On my machine it roughly in the same time frame as yours - ~1min. According to Flight Recorder: - ~49% of the time is spent in reading the strings (city names). Almost all of it in the DataInputStream.readUTF/readFully methods. - ~5% of the time is spent reading temperature (readShort) - ~41% of the time is spent doing hashmap look-ups for computeIfAbsent() - About 50GB of memory is allocated - %99.9 of it for Strings (and the wrapped byte[] array in them). This likely causes quite a bit of GC pressure. Hash-map lookups are not de-serialization, yet the lookup likely affected the benchmarks quite a bit. The rest of the time is mostly spent in reading and allocating strings. I would guess that that is true for some of the other implementations in the original post as well. [1] https://github.com/openjdk/jmc https://github.com/openjdk/jmc edit: better link to JMC
- marginalia_nu 2y agoJMC is indeed a valuable tool, though what you see in any java profiler is to be taken with a grain of salt. The string parsing and hash lookups are present in most of the implementations, yet some of them are up to 10 times faster than the DataInputStream + BufferedInputStream code. It doesn't seem like it can be true that 90% of the time is spent in string parsing and hash lookups if the same operation takes 10% of the time when reading from a filechannel and bytebuffer.
- antonhag 2y agoAren't the versions that take 10% of the time only reading each city name once, and then doing an array lookup rather than a hashmap lookup?
- marginalia_nu 2y agoNope, see for example "Custom 1": var buffer = ByteBuffer.allocate(4096); try (var fc = (FileChannel) Files.newByteChannel(tempFile, StandardOpenOption.READ)) { buffer.flip(); for (int i = 0; i < records; i++) { if (buffer.remaining() < 32) { buffer.compact(); fc.read(buffer); buffer.flip(); } int len = buffer.get(); byte[] cityBytes = new byte[len]; buffer.get(cityBytes); String city = new String(cityBytes); int temperature = buffer.getShort(); stats.computeIfAbsent(city, k -> new ResultsObserver()) .observe(temperature / 100.); } }
- mynegation 2y ago“I admit I don’t understand these results. There’s clearly nothing in the runtime itself that prevents these types of speeds.” Oh, there is. The default Java serialization is sort of like “pickle” module in Python - if you are familiar. It will deal with pretty much anything you throw at it, figuring the data structures and offsets to serialize or parse at runtime. More efficient methods trade universality for speed, where the offsets and calls to read/write the parts of the structure are determined in advance. Also, hard to say without source code but there is a high chance even more efficient methods like Protobuf create a lot of Java objects and that kills cache locality. With Java, you have to go out of your way to maintain good cache locality because you give up control over memory layout for automatic memory management.
- jbellis 2y agoTrue, but none of the slow methods in the article involve this.
- hedora 2y agoThe article does have source code. I don’t think any of the examples use Java’s Serializable. The first attempt reads shorts and utf8 directly from the stream.
- marginalia_nu 2y agoObjectInputStream is one of the faster stream options tested.
- kaba0 2y ago> Also, hard to say without source code but there is a high chance even more efficient methods like Protobuf create a lot of Java objects and that kills cache locality I don’t think this can be claimed that easily without more info, generational GCs work pretty much like an arena allocator, with very good cache locality (think of an ArrayList getting filled with objects that are continuously allocated in short order. The objects will be right next to each other, in memory). If the objects are short-lived, they can be similarly cheap to stack allocation (thread-local allocation buffers that just bumping pointers).
- twoodfin 2y agoI’m probably missing something obvious, but what’s wrong with Apache parquet-java for this use case?
- marginalia_nu 2y agoThe implementation is inexorably merged with hadoop, to the point where it is not useful outside of it. Parquet-floor is a shim that replaces the hadoop depenencies with drop in java.io-ones.
- mike_hearn 2y agoIt'd be interesting to see Cap'n'Proto be profiled here, as the whole idea of that format is to eliminate deserialization overhead entirely.
- garblegarble 2y agoI know it's bad form to comment on style instead of content, but saying Smartphone enjoyers will want to switch to horizontal mode for this article due to code samples that barely fit on desktop while having the article text column shrink to less than 1/3rd of the horizontal space just feels disrespectful
- marginalia_nu 2y agoTry refresh? Should cover 60ch, but some browsers bug out when you turn for reasons i don't understand.
- garblegarble 2y agoTried a refresh, a force-refresh... no change at all (Firefox and Chrome)
- marginalia_nu 2y agoInteresting. Android or iOS?
- garblegarble 2y agoDesktop - that's my complaint... you wrote that the code samples "barely fit on desktop", which is only true because your CSS wastes over 2/3rds of the screen width I have provided to my user agent. (it's fine though - I use Reader Mode on such user-hostile sites)
- marginalia_nu 2y agoOh yeah, now I understand. I'm struggling to find a width for the layout that makes sense for text (where narrow columns are preferable), but also for code snippets (where you want wider columns).
- 2y ago
- splix 2y agoI don't think that Java serialization is designed for such a small object with just two fields. It's designed for large and complex objects. Obviously it would be slower and much larger in size that a columnar implementation designed and heavily optimized for this scenarios. It's not a fair comparison and too far from a real use case. Try with nested objects and at least a dozen of fields across this hierarchy. And different structure for each row. It's still not a use case for Java serialization, but at least closer to what a real code would do. Same for Protobuf, I guess. Also the JSON serialization plays the same role more or less. Maybe something like Avro Data Files is better for a comparison with columnar formats.
- SirYwell 2y agoWhy doesn't it mention the used Java version? And a few flame graphs would be interesting as well.
- charleslmunger 2y agoI noticed: 1. Small read buffers. No reason to sequentially read and parse gigabytes only 4kb at a time. 2. parseDelimitedFrom created a new CodedInputStream on every message, which has its own internal buffer; that's why you don't see a buffered stream wrapper in the examples. Every iteration of the loop is allocating fresh 4kb byte[]s. 3. The nio protobuf code creates wrappers for the allocated ByteBuffer on every iteration of the loop. But the real sin with the protobuf code is serializing the same city names over and over, reading parsing and hashing. Making a header with the city string mapped to an integer would dramatically shrink the file and speed up parsing. If that was done, your cost would essentially be the cost of decoding varints.
- amluto 2y ago> Making a header with the city string mapped to an integer would dramatically shrink the file and speed up parsing. Indeed, Parquet will do this for you if you let it. I also wonder how good Java is at optimizing closure creation in a loop, as in: k -> new ResultObserver() The vast majority of those closures are created and never called. C++ might optimize this well if calling a template but has basically no chance if the parameter is an std::function. Java probably has more ability to optimize it, but I don’t know whether it actually does so.
- the-alchemist 2y agoI believe the term for this is "partial escape analysis" (PEA). GraalVM is the SOTA in the JVM world for this. Here's an oldish blog on this: https://chrisseaton.com/truffleruby/seeing-escape-analysis/ https://chrisseaton.com/truffleruby/seeing-escape-analysis/ I'm sure PEA is better now, but it's not sure if it's moved beyond scalars (int, float, etc.)
- vips7L 2y agoWouldn't the lambda here by always escape? It gets created in the function and passed into another one. Looking at the byte code it looks like it always allocates in when its in the interpreter: invokedynamic #16, 0 // InvokeDynamic #0:apply:()Ljava/util/function/Function;
- marginalia_nu 2y agoA repo with benchmark code is up now: https://github.com/vlofgren/Serialization1BRCBenchmark/ https://github.com/vlofgren/Serialization1BRCBenchmark/
- krackers 2y agoWhat's the reason why reading 3GB via duckdb is _faster_ than raw HDD speed? Is it compression/caching?
- marginalia_nu 2y agoThat's an in memory benchmark, to help give context to the figures in the duckdb parquet benchmarks.