4 ms·

The counter argument here is something like Lucene, which is written in Java but has been heavily optimized over the years using many of the same kinds of trick

The counter argument here is something like Lucene, which is written in Java but has been heavily optimized over the years using many of the same kinds of tricks the Git people used to optimize Git. There are frequent attempts to replicate parts of what Lucene does in different languages; usually under the assumption that it will be faster and better because of magical properties people associate with things like native compilation or the supposed manual memory management skills of the programmers doing the work. Most of these efforts are relatively niche compared to Lucene because they never quite catch up in terms of features, scale, etc. and because the programmers involved are messing around on the fringes of the problem space instead of coming up with algorithmic breakthroughs, which at this point is the quickest way to make things faster. Well, that and cutting some major corners and pretending it's all the same by running some silly benchmark. The fallacy here is confusing language, idioms, memory models, frameworks, etc. and assuming it's all set in stone. It isn't. Just because you are using Java does not mean everything has to be garbage collected, for example. Lucene actually uses memory mapped files, byte buffers, etc. for a lot of things. So, it does not actually need to do a lot of garbage collecting. It uses the same kinds of solutions you'd pick when using C. And they perform ballpark as you'd expect them too meaning that unless you improve the algorithms, you are not going to be magically a lot faster. The same is true of hotspot, the JVM's runtime compiler, which is written in C and, surprise, uses a lot of the same kind of trickery used by the fine people working on e.g. LLVM. So, a lot of things people assume must surely be slower just aren't and a lot of the insurmountable bottlenecks that people assume must surely be there always actually have well known ways of being worked around. Of course there is always room for more optimization. Lucene is well over two decades old now and they still regularly come up with major performance improvements. There's nothing magical about how they do that; just a lot of hard work that goes into it that would be somewhat challenging to improve on just by switching language and compilers.

kaba0 5y ago
Just a nitpick, but Hotspot is written in C++ (and asm).
- pjmlp 5y ago
  GraalVM and JikesRVM aren't though. As for Hotspot, some C++ subsystems are now in Java.
blub 5y ago
That’s not a counter-argument, because most Java developers can’t write such C-like code and more importantly they don’t want to. The debate was always about idiomatic Java vs. idiomatic C. What’s amazing to me is that I’ve had such discussions 15 years ago, 10 years ago, etc. I would have hoped that the Java community would have accepted the reality by now :-) Still, there’s also an interesting new development happening in this area: the members of the Rust community have picked up the mantle of native vs. managed performance. By now they’ve spent so much social capital on “rewrite it in Rust”, that it’s not clear if this will be a positive for native dev advocacy, but the show will definitely be fun to watch.
- pjmlp 5y ago
  Applying the same argument to C, C isn't a system programming language, because the use cases where it is used as such rely on compiler extensions not required for ISO C compliance.
- kaba0 5y ago
  But nor does most code require that sort of control over memory. Eg. would a program with very diverse code paths improve that much from fitting data into L1 cache better? I am genuinely asking it. Because of course for things like pipewire handling audio streams it is very important and Java would not be a good fit. But for a web application I really don’t see C beating Java by much if by any (and the techempower benchmark shows native languages at top only when those are heavily optimized very specifically eg. depending on the exact format of HTTP requests)
- josephg 5y ago
  > it’s not clear if this will be a positive for native dev advocacy I've rewritten a few things in rust. Seems pretty positive to me, because you can mix some of the best optimizations and data structures you'd write in C, with much better developer ergonomics. A few years ago I wrote a rope library in C. This is a library for making very fast, arbitrary insert & delete operations in a large string. My C code was about as fast as I could make it at the time. But recently, I took a stab at porting it to Rust to see if I could improve things. Long story short, the rust version is another ~3x faster than the C version. https://crates.io/crates/jumprope https://crates.io/crates/jumprope (Vs C equivalent: https://github.com/josephg/librope https://github.com/josephg/librope ) The competition absolutely isn't fair. In rust, I managed to add another optimization that doesn't exist in the C code. I could add it in C, but it would have been really awkward to weave in. Possible, but awkward in an already very complex bit of C. In rust it was much easier because of the language's ergonomics. In C I'm using lots of complex memory management and I don't want to add complexity in case I add memory corruption bugs. In rust, well, the optimization was entirely safe code. And as for other languages - I challenge anyone to even approach this level of performance in a non-native language. I'm processing ~30M text edit operations per second. A few years ago I tried something similar in JS and got about 100k edits per second. (300x slower) But these sort of performance results probably won't scale for a broader group of programmers. I've seen rust code run slower than equivalent javascript code because the programmers, used to having a GC, just Box<>'ed everything. And all the heap allocations killed performance. If you naively port python line-by-line to rust, you can't expect to magically get 100x the performance. Its like, if you give a top of the line Porsche to an expert driver, they can absolutely drive faster. But I'm not an expert driver, so I'll probably crash the darn thing. I'd take a simple toyota or something any day. I feel like rust is the porsche, and python is the toyota.
zerocount 5y ago
There's no counter argument to an article that specifically breaks down why C git is faster that JGit back in 2009. Lucene isn't relevant.
JetSetWilly 5y ago
That's just because there's no a lucene equivalent C library with the same level of attention? however, there are increasingly such written in C++ (pisa) and rust (tantivy). They handily beat lucene in benchmark suites [1] - so it seems like lucene does suffer from a java penalty - despite getting even more developer attention than pisa and tantivy I would think. 1: https://tantivy-search.github.io/bench/ https://tantivy-search.github.io/bench/
- jillesvangurp 5y ago
  No, because a lot of the work would end up being the same kind of work with no inherent advantages of one over the other. Easy to predict because there used to be a C implementation of Lucene. It couldn't keep up in terms of features or performance so work on that stopped a long time ago. Most of the libraries you mention don't come close to even implementing a tiny portion of Lucene. Classic case of apples and oranges. Also, good examples of the niche things I was talking about. Benchmarks like you mention are kind of self serving like that. They measure something but not everything and probably very selectively. They are faster at what exactly? Under what circumstances? Why? As soon as you answer those questions, what inherent limitations do the Lucene developers have replicating that? A lot here boils down to how the underlying search engine implements tokenization, stemming, language analyzers, fuzzy matching, and a few other things that you'd need to build a search engine that doesn't suck. The benchmark conveniently does not specify any of that; presumably because it lacks many of those features or has extremely naive implementations of those things. That would be the kind of corner cutting I was talking about. What kind of relevance ranking is being used here? How good is it? Was that even evaluated or considered? Hint, Lucene gives you many options here. Search quality and performance are the big trade off here. Anything that trades off quality over performance is going to look good until you look at the quality. Also things like the index size and document volume are not specified. Or the hardware this ran on. Or the JVM configuration, compiler flags, etc. It's like benchmarking a formula 1 car by how quick it is at parallel parking. Yes, a T-Ford is going to be faster at that maybe. But is that even meaningful to look at?
  - JetSetWilly 5y ago
mohanmca 5y ago
Well, It is not Lucene alone, there are many tools in java written around Lucene such as ElasticSearch/Solr/Elassandra and so on, so no language or runtime can replace everything. Eventually something might catch-up. That is why it is not language alone! It is developer, runtime, ecosystem and problem domain, brainshare.
omginternets 5y ago
Where does one go to learn all these optimization strategies and tactics? I really want to take a more deliberate approach to optimizing my (Go) code, but I’m currently stuck banging on things with a wrench between benchmarks. Frustratingly, most advice I get is non-actionable, and amounts to restating general principles like “be aware of cache”. Are there any books or online resources that can help me build a more principled approach to runtime performance?
- halpert 5y ago
  The general process I’ve seen is to profile the application and look at what’s taking the most time. Then you sit down and think about how you could make the slowest parts faster. There isn’t a one-size-fits-all answer to your question. Benchmarks are good for validating your improvements, but not for finding what to improve. That’s what profiling is for.
  - omginternets 5y ago
    Yeah; this is what I meant by “banging on things with a wrench between benchmarks”. I can’t fight the feeling that there is some background knowledge that makes the search more directed.
    - lossolo 5y ago
      There is some background knowledge, like knowing time and space complexity of algorithms and datastructures that you use and how this abstractions interact with underlying hardware on which you are running your application. Knowing how CPU, CPU caches, OS memory management, as you mentioned that you work with Go so Go internals (like goroutines scheduling, how memory is allocated by Go runtime, how GC works etc), kernel task scheduling (context switching etc) work and how they impact your workload. A lot of it is intuition that you get with working with code that needs optimising throughout the years and a LOT of reading but sometimes you can flush that intuition into the toilet because you often find bottlenecks in places you wouldn't think about. So it really depends on the nature of your application (is it a network application or HPC, multi threaded? single threaded? distributed? you are optimising for high throughput or low latency, do you control hardware, network etc)
burntoutfire 5y ago
> Lucene actually uses memory mapped files, byte buffers How do they cast raw memory pointers into Java types? In C, you just cast a void pointer to a pointer to intended type, and voila - that piece of memory is interpreted as if an object of that particular type resided there. Is anything of this sort possible in Java?
geodel 5y ago
One need to see the amount of resources to run Lucene or its enterprise solution like Elasticsearch. So Java based solution work because enterprise can through lot of resources at it. But it does not mean they are efficient or faster. Else Java folks at Oracle wouldn't be spending decade long effort on flatter memory layout of Java objects.
jcranberry 5y ago
I don't see how this is a counterargument, or highlighting any fallacy. The article talks about hacks used to get around garbage collection in Java and increase performance, including memory mapped files and byte buffers. It also talks about the pain points of using these hacks and the costs they have to pay despite these hacks. It also concludes that it is practical to build Git in a higher level language and JGit performs reasonably well, despite not getting quite C level performance or its memory utilization.
bob1029 5y ago
You also see the same thing with fintech areas like ringbuffer-based order matching engines. Sure, with an infinite amount of time you could make a C implementation run a little bit faster, but practically speaking you would have a hard time building something that gives the business the same level of confidence as a Java/C# solution. Especially, if there's piles of manual memory management and ASM hacks going on. In my experience, C#/.NET is in a really interesting position when it comes to these questions. It does support value types out of the box which can add another order of magnitude over Java implementations of the same code. One example of this is the LMAX Disruptor. This was originally written for Java and then ported to C#. Because C# supports value types, a special ValueDisruptor variant was developed specific to its port, which enables performance that would otherwise be impossible in the Java solution (which is constrained to using reference types in the buffer). https://medium.com/@ocoanet/improving-net-disruptor-performance-part-3-introducing-the-valuedisruptor-5b467730bbe https://medium.com/@ocoanet/improving-net-disruptor-performa...
- jcranberry 5y ago
  You definitely don't need an infinite amount of time to make C or C++ implementation of a matching engine run faster than a Java based one. I've worked at two fintech companies, a prop shop and an investment bank and seen some APIs for other companies' matching engines and they've pretty much all been C/C++.
coliveira 5y ago
Lucene is the typical example of software that works well despite not being as fast as possible, because its features are more important than raw speed that C could provide. Nothing wrong about that. After all, that's why we buy more hardware. However, it doesn't disprove the point of the article. It could be possible to build something at much better speed than Lucene if there was the interest and man power to do that in C.

kaba0 5y ago

Just a nitpick, but Hotspot is written in C++ (and asm).

pjmlp 5y ago
GraalVM and JikesRVM aren't though. As for Hotspot, some C++ subsystems are now in Java.

blub 5y ago

That’s not a counter-argument, because most Java developers can’t write such C-like code and more importantly they don’t want to. The debate was always about idiomatic Java vs. idiomatic C. What’s amazing to me is that I’ve had such discussions 15 years ago, 10 years ago, etc. I would have hoped that the Java community would have accepted the reality by now :-) Still, there’s also an interesting new development happening in this area: the members of the Rust community have picked up the mantle of native vs. managed performance. By now they’ve spent so much social capital on “rewrite it in Rust”, that it’s not clear if this will be a positive for native dev advocacy, but the show will definitely be fun to watch.

pjmlp 5y ago
Applying the same argument to C, C isn't a system programming language, because the use cases where it is used as such rely on compiler extensions not required for ISO C compliance.
kaba0 5y ago
But nor does most code require that sort of control over memory. Eg. would a program with very diverse code paths improve that much from fitting data into L1 cache better? I am genuinely asking it. Because of course for things like pipewire handling audio streams it is very important and Java would not be a good fit. But for a web application I really don’t see C beating Java by much if by any (and the techempower benchmark shows native languages at top only when those are heavily optimized very specifically eg. depending on the exact format of HTTP requests)
josephg 5y ago
> it’s not clear if this will be a positive for native dev advocacy I've rewritten a few things in rust. Seems pretty positive to me, because you can mix some of the best optimizations and data structures you'd write in C, with much better developer ergonomics. A few years ago I wrote a rope library in C. This is a library for making very fast, arbitrary insert & delete operations in a large string. My C code was about as fast as I could make it at the time. But recently, I took a stab at porting it to Rust to see if I could improve things. Long story short, the rust version is another ~3x faster than the C version. https://crates.io/crates/jumprope https://crates.io/crates/jumprope (Vs C equivalent: https://github.com/josephg/librope https://github.com/josephg/librope ) The competition absolutely isn't fair. In rust, I managed to add another optimization that doesn't exist in the C code. I could add it in C, but it would have been really awkward to weave in. Possible, but awkward in an already very complex bit of C. In rust it was much easier because of the language's ergonomics. In C I'm using lots of complex memory management and I don't want to add complexity in case I add memory corruption bugs. In rust, well, the optimization was entirely safe code. And as for other languages - I challenge anyone to even approach this level of performance in a non-native language. I'm processing ~30M text edit operations per second. A few years ago I tried something similar in JS and got about 100k edits per second. (300x slower) But these sort of performance results probably won't scale for a broader group of programmers. I've seen rust code run slower than equivalent javascript code because the programmers, used to having a GC, just Box<>'ed everything. And all the heap allocations killed performance. If you naively port python line-by-line to rust, you can't expect to magically get 100x the performance. Its like, if you give a top of the line Porsche to an expert driver, they can absolutely drive faster. But I'm not an expert driver, so I'll probably crash the darn thing. I'd take a simple toyota or something any day. I feel like rust is the porsche, and python is the toyota.

zerocount 5y ago

There's no counter argument to an article that specifically breaks down why C git is faster that JGit back in 2009. Lucene isn't relevant.

JetSetWilly 5y ago

That's just because there's no a lucene equivalent C library with the same level of attention? however, there are increasingly such written in C++ (pisa) and rust (tantivy). They handily beat lucene in benchmark suites [1] - so it seems like lucene does suffer from a java penalty - despite getting even more developer attention than pisa and tantivy I would think. 1: https://tantivy-search.github.io/bench/ https://tantivy-search.github.io/bench/

jillesvangurp 5y ago
No, because a lot of the work would end up being the same kind of work with no inherent advantages of one over the other. Easy to predict because there used to be a C implementation of Lucene. It couldn't keep up in terms of features or performance so work on that stopped a long time ago. Most of the libraries you mention don't come close to even implementing a tiny portion of Lucene. Classic case of apples and oranges. Also, good examples of the niche things I was talking about. Benchmarks like you mention are kind of self serving like that. They measure something but not everything and probably very selectively. They are faster at what exactly? Under what circumstances? Why? As soon as you answer those questions, what inherent limitations do the Lucene developers have replicating that? A lot here boils down to how the underlying search engine implements tokenization, stemming, language analyzers, fuzzy matching, and a few other things that you'd need to build a search engine that doesn't suck. The benchmark conveniently does not specify any of that; presumably because it lacks many of those features or has extremely naive implementations of those things. That would be the kind of corner cutting I was talking about. What kind of relevance ranking is being used here? How good is it? Was that even evaluated or considered? Hint, Lucene gives you many options here. Search quality and performance are the big trade off here. Anything that trades off quality over performance is going to look good until you look at the quality. Also things like the index size and document volume are not specified. Or the hardware this ran on. Or the JVM configuration, compiler flags, etc. It's like benchmarking a formula 1 car by how quick it is at parallel parking. Yes, a T-Ford is going to be faster at that maybe. But is that even meaningful to look at?
- JetSetWilly 5y ago

mohanmca 5y ago

Well, It is not Lucene alone, there are many tools in java written around Lucene such as ElasticSearch/Solr/Elassandra and so on, so no language or runtime can replace everything. Eventually something might catch-up. That is why it is not language alone! It is developer, runtime, ecosystem and problem domain, brainshare.

omginternets 5y ago

Where does one go to learn all these optimization strategies and tactics? I really want to take a more deliberate approach to optimizing my (Go) code, but I’m currently stuck banging on things with a wrench between benchmarks. Frustratingly, most advice I get is non-actionable, and amounts to restating general principles like “be aware of cache”. Are there any books or online resources that can help me build a more principled approach to runtime performance?

halpert 5y ago
The general process I’ve seen is to profile the application and look at what’s taking the most time. Then you sit down and think about how you could make the slowest parts faster. There isn’t a one-size-fits-all answer to your question. Benchmarks are good for validating your improvements, but not for finding what to improve. That’s what profiling is for.
- omginternets 5y ago
  Yeah; this is what I meant by “banging on things with a wrench between benchmarks”. I can’t fight the feeling that there is some background knowledge that makes the search more directed.
  - lossolo 5y ago
    There is some background knowledge, like knowing time and space complexity of algorithms and datastructures that you use and how this abstractions interact with underlying hardware on which you are running your application. Knowing how CPU, CPU caches, OS memory management, as you mentioned that you work with Go so Go internals (like goroutines scheduling, how memory is allocated by Go runtime, how GC works etc), kernel task scheduling (context switching etc) work and how they impact your workload. A lot of it is intuition that you get with working with code that needs optimising throughout the years and a LOT of reading but sometimes you can flush that intuition into the toilet because you often find bottlenecks in places you wouldn't think about. So it really depends on the nature of your application (is it a network application or HPC, multi threaded? single threaded? distributed? you are optimising for high throughput or low latency, do you control hardware, network etc)

burntoutfire 5y ago

> Lucene actually uses memory mapped files, byte buffers How do they cast raw memory pointers into Java types? In C, you just cast a void pointer to a pointer to intended type, and voila - that piece of memory is interpreted as if an object of that particular type resided there. Is anything of this sort possible in Java?

geodel 5y ago

One need to see the amount of resources to run Lucene or its enterprise solution like Elasticsearch. So Java based solution work because enterprise can through lot of resources at it. But it does not mean they are efficient or faster. Else Java folks at Oracle wouldn't be spending decade long effort on flatter memory layout of Java objects.

jcranberry 5y ago

I don't see how this is a counterargument, or highlighting any fallacy. The article talks about hacks used to get around garbage collection in Java and increase performance, including memory mapped files and byte buffers. It also talks about the pain points of using these hacks and the costs they have to pay despite these hacks. It also concludes that it is practical to build Git in a higher level language and JGit performs reasonably well, despite not getting quite C level performance or its memory utilization.

bob1029 5y ago

You also see the same thing with fintech areas like ringbuffer-based order matching engines. Sure, with an infinite amount of time you could make a C implementation run a little bit faster, but practically speaking you would have a hard time building something that gives the business the same level of confidence as a Java/C# solution. Especially, if there's piles of manual memory management and ASM hacks going on. In my experience, C#/.NET is in a really interesting position when it comes to these questions. It does support value types out of the box which can add another order of magnitude over Java implementations of the same code. One example of this is the LMAX Disruptor. This was originally written for Java and then ported to C#. Because C# supports value types, a special ValueDisruptor variant was developed specific to its port, which enables performance that would otherwise be impossible in the Java solution (which is constrained to using reference types in the buffer). https://medium.com/@ocoanet/improving-net-disruptor-performance-part-3-introducing-the-valuedisruptor-5b467730bbe https://medium.com/@ocoanet/improving-net-disruptor-performa...

jcranberry 5y ago
You definitely don't need an infinite amount of time to make C or C++ implementation of a matching engine run faster than a Java based one. I've worked at two fintech companies, a prop shop and an investment bank and seen some APIs for other companies' matching engines and they've pretty much all been C/C++.

coliveira 5y ago

Lucene is the typical example of software that works well despite not being as fast as possible, because its features are more important than raw speed that C could provide. Nothing wrong about that. After all, that's why we buy more hardware. However, it doesn't disprove the point of the article. It could be possible to build something at much better speed than Lucene if there was the interest and man power to do that in C.