4 ms·

I've been staring down compiler flags for two weeks straight, now, since I've been working on cross-compilation toolchains, but that kind of narrative doesn't f

I've been staring down compiler flags for two weeks straight, now, since I've been working on cross-compilation toolchains, but that kind of narrative doesn't fit your current level of snark, I guess. I have run operations for programs written in Java, C++, and Go. I've done it both at small companies and large companies, for everywhere from a rack of dedicated servers running Apache Tomcat to thousands of virtual machines running a mixture of C++ and Go microservices. The constant thread is... over the last ten years... the people running JVMs complain about tuning the JVM, and always have stories to tell about it. The JVM is fantastically tuneable, and when people online complain about GC pauses or memory problems, there's often some way to tune the JVM to fix those problems. The JVM is amazing, it's a marvelous piece of technology. But there's also a bunch of people who don't know how to do it and just turn knobs. Like, oh, customers are calling in to complain about latency, so I'll increase the size of the Java heap. (Which, for those not familiar with GC, will make throughput better but make latency worse.) Running services on the JVM means that "working with the JVM" is now a skill you need to select for whatever operations team you have, and if you're a company running a mix of different services (like, I don't know, some databases, memcache, load balancers, etc) then "knowing how to tune the JVM" competes with several other skills you'd like your team to learn.

pjmlp 6y ago
Just like there are whole conference talks on how to use PGO data properly and optimization flags on AOT compilation toolchains, naturally one needs to spend the effort to learn how to use them and not just type -O2 and go home. Or those that complain about RDBMS queries being slow, without having normalized their data or written proper indexes. There is always bunch of people that don't know how to do things, and then there are those among them, that care to improve their skill set and get to know how to turn those knobs.
- klodolph 6y ago
  Sure, but it’s only at large scale that you care about stuff like PGO. My experience is that even small shops care about JVM tuning. It’s not a question of whether there are knobs to turn. It’s about the typical experience of an actual person running a JVM app versus, say, C or Go, and those experiences are quite different. And we’re also talking about running someone else’s app. If your database queries are slow, that’s a conversation between the DBA and the devs. If you’re running some Apache app, you’re probably not talking to the devs.
  - bokchoi 6y ago
    > My experience is that even small shops care about JVM tuning. This hasn't been my experience at all after running java on production servers for over a decade.
    - gpapilion 6y ago
      Unlike C/C++ you do have to care about tuning at a minimum to set the memory sizes. That said you probably can’t get very far in c/c++ without worrying about memory the whole way through. So, you only worry about tuning when writing your code.
  - Arelius 6y ago
    Yeah, I've been a C/C++ programmer for 15+ years with heavy focus on performance, and have not once run PGO. I think it's quite often the case for C/C++ programs that you -O2 and walk away, at least for codebases that your team doesn't specifically own.
  - wbl 6y ago
    When you run into that problem with Go you stop dead because there is nothing to do.
  - pjmlp 6y ago
    In 20 years of using Java I only had one project where fine-tuning knobs was actually relevant. Most of the time defaults just work.
vbezhenar 6y ago
I wrote Java for last ten years and never had to tune anything but Xmx. It just works for me.
- hedora 6y ago
  In that time, how often did you care about sub millisecond latency SLAs? (You can read a disk 10 times in series or 100-1000 times in parallel in 1ms; it’s practically an eternity.)
  - vbezhenar 6y ago
    Never. I don't think I would use Java (or any language with GC) for such kind of job, it's more suited for C/C++/Rust.
    - pjmlp 6y ago
      Depends, https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-technology-at-the-of-aegis-the-shield-of-the-fleet https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-te...
    - returnfalse 6y ago
      Then, you might be interested in watching this: https://www.youtube.com/watch?v=Pz-4co8IaI8 https://www.youtube.com/watch?v=Pz-4co8IaI8

pjmlp 6y ago

Just like there are whole conference talks on how to use PGO data properly and optimization flags on AOT compilation toolchains, naturally one needs to spend the effort to learn how to use them and not just type -O2 and go home. Or those that complain about RDBMS queries being slow, without having normalized their data or written proper indexes. There is always bunch of people that don't know how to do things, and then there are those among them, that care to improve their skill set and get to know how to turn those knobs.

klodolph 6y ago
Sure, but it’s only at large scale that you care about stuff like PGO. My experience is that even small shops care about JVM tuning. It’s not a question of whether there are knobs to turn. It’s about the typical experience of an actual person running a JVM app versus, say, C or Go, and those experiences are quite different. And we’re also talking about running someone else’s app. If your database queries are slow, that’s a conversation between the DBA and the devs. If you’re running some Apache app, you’re probably not talking to the devs.
- bokchoi 6y ago
  > My experience is that even small shops care about JVM tuning. This hasn't been my experience at all after running java on production servers for over a decade.
  - gpapilion 6y ago
    Unlike C/C++ you do have to care about tuning at a minimum to set the memory sizes. That said you probably can’t get very far in c/c++ without worrying about memory the whole way through. So, you only worry about tuning when writing your code.
- Arelius 6y ago
  Yeah, I've been a C/C++ programmer for 15+ years with heavy focus on performance, and have not once run PGO. I think it's quite often the case for C/C++ programs that you -O2 and walk away, at least for codebases that your team doesn't specifically own.
- wbl 6y ago
  When you run into that problem with Go you stop dead because there is nothing to do.
- pjmlp 6y ago
  In 20 years of using Java I only had one project where fine-tuning knobs was actually relevant. Most of the time defaults just work.

vbezhenar 6y ago

I wrote Java for last ten years and never had to tune anything but Xmx. It just works for me.

hedora 6y ago
In that time, how often did you care about sub millisecond latency SLAs? (You can read a disk 10 times in series or 100-1000 times in parallel in 1ms; it’s practically an eternity.)
- vbezhenar 6y ago
  Never. I don't think I would use Java (or any language with GC) for such kind of job, it's more suited for C/C++/Rust.
  - pjmlp 6y ago
    Depends, https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-technology-at-the-of-aegis-the-shield-of-the-fleet https://www.ptc.com/en/blogs/plm/ptc-perc-virtual-machine-te...
  - returnfalse 6y ago
    Then, you might be interested in watching this: https://www.youtube.com/watch?v=Pz-4co8IaI8 https://www.youtube.com/watch?v=Pz-4co8IaI8