5 ms·
Story time. This took place around 2009. Back then I was working for Rapid7, on their network vulnerability scanner Nexpose. The codebase was mostly Java and w
by mrb 8y ago
Story time.
This took place around 2009. Back then I was working for Rapid7, on their network vulnerability scanner Nexpose. The codebase was mostly Java and was relatively large at 1M+ lines of code. We had many unit tests. Running the entire test suite took up to 40 minutes on Windows, and 20 minutes on Linux (Windows was always about twice slower on everything: building, product's startup time, etc.) The company had grown quickly to at least 30-50 software engineers. The problem was that every time one of them ran a build on his or her local machine (which happened multiple times a day) it would have to run the test suite and waste up to 40 minutes of this person's time. 40 minutes × dozens of engineers = lots of inefficiencies in the company.
I loved solving performance issues so one day I remember arriving at the office and making it my mission to investigate if there was an easy way to speed up the test suite. Our build system was based on Ant, and our tests used the JUnit framework. After a little time profiling Ant with basic tools (top, strace, ltrace), and taking a few Java stack traces, I realized that most of the wasted time was not actually running the individual tests, but many instances of the JVM kept being started and destroyed between each test. Our Ant build file was running the JUnit test with fork=yes, which was required for a reason I don't recall at the moment. This forks the JVM for running the tests. Then a little googling lead me to this:
https://ant.apache.org/manual/Tasks/junit.html https://ant.apache.org/manual/Tasks/junit.html
While reading this documentation, I stumbled upon an unknown parameter to me: forkmode. What does it do?
"forkmode: Controls how many JVMs get created if you want to fork some tests. Possible values are perTest (the default), perBatch and once. once creates only a single JVM for all tests while perTest creates a new JVM for each TestCase class. perBatch creates a JVM for each nested <batchtest> and one collecting all nested <test>s."
Our Ant build file did not set forkmode, so it meant we were forking a new JVM for every test!
I immediately tried forkmode=perBatch and... the test suite ran 10× faster! 40 minutes down to 4 minutes on Windows. And Linux ran it in 2 minutes instead of 20 minutes. I told my boss right away but he was unbelieving. He asked that I check with our most-experienced Java developer. I showed him my 1-line patch speeding the test suite 10× and he said "I guess you are right, we can commit that." By lunch time the fix was committed and everyone loved me :)
- quickthrower2 8y agoThat’s the time to ask for a raise!
- namdnay 8y agoI’ve always wondered, does this happen in US companies? Or is it only in very small shops? I’ve worked in a variety of big European firms, and salaries and promotions are discussed once a year, at review time. Even when changing jobs within the same firm, the salary will only be aligned at the next cycle
- tonyjstark 8y agoExcept if you have some leverage like an offer from another company, then sometimes you can make your current employer to almost match the offer. Once I improved some workflow greatly because I took extra time for it (it was about some Lua debugging, I don't really remember the circumstances), I was told that this is part of my job, so no raise. Now I don't expect raises for things like that but I only invest extra time if an issue annoys me and even then I open a ticket first. Long build times are annoying.
- jeevest 8y agoLegit reason. He’s saving hours of engineers time, every day. Hope they gave him something in some other form