We run a Spark cluster for distributed processing. Spark is awesome - makes it dead easy to parallelize work. But it's also easy to write bad Spark code that

We run a Spark cluster for distributed processing. Spark is awesome - makes it dead easy to parallelize work. But it's also easy to write bad Spark code that doesn't parallelize well. So I wrote a report to let us view key stats broken out by Spark job, including a sparkline of avg load on the machines running the job for its duration. This provides an easy way to see whether a job uses the cluster efficiently, and if it doesn't, where the key points in the job are that it's sitting idle. It also lets us eyeball the repeatability -- if a given batch job's sparklines are wildly varying from run to run, it usually suggests that something funky is going on in the code. Also, it lets us have "Spark" lines. Worth it for the pun alone, IMO.