> How did you generate those flamegraphs and what other tools did you use to measure performance? We have an internal shell script that takes hwpmc output and

> How did you generate those flamegraphs and what other tools did you use to measure performance? We have an internal shell script that takes hwpmc output and generates flamegraphs from the stacks. It also works with dtrace. I'm a huge fan of dtrace. I also make heavy use of lockstat, AMD uProf, and Intel Vtune. > Did the graph show the bottleneck contention on aio queue? Did the graph show that "a lot of time was spent accessing memory"? See the graph on page 32 or so of the presentation. It shows huge plateaus in lock_delay called out of the aio code. Its also obvious from lockstat stacks (run as lockstat -x aggsize=4m -s 10 sleep 10 > results.txt) See the graph on page 38 or so. The plateaus are mostly memory copy functions (memcpy, copyin, copyout). We already use FreeBSD on our CDN, so it just made sense to do the work in FreeBSD. The talk is on Youtube https://youtu.be/36qZYL5RlgY https://youtu.be/36qZYL5RlgY