4 ms·
I’ve built and run a performance monitoring platform/team at a large tech company. I’ve also worked in security for the past few years. Both have similar probl
by leoqa 4y ago
I’ve built and run a performance monitoring platform/team at a large tech company. I’ve also worked in security for the past few years.
Both have similar problems: if the business goals don’t align with reducing latency or improved security posture then your detailed analysis will get parked in a ticket queue and promptly forgotten. Your OKRs will be missed. You will also be fighting the tide against new features developed sloppily- as latency will increase and then investment in your team seems like it’s not working to the executives. Fortunately we had a supportive directive but being siloed into an infra org didn’t give us the juice to influence product roadmaps.
We built a lab for testing against devices, did a bunch of work to isolate noise, integrated automated regression testing into CI to combat this. It still didn’t move the needle frankly, as the teams building products were not held accountable for the regressions (and don’t forget that it’s likely the right move for the greater business health!).
Netflix is unique in that latency is very core to their product.