3 ms·
Reminds me of a support ticket I had to investigate recently. Performance metrics were taking a dive and triggered automated alerts - average response time jum
by saltminer 3y ago
Reminds me of a support ticket I had to investigate recently.
Performance metrics were taking a dive and triggered automated alerts - average response time jumped from 200ms to 2000, 8000, and eventually began approaching 15000, at which point requests were timing out all over the place. At first I was wondering if my recently-deployed MR was responsible, but upon further investigation, it was from one developer on another team doing two very dumb things with their application prototype:
- Constantly retrying the same query with no filtering and setting the maximum allowed page size
- Sending massive quantities of small queries for specific individuals (50-100 per second) which quickly hit the rate limit, and immediately resuming after the rate limit expired
Some developer outreach was required...