7 ms·
2 years ago, I realised we were accidentally DOS'ing GitHub and they didn't care. I was impressed. We had software for checking a piece of data. To do this, (w
by jarofgreen 6y ago
2 years ago, I realised we were accidentally DOS'ing GitHub and they didn't care. I was impressed.
We had software for checking a piece of data. To do this, (we realised later) it fetched a schema from GitHub every time. The context is this software was used infrequently so that was fine - and also there was good reasons why it should reload the schema every time.
We then took this bit of software and dumbly used it in a tool for checking masses of data at once. We did this quite happily for 2 months or so, until one day I was trying to work out why it didn't run that fast. (It could only check 2 or 3 pieces of data a second).
At this point I realised/discovered all the above, went "oh shit" and quickly slapped in a request caching library. We stopped DOS'ing GitHub and the amount of checks we could do per second went right up.
But I'm pretty sure that at no point did GitHub rate limit us or block us during this - for which I was very impressed.