4 ms·
> I ran a stress test of 14k RPS My built-in Devil's Advocate feels compelled to ask: do you have any reason to believe that your production site would get eve
by sgbeal 2y ago
> I ran a stress test of 14k RPS
My built-in Devil's Advocate feels compelled to ask: do you have any reason to believe that your production site would get even a small fraction (single-digit percentage) of that traffic?
As a point of comparison, i have it on good authority that sqlite3's own sites typically get far less than 0.1% of that traffic. Two of those sites (sqlite.org/src and sqlite.org/forum) are database-driven applications.
- kgeist 2y agoSure I don't expect my application to have 14k RPS all the time. However, some spikes can happen, and it can leave you with a large WAL file which doesn't shrink by itself by default. 14k RPS is the maximum my PC could pull off, I think it can happen with a more modest RPS. I ran the stress test with 12 parallel threads, so technically it can happen with just 12 RPS too, if your read or write transactions take a lot of time each (say, it has to read a large table, or you forgot to add indexes, or write transactions update many rows). In fact, a single open read transaction already can trigger checkpoint starvation. Also, it's not only about HTTP requests, my application is event-driven so a few event handlers processing events in the background can easily trigger it, too. A system can accidentally produce many events without having a lot of user requests (say, you forgot to batch events and generate N separate events for N items).
- anze3db 2y agoWhile this blog post was at the #2 spot on HN it was getting 0.138 rps, so yeah 14k RPS is A LOT! That said it's still good to know when your tools will break! As far as I understand it's not really about RPS but more about constant write operations preventing the WAL file from being flushed. You can reproduce this issue with a lot fewer RPS.