3 ms·

Oh hey, I wrote that last issue linked! What crazy Deja vu. Here’s me discovering the issue that led me to find some wild behavior. Basically Firefox loaded

Oh hey, I wrote that last issue linked! What crazy Deja vu. Here’s me discovering the issue that led me to find some wild behavior. Basically Firefox loaded favicons 4x the number of tabs opened to that website. It would do this every time I opened or closed any tab. https://aggressivelyparaphrasing.me/2022/12/12/why-does-my-low-traffic-wordpress-blog-http-503-service-unavailable-when-i-click-around-using-firefox-on-iphone/ https://aggressivelyparaphrasing.me/2022/12/12/why-does-my-l... It was resolved a while back so maybe it’s similar symptoms but different root cause, or maybe it’s people using older versions?

aaron695 2y ago
[dead]
gwd 2y ago
Wordpress handles 404s really slowly? I'm kind of surprised it works at all then, as at least in my logs there's a very steady stream of bots probing it for vulnerabilities by trying random URLs.
- swiftcoder 2y ago
  It may intentionally 404 slowly? One web service I worked on added a few hundred milliseconds delay in returning 404s to slow down this kind of probing attack
  - gwd 2y ago
    Ooh, that's a good idea actually. But it doesn't explain this: > If I click enough, I’d eventually see HTTP 503 Service Unavailable. That normally only happens when the reverse proxy has a timeout, which would normally only happen when the backend was completely overloaded. Unless WP has an exponential delay, and the 503 is just the exponential delay becoming longer than the reverse proxy timeout? But why would the main page that the guy is loading say 503, when random non-critical parts like favicon.ico get a 503? Unless the exponential delay is per IP address -- so all the misses to favicon.ico are actually slowing down the main connections past the reverse proxy timeout? EDIT: Actually, no, they have graphs of the server actually spiking memory and CPU usage; you'd expect intentional exponential delay to reduce memory and CPU usage.
- markerz 2y ago
  I managed to get around it with litespeed Cache which does cache 404 pages. I was previously using WP Super Cache which does not. Note I also wasn’t running a CDN so there’s no reverse proxy cache either. Over time, I found that BetterLinks was slowing down my site significantly (600ms) . It wasn’t like this when I first investigated. It became slow over the course of a year or so. I ended up replacing it with Simple 301 Redirects. I think this is a separate issue though, unrelated to my original overload, but looked very similar to when Firefox DOSed my site. I experimented with CDNs to cache things reverse proxy style as a catch all. Eventually I caved and enabled Cloudflare CDN because QUIC.cloud kept having problems where a POP node kept hitting 403 Forbidden. I’d say the site is pretty functionally performant now. I think most sites that claim Wordpress handles high loads really well have at least two layers of caching in front of it and are running on dedicated boxes. Remove both of those and suddenly it’s super easy to DOS. Another common DOS exploit is to repeatedly spam the Forgot Password form, since there’s a lot of guaranteed processing with that and it’s not cacheable. I hid mine behind a captcha which helps a lot.

aaron695 2y ago

[dead]

gwd 2y ago

Wordpress handles 404s really slowly? I'm kind of surprised it works at all then, as at least in my logs there's a very steady stream of bots probing it for vulnerabilities by trying random URLs.

swiftcoder 2y ago
It may intentionally 404 slowly? One web service I worked on added a few hundred milliseconds delay in returning 404s to slow down this kind of probing attack
- gwd 2y ago
  Ooh, that's a good idea actually. But it doesn't explain this: > If I click enough, I’d eventually see HTTP 503 Service Unavailable. That normally only happens when the reverse proxy has a timeout, which would normally only happen when the backend was completely overloaded. Unless WP has an exponential delay, and the 503 is just the exponential delay becoming longer than the reverse proxy timeout? But why would the main page that the guy is loading say 503, when random non-critical parts like favicon.ico get a 503? Unless the exponential delay is per IP address -- so all the misses to favicon.ico are actually slowing down the main connections past the reverse proxy timeout? EDIT: Actually, no, they have graphs of the server actually spiking memory and CPU usage; you'd expect intentional exponential delay to reduce memory and CPU usage.
markerz 2y ago
I managed to get around it with litespeed Cache which does cache 404 pages. I was previously using WP Super Cache which does not. Note I also wasn’t running a CDN so there’s no reverse proxy cache either. Over time, I found that BetterLinks was slowing down my site significantly (600ms) . It wasn’t like this when I first investigated. It became slow over the course of a year or so. I ended up replacing it with Simple 301 Redirects. I think this is a separate issue though, unrelated to my original overload, but looked very similar to when Firefox DOSed my site. I experimented with CDNs to cache things reverse proxy style as a catch all. Eventually I caved and enabled Cloudflare CDN because QUIC.cloud kept having problems where a POP node kept hitting 403 Forbidden. I’d say the site is pretty functionally performant now. I think most sites that claim Wordpress handles high loads really well have at least two layers of caching in front of it and are running on dedicated boxes. Remove both of those and suddenly it’s super easy to DOS. Another common DOS exploit is to repeatedly spam the Forgot Password form, since there’s a lot of guaranteed processing with that and it’s not cacheable. I hid mine behind a captcha which helps a lot.