12 ms·
I have a blog where I post a few posts per year. [1] /feed.xml is served with an Expires header of 24 hours. I wrote a tool that allows me to query the webserve
by ruuda 2y ago
I have a blog where I post a few posts per year. [1] /feed.xml is served with an Expires header of 24 hours. I wrote a tool that allows me to query the webserver logs using SQLite [2]. Over the past 90 days, these are the top 10 requesters grouped by ip address (remote_addr column redacted here):
requests_per_day user_agent
283 Reeder/5050001 CFNetwork/1568.300.101 Darwin/24.2.0
274 CommaFeed/4.4.0 (https://github.com/Athou/commafeed)
127 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
52 NetNewsWire (RSS Reader; https://netnewswire.com/)
47 Tiny Tiny RSS/23.04-0578bf80 (https://tt-rss.org/)
47 Refeed Reader/v1 (+https://www.refeed.dev/)
46 Selfoss/2.18 (SimplePie/1.5.1; +https://selfoss.aditu.de)
41 Reeder/5040601 CFNetwork/1568.100.1.1.1 Darwin/24.0.0
39 Tiny Tiny RSS/23.04 (Unsupported) (https://tt-rss.org/)
34 FreshRSS/1.24.3 (Linux; https://freshrss.org)
Reeder is loading the feed every 5 minutes, and in the vast majority of cases it’s getting a 301 response because it tries to access the http version that redirects to https. At least it has state and it gets 304 Not Modified in the remaining cases.
If I order by body bytes served rather than number of requests (and group by remote_addr again), these are the worst consumers:
body_megabytes_per_year user_agent
149.75943975 Refeed Reader/v1 (+https://www.refeed.dev/)
95.90771025 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36
75.00080025 rss-parser
73.023702 Tiny Tiny RSS/24.09-0163884ef (Unsupported) (https://tt-rss.org/)
38.402385 Tiny Tiny RSS/24.11-42ebdb02 (https://tt-rss.org/)
37.984539 Selfoss/2.20-cf74581 (+https://selfoss.aditu.de)
30.3982965 NetNewsWire (RSS Reader; https://netnewswire.com/)
28.18013325 Tiny Tiny RSS/23.04-0578bf80 (https://tt-rss.org/)
26.330142 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
24.838461 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
The top consumer, Refeed, is responsible for about 2.25% of all egress of my webserver. (Counting only body bytes, not http overhead.)
[1]: https://ruudvanasseldonk.com/writing https://ruudvanasseldonk.com/writing
[2]: https://github.com/ruuda/sqlog/blob/d129db35da9bbf95d8c2e97d575b4d5beb3bb40c/queries/feed_readers.sql https://github.com/ruuda/sqlog/blob/d129db35da9bbf95d8c2e97d...