3 ms·

Looks okay in links. Oddly I get a 403 using w3m.

by opan 6y ago

swiley 6y ago
The web is slowly becoming unusable in modern browsers, a reverse of the earlier trend.
davegauer 6y ago
Yes, 403 Forbidden in lynx as well. This is a very unfortunate trend I've seen increasing over the last couple years. The answer almost always comes from an nginx server, but my understanding is that it's User-Agent sniffing being done for DDOS protection and the use of nginx (being used as a proxy?) is incidental. I have not been able to find any good discussions about it.
- cmeacham98 6y ago
  I think this has more to do with nginx being one of (if not the) most popular *nix web servers, rather than nginx administrators hating lynx users. I run plenty of websites with nginx that you can browse with lynx just fine.
  - davegauer 6y ago
    Yes, sorry, I meant to make that more clear: this is NOT an nginx issue. It just HAPPENS that I always see nginx on the 403 page. It appears somebody doesn't like lynx's User-Agent string and it makes me sad.
- neop1x 6y ago
  Lynx is also sending HTTP/1.0: GET /linux-capabilities-in-a-nutshell/ HTTP/1.0 Host: k3a.me User-Agent: Lynx/2.9.0dev.5 libwww-FM/2.14 SSL-MM/1.4.1 GNUTLS/3.6.13 I am quite surprised actually. It is unfortunate that these clients still haven't been updated to at least HTTP/1.1. :( Just patch Lynx to send HTTP/1.1 without implementing keep-alive or any other HTTP/1.1 functionality and your troubles will most likely end. :P It is better to have broken 1.1 implementation than broken 1.0. CURL is using 2.0 for HTTPS and 1.1 for HTTP by default. Someone should upgrade these CLI browsers to use libcurl or something like that...
neop1x 6y ago
Author here. It is Nginx with ModSecurity. It looks like your client uses HTTP/1.0 which I blacklisted as HTTP/1.0 doesn't make sense nowadays as it originally didn't even support Host header for virtual hosts. HTTP/1.0 is often used by badly-written web scrappers.

Looks okay in links. Oddly I get a 403 using w3m.

by opan 6y ago

Looks okay in links. Oddly I get a 403 using w3m.

swiley 6y ago

The web is slowly becoming unusable in modern browsers, a reverse of the earlier trend.

davegauer 6y ago

Yes, 403 Forbidden in lynx as well. This is a very unfortunate trend I've seen increasing over the last couple years. The answer almost always comes from an nginx server, but my understanding is that it's User-Agent sniffing being done for DDOS protection and the use of nginx (being used as a proxy?) is incidental. I have not been able to find any good discussions about it.

cmeacham98 6y ago
I think this has more to do with nginx being one of (if not the) most popular *nix web servers, rather than nginx administrators hating lynx users. I run plenty of websites with nginx that you can browse with lynx just fine.
- davegauer 6y ago
  Yes, sorry, I meant to make that more clear: this is NOT an nginx issue. It just HAPPENS that I always see nginx on the 403 page. It appears somebody doesn't like lynx's User-Agent string and it makes me sad.
neop1x 6y ago
Lynx is also sending HTTP/1.0: GET /linux-capabilities-in-a-nutshell/ HTTP/1.0 Host: k3a.me User-Agent: Lynx/2.9.0dev.5 libwww-FM/2.14 SSL-MM/1.4.1 GNUTLS/3.6.13 I am quite surprised actually. It is unfortunate that these clients still haven't been updated to at least HTTP/1.1. :( Just patch Lynx to send HTTP/1.1 without implementing keep-alive or any other HTTP/1.1 functionality and your troubles will most likely end. :P It is better to have broken 1.1 implementation than broken 1.0. CURL is using 2.0 for HTTPS and 1.1 for HTTP by default. Someone should upgrade these CLI browsers to use libcurl or something like that...

neop1x 6y ago

Author here. It is Nginx with ModSecurity. It looks like your client uses HTTP/1.0 which I blacklisted as HTTP/1.0 doesn't make sense nowadays as it originally didn't even support Host header for virtual hosts. HTTP/1.0 is often used by badly-written web scrappers.