4 ms·

Looks like you need to set the user agent for this to work (I went with "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (K HTML,

• 1vuio0pswjnm7 5y ago
  curl is for examples on HN. In practice, I never use curl and I never send a user-agent header. If use -H"User-Agent" (no space after the H) it should work fine. Example below. I cannot even think of a single website that requires a user-agent header in order to retrieve a page, but if you send one it will likely be used for something. Fingerprinting, changing the "UI" and blocking are a few uses that come to mind. curl -H"User-Agent" https://www.wsj.com/amp/articles/if-you-sell-a-house-these-days-the-buyer-might-be-a-pension-fund-11617544801 \ |grep -Eo "(<p.*<\/p>)|(<h[12].*</h[12]>)|(<a href=.[^{].*</a>)" \ |tr -cd '[ -~]' > 1.htm firefox ./1.htm