3 ms·
curl is for examples on HN. In practice, I never use curl and I never send a user-agent header. If use -H"User-Agent" (no space after the H) it should work fi
by 1vuio0pswjnm7 5y ago
curl is for examples on HN. In practice, I never use curl and I never send a user-agent header. If use -H"User-Agent" (no space after the H) it should work fine. Example below. I cannot even think of a single website that requires a user-agent header in order to retrieve a page, but if you send one it will likely be used for something. Fingerprinting, changing the "UI" and blocking are a few uses that come to mind.
curl -H"User-Agent" https://www.wsj.com/amp/articles/if-you-sell-a-house-these-days-the-buyer-might-be-a-pension-fund-11617544801 \
|grep -Eo "(<p.*<\/p>)|(<h[12].*</h[12]>)|(<a href=.[^{].*</a>)" \
|tr -cd '[ -~]' > 1.htm
firefox ./1.htm