4 ms·
curl https://www.washingtonpost.com/outlook/the-accidental-book-review-that-made-jack-kerouac-famous/2019/12/13/23e94230-04b3-11ea-b17d-8b867891d39d_story.html?
by 3xblah 7y ago
curl https://www.washingtonpost.com/outlook/the-accidental-book-review-that-made-jack-kerouac-famous/2019/12/13/23e94230-04b3-11ea-b17d-8b867891d39d_story.html?outputType=amp |tr -cd '\11\12\15\40-\176' |sed 's/</\n</g' | sed -n '/./{/<p class=.font--body/,/<\/p>/p;}' > 1.htm
firefox 1.htm
If desired, substitute browser of choice for "firefox". Can leave Javascript enabled.
- pjc50 7y agoIf you're calling from the EU, this gets stuck in an endless loop of GDPR consent redirects.
- spurgu 7y agoWhat I did was: 1. Visit the link until it complains about adblocker 2. Disable JS for washingtonpost.com (using NoScript) If I block JS directly I just get a GDPR (almost) blank page.
- 3xblah 7y agoNothing in the filtered output that could cause a redirect. If HTTP 3xx redirect, maybe try using a cache URL instead, e.g., curl http://web.archive.org/web/20191214075242/https://www.washingtonpost.com/outlook/the-accidental-book-review-that-made-jack-kerouac-famous/2019/12/13/23e94230-04b3-11ea-b17d-8b867891d39d_story.html?outputType=amp |tr -cd '\11\12\15\40-\176' |sed 's/</\n</g' | sed -n '/./{/<p class=.font--body/,/<\/p>/p;}' > 1.htm firefox 1.htm
- sogen 7y agonice! any way to make it work in Lynx/Elinks?
- dredmorbius 7y agolynx will read direct from stdin. <command> | lynx --force-html -stdin
- sogen 7y agothanks! it worked curl https://www.washingtonpost.com/outlook/the-accidental-book-review-that-made-jack-kerouac-famous/2019/12/13/23e94230-04b3-11ea-b17d-8b867891d39d_story.html\?outputType\=amp |tr -cd '\11\12\15\40-\176' |sed 's/</\n</g' | sed -n '/./{/<p class=.font--body/,/<\/p>/p;}' | lynx --force-html -stdin
- 3xblah 7y agoThere is little need for this if one is using a text-only browser such as links/elinks that does not process Javascript or CSS. AMP look great in links/elinks, without any filtering. This is for folks who do not use text-only browsers and prefer to leave Javascript enabled.
- dredmorbius 7y agoAs a shell script, feeding w3m directly (it will read stdin, as will lynx): #!/bin/sh set -e curl -s "${1}" | tr -cd '\11\12\15\40-\176' | sed 's/</\n</g' | sed -n '/./{/<p class=.font--body/,/<\/p>/p;}' | /usr/bin/w3m -T text/html I've called this "washpo". The argument is the URL you want view. If you're a stickler for well-formed HTML, you can add a pipeline for tidy as well ;-) tidy -i -w 120 --quiet | Immediately before your browser of choice. And if you want to retain the title and dateline, use this version: https://pastebin.com/raw/LcBp6gPB https://pastebin.com/raw/LcBp6gPB
- sogen 7y agow3m, nice! forgot I had it installed
- dredmorbius 7y agoA few further findings: 1. WashPo delay curl requests by about 8 seconds. Changing user-agent to anything other will defeat this. 2. The size of the actually-useful bits of the page is about 3% of just the primary HTML page transferred. That's excluding images, external CSS/JS, and prefetched content. That's a pretty considerable chunk.