4 ms·
"Maybe if you have script disabled." You are making the assumption that I am using a browser to retrieve the page. I can use a simple http client with no feat
by 3xblah 7y ago
"Maybe if you have script disabled."
You are making the assumption that I am using a browser to retrieve the page. I can use a simple http client with no features such as JS to retrieve the page. I have full control over what headers I send and the contents of those headers.
"You clearly haven't seen mandatory javascript sites."
I have seen many sites that prod the user to enable JS yet still return the same text/html even when JS is disabled. Sometimes I am only after the content of the page: text, URLs pointing to images or other resources, URLs pointing to other sites, etc.
I have seen some sites where there is "no content" in the initial page fetched, with or without JS enabled. These are dummy sites, hollow shells, skeleton pages. The technique is simply misdirection. All the useful content is pulled from another site via a second request (JS triggering the request). Maybe this is a CDN. I simply read the JS or use a debugger to find this "endpoint". Then use the simple http client to request the content as usual. These dummy sites are not the majority of sites on the web, nor even the majority of sites posted to HN.
The endpoints supporting these "no content", dummy sites often deliver the text with fewer or no tags, without HTML. Maybe it is JSON. This makes it easier for me to format it the way I like it. What many HN readers may perceive as a growing nuisance I believe could be a blessing in disguise. I prefer delivery of content in a more "raw" format allowing the user to process it as she pleases client-side, with the software of her choice. This leaves no room for advertising. It is more efficient and the user becomes the creative one, deciding how she wants to present the content to her own eyes.
- muxator 7y agoIs this really your day-to-day way of browsing the internet? Like if you were web scraping all the time? If so, I want to shake your hand, sir.
- 3xblah 7y agoFor commercial use of the web, things like internet banking, purchasing, etc., I use what is "recommended". The popular web browser. For non-commercial use of the web, I use what I want to use. A simple http client and/or text-only browser. My day-to-day use of the internet is almost always 100% non-commercial. I never want to use a Javascript-enabled browser for simply retrieving information from the web. For example, reading. This is non-commercial activity. For simple information retrieval, I find a large, complex, popular, recommended "browser", a single, "do-everything" program, is overkill and, counterintuitively perhaps, such browsers loaded with "features" actually limit what I can do. Using smaller, simpler "one-purpose" programs to retrieve web pages allows me more flexibiilty. I can be more productive. Using single-purpose programs, I also find the retrieval process to be more reliable and robust, not to mention more transparent. I know exactly what I am sending. Unlike the popular browsers, these programs are not fetching and running code from the internet automatically. I feel a greater sense of control.