7 ms·

> not a single website will work with xhr disabled That's total nonsense. I browse with uMatrix blocking everything by default and I rarely need to enable xhr

> not a single website will work with xhr disabled That's total nonsense. I browse with uMatrix blocking everything by default and I rarely need to enable xhr to make a site work. Most of the time it's only used to bloat a site, not deliver the actual content. The same is mostly true of javascript as well. This is particularly true on newspaper websites. A great many news sites are reasonable with everything disabled but utter cancer by (typical) default.

cookiengineer 6y ago
> That's total nonsense. Well, maybe we have a different pool of websites we visit. But usually, in my case, pretty much all websites built with vue.js, react, angular and others usually don't have server side rendering implemented correctly. Just as an example, what I visited just yesterday: https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/CVE-2020-1472 https://portal.msrc.microsoft.com/en-US/security-guidance/ad... Doesn't contain content, it's just a blank page without the XHR request. And all webapps I've seen so far basically just scaffold all polyfills and stuff, without any kind of content being delivered (or serialized) inside the HTML. Additionally, all newspaper websites that I've seen in my country blank out everything with white-on-white if you don't allow JS with XHR. Either that or the article teaser is faded out with an overlayed blur image. Well, that is at least when you don't set the user-agent to Googlebot :)
- liability 6y ago
  Yeah man I'm talking about really obscure websites like The New York Times which works great without any javascript enabled. https://0x0.st/ilMO.png https://0x0.st/ilMO.png
  - Cyphase 6y ago
    It looks a lot better if you allow CSS and images from *.nyt.com. A bunch of images still won't load, but the layout and some images are there.
- nullc 6y ago
  One of the great things about umatrix is the above mentioned multidimensionality. Your example displays content for me just fine in my default config, because the XHR requests are to the origin. Yet it blocks useless requests for two dozen different resources on other domains.
- inetknght 6y ago
  > Just as an example, what I visited just yesterday: https://portal.msrc.microsoft.com/en-US/security-guidance/ad https://portal.msrc.microsoft.com/en-US/security-guidance/ad... Well there's your problem. Microsoft counts as one of the sites that uses web tech to invade your privacy. Many parts of microsoft-dot-com don't work without javascript.
- m463 6y ago
  > white-on-white if you don't allow JS with XHR umatrix -> reader mode -> read your article
  - cookiengineer 6y ago
    Nope, because then I can only read the teaser, not the whole article.
- matheusmoreira 6y ago
  > Well, that is at least when you don't set the user-agent to Googlebot :) Isn't serving different pages to users and search engines against Google's policies? They call it cloaking. https://support.google.com/webmasters/answer/66355 https://support.google.com/webmasters/answer/66355
  - GarethX 6y ago
    Yes, but there are exceptions, like for rendering non-JS versions of pages for example. That’s why Google themselves created Rendertron. https://developers.google.com/search/docs/guides/dynamic-rendering https://developers.google.com/search/docs/guides/dynamic-ren...
    - bscphil 6y ago
      Hmm, this would suggest that any website that dynamically renders content should give me a server-side rendered version if I just switch my user agent to Googlebot. I may start doing that for select sites.
- 1vuio0pswjnm7 6y ago
  "Just as an example, what I visited yesterday: ..." The content is served from a different URL. The simplest solution is to use that URL, not the "empty container" one. For example, to retrieve the content and extract just the FAQ part: curl https://portal.msrc.microsoft.com/api/security-guidance/en-us/CVE/CVE-2020-1472|grep -o "frequentlyAskedQuestions\":.*</p>"|sed 's/\\n//g;s/frequentlyAskedQuestions\":\"/FAQ/' > 1.htm firefox ./1.htm
  - saagarjha 6y ago
    Why don't you just write an API that bounces requests through your webserver, where you render the page using a browser that runs the JavaScript and XHR requests, OCR a screenshot, and then send it through? That would help you win this argument, right? This conversation is about browsing the web, not the strawman you've constructed. We're talking about websites that you go to and they don't render if you turn off JavaScript, of which there are many. Dragging it into the corner where you're using cURL+grep on an plain-text endpoint which happened to exist for the example provided is not a valid response.
    - deleted 6y ago
      [deleted]
    - 1vuio0pswjnm7 6y ago
      Your proposed solution would not be appropriate for me since I use a text-only browser and prefer to use the web as either a text-only information retrieval source or a media download source. I do not use a "modern" graphical browser except for commercial, interactive, transactional uses, which comprise a very, very small fraction of my personal web use. It is probably a mischaracterisation to suggest the "plain-text endpoint" existed by chance. Many, many websites use the same or similar frameworks and "plain-text endpoints" have become commonplace. Regardless of the trends in web development, the solutions I use for text retrieval work reliably across almost any website, otherwise I would not use them.
- smichel17 6y ago
  I use uMatrix with very restrictive default settings. These[0]: https://smichel.me/hn/umatrix-global.png https://smichel.me/hn/umatrix-global.png When a page doesn't work, and I care enough to un-break it, I will first try enabling just js. If that doesn't work, I'll add XHRs, too. So, I can say with confidence that there are a good deal of sites that fall into both camps. I cannot say the ratio with confidence, but there are enough sites that work enough with just js that I choose not to enable XHRs. Here are a few examples: - Kotlin documentation gets collapsing menus and nicer code formatting: https://kotlinlang.org/docs/reference/visibility-modifiers.html https://kotlinlang.org/docs/reference/visibility-modifiers.h... - This person's blog gets inline/popup footnotes and, er, a functional "hard mode" (top right): https://mango.pdf.zone/finding-former-australian-prime-minister-tony-abbotts-passport-number-on-instagram https://mango.pdf.zone/finding-former-australian-prime-minis... - https://represent.us/ https://represent.us/ renders about halfway without js. The final example is the most relevant to this conversation, I think. Especially, there is a phenomenon, common enough that I've noticed it as a pattern of sites that display images with js (and sometimes content), and don't require XHRs to do so. ---------- Looking through my history to find these examples, two things become clear (about the sites I visit; I don't claim my browsing is representative): 1. Regardless of whether they also need XHRs, sites that require JS fall overwhelmingly into one of two categories: either I use them regularly and they're already on my saved whitelist (eg, YouTube), or I decide that I didn't care enough about them to bother enabling javascript 2. Yes, you're right; most sites that require js, also require XHRs. However, if we limit ourselves to the subset of sites above that I haven't whitelisted and don't want to walk away from — ie, the ones where I'm actually fiddling with uMatrix — it's somewhere around 50%. In conclusion, I think the functionality is useful and should not be axed, but it's probably a also good idea to have a simplified mode like you suggest: combining js, xhr, and other into one column. Maybe also removes the "cookies" column — most browsers have built-in preferences for those; 3rd party cookies almost never have to be unblocked; and I don't know (m)any people besides me who block 1st party cookies by default. [0]: Here's what that looks like when actually applied to the page. The keen observer will note that scripts are only temporarily enabled (I do this only when I want to use the collapse functionality). This (enable it when I want it) is a common browsing pattern of mine on sites that do progressive enhancement. https://smichel.me/hn/umatrix-hn.png https://smichel.me/hn/umatrix-hn.png
- GoblinSlayer 6y ago
  Oh, microsoft.com is absolute cancer, but it's one of a few exceptions. I see no shortage of news websites, and most of them are static.

cookiengineer 6y ago

> That's total nonsense. Well, maybe we have a different pool of websites we visit. But usually, in my case, pretty much all websites built with vue.js, react, angular and others usually don't have server side rendering implemented correctly. Just as an example, what I visited just yesterday: https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/CVE-2020-1472 https://portal.msrc.microsoft.com/en-US/security-guidance/ad... Doesn't contain content, it's just a blank page without the XHR request. And all webapps I've seen so far basically just scaffold all polyfills and stuff, without any kind of content being delivered (or serialized) inside the HTML. Additionally, all newspaper websites that I've seen in my country blank out everything with white-on-white if you don't allow JS with XHR. Either that or the article teaser is faded out with an overlayed blur image. Well, that is at least when you don't set the user-agent to Googlebot :)

liability 6y ago
Yeah man I'm talking about really obscure websites like The New York Times which works great without any javascript enabled. https://0x0.st/ilMO.png https://0x0.st/ilMO.png
- Cyphase 6y ago
  It looks a lot better if you allow CSS and images from *.nyt.com. A bunch of images still won't load, but the layout and some images are there.
nullc 6y ago
One of the great things about umatrix is the above mentioned multidimensionality. Your example displays content for me just fine in my default config, because the XHR requests are to the origin. Yet it blocks useless requests for two dozen different resources on other domains.
inetknght 6y ago
> Just as an example, what I visited just yesterday: https://portal.msrc.microsoft.com/en-US/security-guidance/ad https://portal.msrc.microsoft.com/en-US/security-guidance/ad... Well there's your problem. Microsoft counts as one of the sites that uses web tech to invade your privacy. Many parts of microsoft-dot-com don't work without javascript.
m463 6y ago
> white-on-white if you don't allow JS with XHR umatrix -> reader mode -> read your article
- cookiengineer 6y ago
  Nope, because then I can only read the teaser, not the whole article.
matheusmoreira 6y ago
> Well, that is at least when you don't set the user-agent to Googlebot :) Isn't serving different pages to users and search engines against Google's policies? They call it cloaking. https://support.google.com/webmasters/answer/66355 https://support.google.com/webmasters/answer/66355
- GarethX 6y ago
  Yes, but there are exceptions, like for rendering non-JS versions of pages for example. That’s why Google themselves created Rendertron. https://developers.google.com/search/docs/guides/dynamic-rendering https://developers.google.com/search/docs/guides/dynamic-ren...
  - bscphil 6y ago
    Hmm, this would suggest that any website that dynamically renders content should give me a server-side rendered version if I just switch my user agent to Googlebot. I may start doing that for select sites.
1vuio0pswjnm7 6y ago
"Just as an example, what I visited yesterday: ..." The content is served from a different URL. The simplest solution is to use that URL, not the "empty container" one. For example, to retrieve the content and extract just the FAQ part: curl https://portal.msrc.microsoft.com/api/security-guidance/en-us/CVE/CVE-2020-1472|grep -o "frequentlyAskedQuestions\":.*</p>"|sed 's/\\n//g;s/frequentlyAskedQuestions\":\"/FAQ/' > 1.htm firefox ./1.htm
- saagarjha 6y ago
  Why don't you just write an API that bounces requests through your webserver, where you render the page using a browser that runs the JavaScript and XHR requests, OCR a screenshot, and then send it through? That would help you win this argument, right? This conversation is about browsing the web, not the strawman you've constructed. We're talking about websites that you go to and they don't render if you turn off JavaScript, of which there are many. Dragging it into the corner where you're using cURL+grep on an plain-text endpoint which happened to exist for the example provided is not a valid response.
  - deleted 6y ago
    [deleted]
  - 1vuio0pswjnm7 6y ago
    Your proposed solution would not be appropriate for me since I use a text-only browser and prefer to use the web as either a text-only information retrieval source or a media download source. I do not use a "modern" graphical browser except for commercial, interactive, transactional uses, which comprise a very, very small fraction of my personal web use. It is probably a mischaracterisation to suggest the "plain-text endpoint" existed by chance. Many, many websites use the same or similar frameworks and "plain-text endpoints" have become commonplace. Regardless of the trends in web development, the solutions I use for text retrieval work reliably across almost any website, otherwise I would not use them.
smichel17 6y ago
I use uMatrix with very restrictive default settings. These[0]: https://smichel.me/hn/umatrix-global.png https://smichel.me/hn/umatrix-global.png When a page doesn't work, and I care enough to un-break it, I will first try enabling just js. If that doesn't work, I'll add XHRs, too. So, I can say with confidence that there are a good deal of sites that fall into both camps. I cannot say the ratio with confidence, but there are enough sites that work enough with just js that I choose not to enable XHRs. Here are a few examples: - Kotlin documentation gets collapsing menus and nicer code formatting: https://kotlinlang.org/docs/reference/visibility-modifiers.html https://kotlinlang.org/docs/reference/visibility-modifiers.h... - This person's blog gets inline/popup footnotes and, er, a functional "hard mode" (top right): https://mango.pdf.zone/finding-former-australian-prime-minister-tony-abbotts-passport-number-on-instagram https://mango.pdf.zone/finding-former-australian-prime-minis... - https://represent.us/ https://represent.us/ renders about halfway without js. The final example is the most relevant to this conversation, I think. Especially, there is a phenomenon, common enough that I've noticed it as a pattern of sites that display images with js (and sometimes content), and don't require XHRs to do so. ---------- Looking through my history to find these examples, two things become clear (about the sites I visit; I don't claim my browsing is representative): 1. Regardless of whether they also need XHRs, sites that require JS fall overwhelmingly into one of two categories: either I use them regularly and they're already on my saved whitelist (eg, YouTube), or I decide that I didn't care enough about them to bother enabling javascript 2. Yes, you're right; most sites that require js, also require XHRs. However, if we limit ourselves to the subset of sites above that I haven't whitelisted and don't want to walk away from — ie, the ones where I'm actually fiddling with uMatrix — it's somewhere around 50%. In conclusion, I think the functionality is useful and should not be axed, but it's probably a also good idea to have a simplified mode like you suggest: combining js, xhr, and other into one column. Maybe also removes the "cookies" column — most browsers have built-in preferences for those; 3rd party cookies almost never have to be unblocked; and I don't know (m)any people besides me who block 1st party cookies by default. [0]: Here's what that looks like when actually applied to the page. The keen observer will note that scripts are only temporarily enabled (I do this only when I want to use the collapse functionality). This (enable it when I want it) is a common browsing pattern of mine on sites that do progressive enhancement. https://smichel.me/hn/umatrix-hn.png https://smichel.me/hn/umatrix-hn.png
GoblinSlayer 6y ago
Oh, microsoft.com is absolute cancer, but it's one of a few exceptions. I see no shortage of news websites, and most of them are static.