9 ms·

Mozilla HTTP Observatory

185 points by chynkm 5y ago

sciurus 5y ago
For anyone curious, you can find the source code at https://github.com/mozilla/http-observatory https://github.com/mozilla/http-observatory
exciteabletom 5y ago
I was previously using https://ssllabs.com https://ssllabs.com, but this is much more comprehensive! It even includes ssllabs as a third party test!
- hannob 5y ago
  Isn't it just very different from SSLLabs? Like SSLLabs is testing for TLS configuration and vulnerabilities, this is testing for HTTP security headers. There's some overlap (HSTS), but for the most part these are just two different tools doing different things.
  - input_sh 5y ago
    Yup, this is more like securityheaders.com than ssllabs.
kiryin 5y ago
I'm interested in the SSH test, but I use a non-standard port for SSH. I've been lead to believe this is a common practice, is there an option hidden somewhere? From what I can tell it just automatically tries 22 and fails.
- bugmen0t 5y ago
  There is no options for non-standard ports, but you may perform a local scan using https://github.com/mozilla/observatory-cli https://github.com/mozilla/observatory-cli
  - mimsee 5y ago
    The link you posted is for performing HTTPS & CSP tests. OP mentioned SSH testing which can be run locally with https://github.com/mozilla/ssh_scan https://github.com/mozilla/ssh_scan
- iaml 5y ago
  I recently something like this on hn, might work for you: https://sshcheck.com/ https://sshcheck.com/
m_eiman 5y ago
Ok, seems a bit "modern web" focused. My score: -25 for not defending against Javascript attacks on my javascript free domain -40 for not forcing HTTPS -35 more for not protecting against non-existing javascript being manipulated
- KronisLV 5y ago
  > -40 for not forcing HTTPS For most of the websites out there (that might want to accept user input, or ensure that the page content isn't manipulated or hijacked), that indeed would be a good recommendation. What would prevent someone from adding a few <script> tags here and there with ads on your site, or code to spy on your users? ISPs have historically already abused this, here's just one example, though you can look up plenty more through your search engine of choice: https://stackoverflow.com/questions/30505416/eliminate-isp-injects-pages-with-iframe-script-for-ads https://stackoverflow.com/questions/30505416/eliminate-isp-i... Personally, i really dislike that the web has come to this.
  - m_eiman 5y ago
    > What would prevent someone from adding a few <script> tags here and there with ads on your site, or code to spy on your users? Nothing, probably. In a sane country and legal system doing things like that would be illegal. But on the other hand forcing HTTPS means that some users will never be able access it due to old browsers and/or hardware. More likely though is that I mess up the HTTPS certificates, either by mistake or inaction, and lock out everyone who doesn't dare click the correct sequence of "ignore warning" buttons. I've already managed to block access for normal users to several sites, several times, by running too old certbot versions, not integrating things properly and whatnot. It's a good thing I'll never use HSTS and HPKP, or I'll make permanent messes.
    - Aeolun 5y ago
      > It's a good thing I'll never use HSTS Always fun when you lock yourself out of your own website for several days.
    - 5y ago
KronisLV 5y ago
This is pretty nice! Added CSP headers and fixed the cookie attributes on my personal site thanks to it, had forgotten about those. The CSP analysis section (and maybe some others) could use a bit of improvement. For example, currently you get the following output: Clickjacking protection, using frame-ancestors With the following popover text: The use of CSP's <code>frame-ancestors</code> directive offers fine-grained control over who can frame your site. And yet, nowhere does it recommend you actionable steps. The page that you're probably looking for in that situation might as well be a clickable link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Co... Thankfully, the recommendation section does have some useful links apart from that! Not step by step guides, but still would lead you in the general direction of what you need.
ajnin 5y ago
Got an F because I didn't implement XSS protection on my static non-interactive non-JS website.
- IggleSniggle 5y ago
  To be fair, you don’t need JS in the exploited website to do XSS.
  - calcifer 5y ago
    But they also said non-interactive, so I'm assuming no forms of any kind either.
    - IggleSniggle 5y ago
      True. Although…you could also potentially use reflected URLs or cookies to pull off some kind of XSS attack.
    - schemescape 5y ago
      I just confirmed this on my own static site which has no forms or any input whatsoever (and no JavaScript, cookies, or external resources either). But I guess I wouldn’t use this tool on such a trivial site anyway.
  - agilob 5y ago
    I moved my website from wordpress with 20 plugins, comments, disqus and lazy loaded images to static pages generated from markdown using Hugo. It's literally plaintext no JS, no tracking, no cookies at all. I got downgrade from B to D for not having CSP, XSS and XFrames protections. I don't even have forms or JS on my blog anymore. All content is loaded from 1 domain.
- chrismorgan 5y ago
  Yeah, −60 for lacking Content-Security-Policy, X-Content-Type-Options, X-Frame-Options and X-XSS-Protection is grossly excessive. As is declaring XFO “mandatory for all new websites, and all existing websites are expected to add support for [it] as soon as possible” <https://infosec.mozilla.org/guidelines/web_security#x-frame-options https://infosec.mozilla.org/guidelines/web_security#x-frame-...>. XFO is entirely unnecessary for stateless and actionless websites, CSP of no value on static sites that load no third-party resources, and the rest of them of no value on static sites. You could say they protect against man-in-the-middle attacks or attacks on your static file server (nginx or equivalent), but any competent MITM will modify headers, and with attacks on the static file server you’re hosed anyway. I also think they should significantly downgrade at least XFO (they last touched that descriptive document I quoted over three years ago), because the browsers that want it are all now obsolete (IE11 being the only one that’s even vaguely still in use) and entirely unsupported by many systems. I get 40⁄100, D⁺, because of this stuff, and I have not the slightest intention of changing it because I’m stubborn and know that I don’t need them. Well, it’s better than the 0⁄100 it gave me at the start of 2019, or the 20⁄100 back in 2016. It needs some sort of profile system for the rankings, so that you can say “anything”, “static site”, “static site with no third-party resources”, that sort of thing, and in the lattermost case have it say “OK, then we’ll just suggest CSP, XCTO, XFO and XXP rather than screaming about them”.
weinzierl 5y ago
It is an excellent tool. For a bit more background I found [1] (from 2016) quite insightful. In addition here are a few notes that I collected using it, no criticism - just (hopefully) constructive feedback: - The SSH (H not L) Observatory part seems to be broken for a long time (months at least). Not exactly sure what it was supposed to do anyway and how useful it would have been. - I find the nomenclature for the Compatibility Level a bit unfortunate. As far as I understand, the highly secure and practically useful configurations recommended by Mozilla and elsewhere, all end up classified as Intermediate. The more desirable sounding Modern seems to be unachievable for any real world site. I'd love to see counterexamples if I'm wrong. - It seems not to be very actively maintained since its main (and original?) author April King left Mozilla. About a half a year ago I filed an issue where the Observatory scan hung forever for certain sites [2], but apparently no one ever looked at it. (Maybe it is not an issue with the Observatory, but I think I wrote a decent report and hoped for some feedback). [1] https://pokeinthe.io/2016/08/25/observatory-by-mozilla-a-new-tool/ https://pokeinthe.io/2016/08/25/observatory-by-mozilla-a-new... [2] https://github.com/mozilla/http-observatory-website/issues/244 https://github.com/mozilla/http-observatory-website/issues/2...
- ff317 5y ago
  > The more desirable sounding Modern seems to be unachievable for any real world site. I'd love to see counterexamples if I'm wrong. I'd agree that the description of "M" in https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_configurations https://wiki.mozilla.org/Security/Server_Side_TLS#Recommende... is unrealistic for a site with a large and diverse audience, so far. The primary issue is that it requires turning off TLSv1.2. Wikipedia is a good example of "about as Modern as you can get in the real world with a big global audience". It's a little stricter than "Intermediate", but doesn't quite meet the "Modern" description. The key items there are that Wikipedia still supports TLSv1.2 (but only with TLSv1.3-like ciphers) and it still supports dual cert compatibility (ECDSA+RSA). The RSA support is likely to be on chopping block Soon, as the only real use-case for RSA in this config is to support ancient installs of the last-known-good version of Chrome (49) on WinXP SP3, but Wikipedia will likely have to continue supporting TLSv1.2 for quite some time. In any case, though, Wikipedia still gets an "M" rating in the check, so either the description is wrong or the check is buggy: https://observatory.mozilla.org/analyze/en.wikipedia.org#tls https://observatory.mozilla.org/analyze/en.wikipedia.org#tls
tootie 5y ago
Is CSP still recommended? I thought it was considered overkill for little benefit
- doliveira 5y ago
  A lot of it seems specially targeted towards websites with tons of third-party scripts
  - bleuarff 5y ago
    Isn't that the majority of the web today?
    - doliveira 5y ago
      Indeed, but it's messed up that we had to invent all this new standard just to keep including hundreds of Analytics scripts
  - tootie 5y ago
    Yeah, I work on some public-facing sites that have analytics and programmatic ads and the like. Our list of script and frame allows would be pretty long. And since the analytics team own and operate out tag manager, they can inject third-party scripts at will without needing a release which makes maintaining CSP a whole job on its own.
- rnicholus 5y ago
  There is enormous benefit with a _strict_ CSP. It's unfortunately common for a CSP that whitelists CDNs, allows eval, etc, etc. These are arguably worse than not having a CSP at all due to the false sense of security. More details in this excellent writeup at https://storage.googleapis.com/pub-tools-public-publication-data/pdf/45542.pdf https://storage.googleapis.com/pub-tools-public-publication-....
marginalia_nu 5y ago
I have really mixed feelings about deprecating HTTP for HTTPS. There are a lot of websites that are never going to migrate, websites with quality content. There's also a lot of clients that are never going to support HTTPS. There's nothing wrong with them otherwise, the hardware is still good, but they can't be used anymore and it's not the owners choice to decide, but a few big tech companies pushing this change. Even if we do care about these nefarious men in the middle, the elephant in the room is that a large part of the encrypted traffic these days go through major CDNs, and for them to actually pages and and route requests and be anything more than a glorified NAT, they need to inspect what's being sent, and keep track of who is sending it. Even if they totally pinky swear they aren't doing anything nefarious with their position of being able to inspect and analyze a huge portion of the Internet's traffic, and even if we believe them, that can change. Remember SourceForge? Remember when they were the heroes of open source, the go-to host for source code and binaries? Remember when they were bought up and subsequently were caught with their fingers in the cookie jar bundling malware in said open source packages? All I'm saying is that there sure is a lot of eggs in that basket. Is a lot easier to lean on one or two CDN operators than it is to lean on every ISP in the country.
- thayne 5y ago
  > Is a lot easier to lean on one or two CDN operators than it is to lean on every ISP in the country. First of all, that's ignoring how easy it is to MiTM on a public wifi network. Secondly, if a CDN starts misbehaving, customers can switch to a different one. For many people in the US at least if an ISP is misbehaving, there may not be any other option to switch to.
  - marginalia_nu 5y ago
    It's a matter of scale. You can perform opportunistic small scale mitm attacks on wifi. Maybe that is a concern for interactive websites. But servers that only host static content, why do they need encryption? And how would you find out if a cdn was misbehaving, especially in the context of gag orders? And even if we did find out, it's out in the open that Facebook and Google is doing all this really invasive tracking, yet nobody seems to be leaving them in any hurry.
hidalgopl 5y ago
Had almost identical idea for startup about a year ago. I was thinking about it as a SaaS, but then I figured out there is not enough interest for such product. Idea was to run almost same set of checks as tab HTTP Observatory does using CLI I created: sailor. We decided to have it as CLI, for sake of simplicity of integrating it into CI & CD. After I decided we won't be trying to build a business around it, I removed SaaS dependency and open-sourced it. You can check it here: https://github.com/hidalgopl/sailor https://github.com/hidalgopl/sailor
offsky 5y ago
Here is a more comprehensive website testing tool. https://www.validbot.com https://www.validbot.com Full disclosure, this is my project.
- chrismorgan 5y ago
  Tried it on my site: https://www.validbot.com/report/b6c2b0aec340f6133de16148a495d5d2e https://www.validbot.com/report/b6c2b0aec340f6133de16148a495... Some of the icon tests are bogus. I deliberately don’t put any <link rel=icon> on my site, but have a favicon.ico containing (among other sizes) 16×16 and 32×32. Your tool complains about meta tags for 16×16 and 32×32 not being found in the HTML. Well, they’re not, but they’re not necessary, because I haven’t put anything in place that would disrupt the favicon.ico fallback. 192×192: … why? Won’t things gladly scale that 512×512 you want down? Manifest and other large icon sizes: this stuff isn’t relevant to all sites. And that’s a problem with these sorts of tools in general, they give scores tuned to a single usage profile which simply isn’t suitable in all cases. As with HTTP Observatory’s XSS stuff commented about elsewhere in this thread. What we need for tools like this is profiles that twiddle rankings. Things like “personal content site” which changes manifest and Apple/Safari/large icons to optional. As it stands, the weighting of this extra stuff is way off base—I get given an F for that section, when I honestly think it should get at least an A, when operating under my hypothetical “personal content site” profile. Test 48 is bogus, the <body> start tag is optional. Test 111, wanting initial-scale on the viewport meta tag, I’ve been casually searching for someone to confirm what it actually does, and if it’s still needed. Most indications suggest it was basically a workaround for an ancient iOS Safari rotation bug, but I’ve come across at least one person stating (without detail) that it still did something. Any chance you have Apple stuff and can investigate more as to whether it’s actually still at all useful? Test 33, DMARC record formatting, looks bogus.
  - offsky 5y ago
    No general purpose testing tool like this can be a one-size-fits-all sort of thing. In the future I plan on adding configuration options so you can disable tests that you don't care about. If you know what you are doing, but all means feel free to disregard any tests that you don't agree with. The suggestions that Validbot makes are meant to be general purpose "best practices" to help web developers make sure they are paying attention to everything they should be. Sounds like you are and have made some good decisions.
deleted 5y ago
[deleted]
facorreia 5y ago
Sites on GitHub Pages get a D+. https://observatory.mozilla.org/analyze/government.github.com https://observatory.mozilla.org/analyze/government.github.co...

185 points by chynkm 5y ago

For anyone curious, you can find the source code at https://github.com/mozilla/http-observatory https://github.com/mozilla/http-observatory

exciteabletom 5y ago

I was previously using https://ssllabs.com https://ssllabs.com, but this is much more comprehensive! It even includes ssllabs as a third party test!

hannob 5y ago
Isn't it just very different from SSLLabs? Like SSLLabs is testing for TLS configuration and vulnerabilities, this is testing for HTTP security headers. There's some overlap (HSTS), but for the most part these are just two different tools doing different things.
- input_sh 5y ago
  Yup, this is more like securityheaders.com than ssllabs.

kiryin 5y ago

I'm interested in the SSH test, but I use a non-standard port for SSH. I've been lead to believe this is a common practice, is there an option hidden somewhere? From what I can tell it just automatically tries 22 and fails.

bugmen0t 5y ago
There is no options for non-standard ports, but you may perform a local scan using https://github.com/mozilla/observatory-cli https://github.com/mozilla/observatory-cli
- mimsee 5y ago
  The link you posted is for performing HTTPS & CSP tests. OP mentioned SSH testing which can be run locally with https://github.com/mozilla/ssh_scan https://github.com/mozilla/ssh_scan
iaml 5y ago
I recently something like this on hn, might work for you: https://sshcheck.com/ https://sshcheck.com/

m_eiman 5y ago

Ok, seems a bit "modern web" focused. My score: -25 for not defending against Javascript attacks on my javascript free domain -40 for not forcing HTTPS -35 more for not protecting against non-existing javascript being manipulated

KronisLV 5y ago
> -40 for not forcing HTTPS For most of the websites out there (that might want to accept user input, or ensure that the page content isn't manipulated or hijacked), that indeed would be a good recommendation. What would prevent someone from adding a few <script> tags here and there with ads on your site, or code to spy on your users? ISPs have historically already abused this, here's just one example, though you can look up plenty more through your search engine of choice: https://stackoverflow.com/questions/30505416/eliminate-isp-injects-pages-with-iframe-script-for-ads https://stackoverflow.com/questions/30505416/eliminate-isp-i... Personally, i really dislike that the web has come to this.
- m_eiman 5y ago
  > What would prevent someone from adding a few <script> tags here and there with ads on your site, or code to spy on your users? Nothing, probably. In a sane country and legal system doing things like that would be illegal. But on the other hand forcing HTTPS means that some users will never be able access it due to old browsers and/or hardware. More likely though is that I mess up the HTTPS certificates, either by mistake or inaction, and lock out everyone who doesn't dare click the correct sequence of "ignore warning" buttons. I've already managed to block access for normal users to several sites, several times, by running too old certbot versions, not integrating things properly and whatnot. It's a good thing I'll never use HSTS and HPKP, or I'll make permanent messes.
  - Aeolun 5y ago
    > It's a good thing I'll never use HSTS Always fun when you lock yourself out of your own website for several days.
  - 5y ago

KronisLV 5y ago

This is pretty nice! Added CSP headers and fixed the cookie attributes on my personal site thanks to it, had forgotten about those. The CSP analysis section (and maybe some others) could use a bit of improvement. For example, currently you get the following output: Clickjacking protection, using frame-ancestors With the following popover text: The use of CSP's <code>frame-ancestors</code> directive offers fine-grained control over who can frame your site. And yet, nowhere does it recommend you actionable steps. The page that you're probably looking for in that situation might as well be a clickable link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Co... Thankfully, the recommendation section does have some useful links apart from that! Not step by step guides, but still would lead you in the general direction of what you need.

ajnin 5y ago

Got an F because I didn't implement XSS protection on my static non-interactive non-JS website.

IggleSniggle 5y ago
To be fair, you don’t need JS in the exploited website to do XSS.
- calcifer 5y ago
  But they also said non-interactive, so I'm assuming no forms of any kind either.
  - IggleSniggle 5y ago
    True. Although…you could also potentially use reflected URLs or cookies to pull off some kind of XSS attack.
  - schemescape 5y ago
    I just confirmed this on my own static site which has no forms or any input whatsoever (and no JavaScript, cookies, or external resources either). But I guess I wouldn’t use this tool on such a trivial site anyway.
- agilob 5y ago
  I moved my website from wordpress with 20 plugins, comments, disqus and lazy loaded images to static pages generated from markdown using Hugo. It's literally plaintext no JS, no tracking, no cookies at all. I got downgrade from B to D for not having CSP, XSS and XFrames protections. I don't even have forms or JS on my blog anymore. All content is loaded from 1 domain.
chrismorgan 5y ago
Yeah, −60 for lacking Content-Security-Policy, X-Content-Type-Options, X-Frame-Options and X-XSS-Protection is grossly excessive. As is declaring XFO “mandatory for all new websites, and all existing websites are expected to add support for [it] as soon as possible” <https://infosec.mozilla.org/guidelines/web_security#x-frame-options https://infosec.mozilla.org/guidelines/web_security#x-frame-...>. XFO is entirely unnecessary for stateless and actionless websites, CSP of no value on static sites that load no third-party resources, and the rest of them of no value on static sites. You could say they protect against man-in-the-middle attacks or attacks on your static file server (nginx or equivalent), but any competent MITM will modify headers, and with attacks on the static file server you’re hosed anyway. I also think they should significantly downgrade at least XFO (they last touched that descriptive document I quoted over three years ago), because the browsers that want it are all now obsolete (IE11 being the only one that’s even vaguely still in use) and entirely unsupported by many systems. I get 40⁄100, D⁺, because of this stuff, and I have not the slightest intention of changing it because I’m stubborn and know that I don’t need them. Well, it’s better than the 0⁄100 it gave me at the start of 2019, or the 20⁄100 back in 2016. It needs some sort of profile system for the rankings, so that you can say “anything”, “static site”, “static site with no third-party resources”, that sort of thing, and in the lattermost case have it say “OK, then we’ll just suggest CSP, XCTO, XFO and XXP rather than screaming about them”.

weinzierl 5y ago

It is an excellent tool. For a bit more background I found [1] (from 2016) quite insightful. In addition here are a few notes that I collected using it, no criticism - just (hopefully) constructive feedback: - The SSH (H not L) Observatory part seems to be broken for a long time (months at least). Not exactly sure what it was supposed to do anyway and how useful it would have been. - I find the nomenclature for the Compatibility Level a bit unfortunate. As far as I understand, the highly secure and practically useful configurations recommended by Mozilla and elsewhere, all end up classified as Intermediate. The more desirable sounding Modern seems to be unachievable for any real world site. I'd love to see counterexamples if I'm wrong. - It seems not to be very actively maintained since its main (and original?) author April King left Mozilla. About a half a year ago I filed an issue where the Observatory scan hung forever for certain sites [2], but apparently no one ever looked at it. (Maybe it is not an issue with the Observatory, but I think I wrote a decent report and hoped for some feedback). [1] https://pokeinthe.io/2016/08/25/observatory-by-mozilla-a-new-tool/ https://pokeinthe.io/2016/08/25/observatory-by-mozilla-a-new... [2] https://github.com/mozilla/http-observatory-website/issues/244 https://github.com/mozilla/http-observatory-website/issues/2...

ff317 5y ago
> The more desirable sounding Modern seems to be unachievable for any real world site. I'd love to see counterexamples if I'm wrong. I'd agree that the description of "M" in https://wiki.mozilla.org/Security/Server_Side_TLS#Recommended_configurations https://wiki.mozilla.org/Security/Server_Side_TLS#Recommende... is unrealistic for a site with a large and diverse audience, so far. The primary issue is that it requires turning off TLSv1.2. Wikipedia is a good example of "about as Modern as you can get in the real world with a big global audience". It's a little stricter than "Intermediate", but doesn't quite meet the "Modern" description. The key items there are that Wikipedia still supports TLSv1.2 (but only with TLSv1.3-like ciphers) and it still supports dual cert compatibility (ECDSA+RSA). The RSA support is likely to be on chopping block Soon, as the only real use-case for RSA in this config is to support ancient installs of the last-known-good version of Chrome (49) on WinXP SP3, but Wikipedia will likely have to continue supporting TLSv1.2 for quite some time. In any case, though, Wikipedia still gets an "M" rating in the check, so either the description is wrong or the check is buggy: https://observatory.mozilla.org/analyze/en.wikipedia.org#tls https://observatory.mozilla.org/analyze/en.wikipedia.org#tls

tootie 5y ago

Is CSP still recommended? I thought it was considered overkill for little benefit

doliveira 5y ago
A lot of it seems specially targeted towards websites with tons of third-party scripts
- bleuarff 5y ago
  Isn't that the majority of the web today?
  - doliveira 5y ago
    Indeed, but it's messed up that we had to invent all this new standard just to keep including hundreds of Analytics scripts
- tootie 5y ago
  Yeah, I work on some public-facing sites that have analytics and programmatic ads and the like. Our list of script and frame allows would be pretty long. And since the analytics team own and operate out tag manager, they can inject third-party scripts at will without needing a release which makes maintaining CSP a whole job on its own.
rnicholus 5y ago
There is enormous benefit with a _strict_ CSP. It's unfortunately common for a CSP that whitelists CDNs, allows eval, etc, etc. These are arguably worse than not having a CSP at all due to the false sense of security. More details in this excellent writeup at https://storage.googleapis.com/pub-tools-public-publication-data/pdf/45542.pdf https://storage.googleapis.com/pub-tools-public-publication-....

marginalia_nu 5y ago

I have really mixed feelings about deprecating HTTP for HTTPS. There are a lot of websites that are never going to migrate, websites with quality content. There's also a lot of clients that are never going to support HTTPS. There's nothing wrong with them otherwise, the hardware is still good, but they can't be used anymore and it's not the owners choice to decide, but a few big tech companies pushing this change. Even if we do care about these nefarious men in the middle, the elephant in the room is that a large part of the encrypted traffic these days go through major CDNs, and for them to actually pages and and route requests and be anything more than a glorified NAT, they need to inspect what's being sent, and keep track of who is sending it. Even if they totally pinky swear they aren't doing anything nefarious with their position of being able to inspect and analyze a huge portion of the Internet's traffic, and even if we believe them, that can change. Remember SourceForge? Remember when they were the heroes of open source, the go-to host for source code and binaries? Remember when they were bought up and subsequently were caught with their fingers in the cookie jar bundling malware in said open source packages? All I'm saying is that there sure is a lot of eggs in that basket. Is a lot easier to lean on one or two CDN operators than it is to lean on every ISP in the country.

thayne 5y ago
> Is a lot easier to lean on one or two CDN operators than it is to lean on every ISP in the country. First of all, that's ignoring how easy it is to MiTM on a public wifi network. Secondly, if a CDN starts misbehaving, customers can switch to a different one. For many people in the US at least if an ISP is misbehaving, there may not be any other option to switch to.
- marginalia_nu 5y ago
  It's a matter of scale. You can perform opportunistic small scale mitm attacks on wifi. Maybe that is a concern for interactive websites. But servers that only host static content, why do they need encryption? And how would you find out if a cdn was misbehaving, especially in the context of gag orders? And even if we did find out, it's out in the open that Facebook and Google is doing all this really invasive tracking, yet nobody seems to be leaving them in any hurry.

hidalgopl 5y ago

Had almost identical idea for startup about a year ago. I was thinking about it as a SaaS, but then I figured out there is not enough interest for such product. Idea was to run almost same set of checks as tab HTTP Observatory does using CLI I created: sailor. We decided to have it as CLI, for sake of simplicity of integrating it into CI & CD. After I decided we won't be trying to build a business around it, I removed SaaS dependency and open-sourced it. You can check it here: https://github.com/hidalgopl/sailor https://github.com/hidalgopl/sailor

offsky 5y ago

Here is a more comprehensive website testing tool. https://www.validbot.com https://www.validbot.com Full disclosure, this is my project.

chrismorgan 5y ago
Tried it on my site: https://www.validbot.com/report/b6c2b0aec340f6133de16148a495d5d2e https://www.validbot.com/report/b6c2b0aec340f6133de16148a495... Some of the icon tests are bogus. I deliberately don’t put any <link rel=icon> on my site, but have a favicon.ico containing (among other sizes) 16×16 and 32×32. Your tool complains about meta tags for 16×16 and 32×32 not being found in the HTML. Well, they’re not, but they’re not necessary, because I haven’t put anything in place that would disrupt the favicon.ico fallback. 192×192: … why? Won’t things gladly scale that 512×512 you want down? Manifest and other large icon sizes: this stuff isn’t relevant to all sites. And that’s a problem with these sorts of tools in general, they give scores tuned to a single usage profile which simply isn’t suitable in all cases. As with HTTP Observatory’s XSS stuff commented about elsewhere in this thread. What we need for tools like this is profiles that twiddle rankings. Things like “personal content site” which changes manifest and Apple/Safari/large icons to optional. As it stands, the weighting of this extra stuff is way off base—I get given an F for that section, when I honestly think it should get at least an A, when operating under my hypothetical “personal content site” profile. Test 48 is bogus, the <body> start tag is optional. Test 111, wanting initial-scale on the viewport meta tag, I’ve been casually searching for someone to confirm what it actually does, and if it’s still needed. Most indications suggest it was basically a workaround for an ancient iOS Safari rotation bug, but I’ve come across at least one person stating (without detail) that it still did something. Any chance you have Apple stuff and can investigate more as to whether it’s actually still at all useful? Test 33, DMARC record formatting, looks bogus.
- offsky 5y ago
  No general purpose testing tool like this can be a one-size-fits-all sort of thing. In the future I plan on adding configuration options so you can disable tests that you don't care about. If you know what you are doing, but all means feel free to disregard any tests that you don't agree with. The suggestions that Validbot makes are meant to be general purpose "best practices" to help web developers make sure they are paying attention to everything they should be. Sounds like you are and have made some good decisions.

deleted 5y ago

[deleted]

facorreia 5y ago

Sites on GitHub Pages get a D+. https://observatory.mozilla.org/analyze/government.github.com https://observatory.mozilla.org/analyze/government.github.co...