3 ms·

Amusingly, the Solid website's SSL cert isn't compatible with centos 7, either curl or Python. Which is kinda the state of web security in a nutshell, sometimes

Amusingly, the Solid website's SSL cert isn't compatible with centos 7, either curl or Python. Which is kinda the state of web security in a nutshell, sometimes. I wanted to run my web metadata extractor on it, but there isn't any metadata there anyway. Which is kind of fun for a project about web metadata.

extra88 8y ago
I’m on mobile so I can’t fully inspect the cert but since it’s hosted at MIT, my guess is the Comodo-backed cert is though Internet2. The site maintainer probably neglected to include the intermediate cert which is fine for many browsers because they cache those but is a problem for some.
- greglindahl 8y ago
  Yeah, I think Qualys is agreeing with you, they say the chain is incomplete: https://www.ssllabs.com/ssltest/analyze.html?d=solid.mit.edu https://www.ssllabs.com/ssltest/analyze.html?d=solid.mit.edu
lsh 8y ago
is your extractor open source? (if so, could I get a link?)
- greglindahl 8y ago
  This: https://github.com/scrapinghub/extruct https://github.com/scrapinghub/extruct does ld+json, microformats, microdata, rdf3a, and opengraph mostly by calling other packages.
  - lsh 8y ago
    thank you so much for the link. I feel if we could give users/devs/whoever an appreciation for the structured data (or lack thereof) in a page and it's benefits we might see a general improvement in usability and interoperability. One can dream.
    - greglindahl 8y ago
      Yeah, one thing I would like to create in my web-scale crawling activities is some structured data discovery tools. Google surfaces a few things as part of their search results, but not in bulk. This project http://webdatacommons.org/ http://webdatacommons.org/ extracts from Common Crawl, but it's just raw data and you're on your own for visualization and discovery tools!
      - lsh 8y ago
        Thanks again - it was very easy to install and use. I think something like this: https://www.ssllabs.com/ssltest/ https://www.ssllabs.com/ssltest/ but for embedded data would be interesting.

extra88 8y ago

I’m on mobile so I can’t fully inspect the cert but since it’s hosted at MIT, my guess is the Comodo-backed cert is though Internet2. The site maintainer probably neglected to include the intermediate cert which is fine for many browsers because they cache those but is a problem for some.

greglindahl 8y ago
Yeah, I think Qualys is agreeing with you, they say the chain is incomplete: https://www.ssllabs.com/ssltest/analyze.html?d=solid.mit.edu https://www.ssllabs.com/ssltest/analyze.html?d=solid.mit.edu

lsh 8y ago

is your extractor open source? (if so, could I get a link?)

greglindahl 8y ago
This: https://github.com/scrapinghub/extruct https://github.com/scrapinghub/extruct does ld+json, microformats, microdata, rdf3a, and opengraph mostly by calling other packages.
- lsh 8y ago
  thank you so much for the link. I feel if we could give users/devs/whoever an appreciation for the structured data (or lack thereof) in a page and it's benefits we might see a general improvement in usability and interoperability. One can dream.
  - greglindahl 8y ago
    Yeah, one thing I would like to create in my web-scale crawling activities is some structured data discovery tools. Google surfaces a few things as part of their search results, but not in bulk. This project http://webdatacommons.org/ http://webdatacommons.org/ extracts from Common Crawl, but it's just raw data and you're on your own for visualization and discovery tools!
    - lsh 8y ago
      Thanks again - it was very easy to install and use. I think something like this: https://www.ssllabs.com/ssltest/ https://www.ssllabs.com/ssltest/ but for embedded data would be interesting.