10 ms·
Google's Certificate Transparency Search page to be discontinued May 15th, 2022
- efitz 5y agoThis was not very satisfying. Why?
- sp332 5y agoNot much more data in the Google Groups archive. https://groups.google.com/g/certificate-transparency/ https://groups.google.com/g/certificate-transparency/
- progbits 5y agoBefore the inevitable flood of the super insightful and valuable comments like "hurr durr another google product killed", let me ask: Is there a good alternative frontend for these queries? As far I understand the underlying data is still public and available from many sources, but this was the only easy to use search I know of. Cloudflare sells some services to notify you on changes to your domains, but what about just checking on it manually, or exploring history of other domains?
- RL_Quine 5y agohttps://crt.sh/ https://crt.sh/
- politelemon 5y agoI didn't even know Google had a CT search. The search results are better presented/more accessible on crt.sh from my one minute of testing.
- toomuchtodo 5y agoDoes Let's Encrypt offer a similar functionality for CT queries? Could they if donations were directed to building said functionality?
- chockchocschoir 5y agoCertificates generated via Lets Encrypt already end up in the Certificate Transparency log. You can check this yourself via https://crt.sh https://crt.sh
- toomuchtodo 5y agoI am aware of that, but as a non profit providing certificate services, it aligns with their mission (imho) to also provide CT log storage and search functionality imho (not to denigrate crt.sh, which is good tooling, but yet another for profit org [Sectigo.com, formerly Comodo CA] benevolently providing the service for now). Personally, I think it's good to move away from Google, Facebook, and similar providing these core internet services. Throw some money at Let's Encrypt, the Internet Archive, and whomever is going to run Elasticsearch for this corpus, call it a day. The primitives are cheap to do so (compute and storage).
- est31 5y agoFTR, any certificate that wants to be recognized by Chrome ends up there.
- tptacek 5y agoThere's a zillion of them, right? crt.sh is maybe the best known, but if you Google you'll find a bunch (for instance Facebook runs one).
- progbits 5y agocrt.sh came up but I was getting 5xx errors. Looks like it might be overloaded now, or I picked "expensive" queries. But thanks for the recommendation, I'll try it some time later. Edit: Regarding the facebook one: "Log into Facebook to use this tool." Yeah no thanks.
- advisedwang 5y agoSome altarnatives: * https://crt.sh/ https://crt.sh/ * https://developers.facebook.com/tools/ct https://developers.facebook.com/tools/ct * https://ui.ctsearch.entrust.com/ui/ctsearchui https://ui.ctsearch.entrust.com/ui/ctsearchui
- freedomben 5y agohttps://ui.ctsearch.entrust.com/ui/ctsearchui https://ui.ctsearch.entrust.com/ui/ctsearchui is the only one I could get to work, but I uMatrix things so it could be my own fault.
- RL_Quine 5y ago> Log into Facebook to use this tool. Not sure I'd even list that as a reasonable alternative.
- cpach 5y agoFeel free to not use it.
- technion 5y agoI'd just like to chime in with my own alerting service: https://ctadvisor.lolware.net/ https://ctadvisor.lolware.net/ https://github.com/technion/ct_advisor https://github.com/technion/ct_advisor Building this was exciting, I've been proud to run it and and it predated services from Facebook or Cloudflare. I'm not sure what the future should hold however, it's hard to bring it up anywhere without being told people should just use Facebook.
- deleted 5y ago[deleted]
- cubesnooper 5y agoI know there are CT search services like crt.sh, but is it practical to download the raw data and search it locally? If the logs are append‐only, it feels like a perfect usecase for rsync.
- middleclick 5y agohttps://github.com/SSLMate/certspotter/ https://github.com/SSLMate/certspotter/ possibly?
- RL_Quine 5y agoCertspotter is basically just crawling for matches, it doesn't retain them.
- RL_Quine 5y agoIt's impractical due to the size of the data. You're talking hundreds of millions, or billions of certificates.
- hannob 5y agoBit more than 6 billion. And yes, it's quite challenging. It would be a nice service to the community if someone would host downloadable dumps of CT logs.
- bushbaba 5y ago100 billion * 2048 bytes per cert is just 205 terabytes. That’s not that impractical. Could easily be stored on gcs and s3 with requestor pays for the api and any associated egress charges. 205 terabytes at 2 cent per GB/month is just 50k/year.
- exikyut 5y agoAnd FTP is a perfectly viable and straightforward alternative to Dropbox. You just need to...
- 5y ago
- 0xbadcafebee 5y agoHow many people here do any sort of regular review of CT logs, and do you really think an attacker couldn't get away with whatever their attack was by the time you found out?
- freedomben 5y agoI did for a long time but I stopped. Real time is still super useful though. I have Cloudflare and Let's Encrypt email me now. Mostly it's been helpful for tracking down old boxes/services that are still running that have been lost/forgotten.
- tempnow987 5y agoReview or get emailed when a cert is issued with your domain in it somewhere. Secure-corpdomain.com etc?
- tomkaos 5y agoIt would be nice to learn about great google tool by a another way than the news they are discontinued.
- hannob 5y agoI would claim that I'm quite familiar with the TLS and certificate ecosystem, and up until this news item I didn't know Google runs a CT search engine. Everyone uses either crt.sh or to a lesser extend Censys. I think this space is covered well and there's just no need for a Google-operated CT search engine nobody knows about.
- bikingbismuth 5y agoI am in the same boat. I built a CT Log scraping system back in 2016 and once storage got too expensive I migrated the solution to the Censys API. Our use case is finding phishing sites targeting our customers, CT Logs have been amazing for detecting campaigns early.
- tialaramex 5y agoI think I would claim to be similarly familiar :) but I did know Google operated this search and I've even used it in the past when Rob's site was down. As I think I expressed in a post last year it's important that we could migrate off crt.sh but it isn't a problem if we never do. One thing I don't like but can't attest to still being a problem since maintaining one is no longer my job, is some logs are (were) unreliable for a crawler, which is a problem if you wanted to spin up an alternative to crt.sh. Google checks things work for them as part of qualification, but that's at best a 90% solution
- fomine3 5y agoAlso not that crt.sh is operated by Sectigo (formerly Comodo). Previously they had not good reputation.
- freedomben 5y agoHow many people feel they get positive value from CT services in general? I have Cloudflare email me whenever my domain gets a cert, and that is a positive value for me. However, it is very clear to me that bots are watching these logs (in near real time) because obscure hostnames (sometimes UUIDs) that I set up for a quick test but need a real cert will start getting traffic almost immediately after the cert comes back from Let's Encrypt. As an attacker, this log is a fantastic tool for service/host discovery, but as a defender I kind of wish I could opt out. Note: a wildcard cert is a good way to avoid the problem, but I prefer to avoid wildcards if possible.
- decodebytes 5y agoA lot do, but without knowing it. Chrome uses an 'expect-CT' header, which means the sites TLS cert should chain to a root CA, with both stored in a CTL.
- tialaramex 5y agoI don't think that's what your parent post meant, but, Chrome actually mechanically requires CT logging in order to trust a site, the expect-CT header doesn't come into it. Safari has the same policy, and Mozilla intends to some day do likewise in Firefox but the politics of the exact rules are tricky for reasons we could get into if somebody cares. When your Chrome connects to a TLS server, the server has to provide a certificate, and Chrome examines that certificate before any HTTP traffic happens (and thus before you could send an HTTP header like Expect-CT anywhere) unless you've got Group Policy or similar rules saying otherwise: - The certificate must have proof it was logged in the form of SCTs. For most sites the SCTs are baked into the certificate when they got it, they're all that incomprehensible gibberish near the end of your certificate if you've read it. - The certificate must have been issued relatively recently (825 days or less previously) and it must expire within 825 days of issuance or if it was issued since some time in 2020 when Apple's policy change happened, 398 days. - The certificate says it is for TLS Servers (in most cases certificates say they're also for TLS Clients, and sometimes other things too, but Chrome checks it says specifically TLS Servers here) - The certificate has a Subject Alternative Name matching the DNS name of the server, or, if the URL we're resolving is for a numeric IP address, a SAN matches that IP Address. SANs are typed, and are not free human text, so a DNS SAN and an IP address SAN are distinguishable even if 10.20.30.40 was a valid DNS name, which it is not. Old-fashioned "Common" names are disregarded. - The certificate must be signed by a trusted CA (or by an intermediate which in turn was trusted, and so on recursively). I think I hit the big ticket items, there might be some others.
- deleted 5y ago[deleted]