3 ms·

Yes, you can use the certificate-transparency go code to pull down from the trillian API https://github.com/google/certificate-transparency-go/blob/master/clien

Yes, you can use the certificate-transparency go code to pull down from the trillian API https://github.com/google/certificate-transparency-go/blob/master/client/getentries.go https://github.com/google/certificate-transparency-go/blob/m... You would need to know the index, or you could just iterate over a range

westurner 5y ago
Do you think that CT log data replication would be more secure and efficient if the CT logs were stored by a zero trust distributed application like a blockchain, instead of Merkle signatures in a database owned by one party (that's now discontinuing free indexing, at least)? From "Oak, a Free and Open Certificate Transparency Log" (LetsEncrypt 2019) https://news.ycombinator.com/item?id=19920002 https://news.ycombinator.com/item?id=19920002 : > Trillian is a centralized Merkle tree: it doesn't support native replication [...] According to the trillian README, trillian depends upon MySQL/MariaDB and thus internal/private replication is as good as the SQL replication model (which doesn't have a distributed consensus algorithm like e.g. paxos). And what about indexing and search queries at volume, again without replication? From "A future for SQL on the web" https://news.ycombinator.com/item?id=28158491 https://news.ycombinator.com/item?id=28158491 : > https://thegraph.com/docs/indexing https://thegraph.com/docs/indexing >> Indexers are node operators in The Graph Network that stake Graph Tokens (GRT) in order to provide indexing and query processing services. Indexers earn query fees and indexing rewards for their services. They also earn from a Rebate Pool that is shared with all network contributors proportional to their work, following the Cobbs-Douglas Rebate Function. >> GRT that is staked in the protocol is subject to a thawing period and can be slashed if Indexers are malicious and serve incorrect data to applications or if they index incorrectly. Indexers can also be delegated stake from Delegators, to contribute to the network. >> Indexers select subgraphs to index based on the subgraph’s curation signal, where Curators stake GRT in order to indicate which subgraphs are high-quality and should be prioritized. Consumers (eg. applications) can also set parameters for which Indexers process queries for their subgraphs and set preferences for query fee pricing.
- westurner 5y ago
  FWIW, here's how OSV affords search queries: https://github.com/google/osv#data-dumps https://github.com/google/osv#data-dumps > For convenience, these sources are aggregated and continuously exported to a GCS bucket maintained by OSV: gs://osv-vulnerabilities > This bucket contains individual entries of the format gs://osv-vulnerabilities/<ECOSYSTEM>/<ID>.json as well as a zip containing all vulnerabilities for each ecosystem at gs://osv-vulnerabilities/<ECOSYSTEM>/all.zip > E.g. for PyPI vulnerabilities: # Or download over HTTP via https://osv-vulnerabilities.storage.googleapis.com/PyPI/all.zip gsutil cp gs://osv-vulnerabilities/PyPI/all.zip Hopefully, with an incentivized Blockchain Indexing service and/or e.g. GCS buckets that you just always `cp` and then load locally and then query locally, we can find a solution for queries of the growing CT Certificate Transparency logs.

westurner 5y ago

Do you think that CT log data replication would be more secure and efficient if the CT logs were stored by a zero trust distributed application like a blockchain, instead of Merkle signatures in a database owned by one party (that's now discontinuing free indexing, at least)? From "Oak, a Free and Open Certificate Transparency Log" (LetsEncrypt 2019) https://news.ycombinator.com/item?id=19920002 https://news.ycombinator.com/item?id=19920002 : > Trillian is a centralized Merkle tree: it doesn't support native replication [...] According to the trillian README, trillian depends upon MySQL/MariaDB and thus internal/private replication is as good as the SQL replication model (which doesn't have a distributed consensus algorithm like e.g. paxos). And what about indexing and search queries at volume, again without replication? From "A future for SQL on the web" https://news.ycombinator.com/item?id=28158491 https://news.ycombinator.com/item?id=28158491 : > https://thegraph.com/docs/indexing https://thegraph.com/docs/indexing >> Indexers are node operators in The Graph Network that stake Graph Tokens (GRT) in order to provide indexing and query processing services. Indexers earn query fees and indexing rewards for their services. They also earn from a Rebate Pool that is shared with all network contributors proportional to their work, following the Cobbs-Douglas Rebate Function. >> GRT that is staked in the protocol is subject to a thawing period and can be slashed if Indexers are malicious and serve incorrect data to applications or if they index incorrectly. Indexers can also be delegated stake from Delegators, to contribute to the network. >> Indexers select subgraphs to index based on the subgraph’s curation signal, where Curators stake GRT in order to indicate which subgraphs are high-quality and should be prioritized. Consumers (eg. applications) can also set parameters for which Indexers process queries for their subgraphs and set preferences for query fee pricing.

westurner 5y ago
FWIW, here's how OSV affords search queries: https://github.com/google/osv#data-dumps https://github.com/google/osv#data-dumps > For convenience, these sources are aggregated and continuously exported to a GCS bucket maintained by OSV: gs://osv-vulnerabilities > This bucket contains individual entries of the format gs://osv-vulnerabilities/<ECOSYSTEM>/<ID>.json as well as a zip containing all vulnerabilities for each ecosystem at gs://osv-vulnerabilities/<ECOSYSTEM>/all.zip > E.g. for PyPI vulnerabilities: # Or download over HTTP via https://osv-vulnerabilities.storage.googleapis.com/PyPI/all.zip gsutil cp gs://osv-vulnerabilities/PyPI/all.zip Hopefully, with an incentivized Blockchain Indexing service and/or e.g. GCS buckets that you just always `cp` and then load locally and then query locally, we can find a solution for queries of the growing CT Certificate Transparency logs.