6 ms·

> How long was it visible for? And no, "briefly" doesn't cut it. I don't know how long exactly, but in theory you can subscribe to a stream of ALL events happe

> How long was it visible for? And no, "briefly" doesn't cut it. I don't know how long exactly, but in theory you can subscribe to a stream of ALL events happening at GitHub by fetching from this endpoint: https://api.github.com/events https://api.github.com/events With these events you know what new repositories are created and what changes are pushed, so you can fetch every new change. Once something gets pushed to a public repository, it's very likely that some spider will have fetched it within a few minutes.

Maxious 4y ago
Especially something easily identifiable as a SSH private key, you get emails from a variety of security vendors to the address associated with the commit offering their services
- pbhjpbhj 4y ago
  I thought github themselves did checks for credentials? Isn't that party of their offered services?
  - acdha 4y ago
    I’d bet that’s why it said “briefly” — someone got the alert, realized what it meant and called their boss.
KirillPanov 4y ago
> you can subscribe to a stream of ALL events happening at GitHub by fetching from this endpoint: https://api.github.com/events https://api.github.com/events Wow I am shocked that they allow "firehose" access not only for free, but without even an API key. Given enough disk and bandwidth, does this mean you could keep your own copy of all of github? I'd love to be able to grep the whole thing.
- jonas-w 4y ago
  https://www.gharchive.org/ https://www.gharchive.org/
  - bspammer 4y ago
    The leaked private key should be somewhere in this archive then no?
    - jonas-w 4y ago
      The events stream does not contain files or anything in your git repo. Only stuff like commit messages, issues, comments, etc.
      - 112 4y ago
        i think it does contain code
        squeaky-clean 4y ago
        You can view it yourself, it doesn't contain the code directly, but it does contain a link to the code blob.
  - KirillPanov 4y ago
    Wow thanks. I looked all over that page but couldn't get a rough idea of the dataset size for the event stream (not the commits). Are we talking 1TB/year, 10TB/year, 100TB/year?
    - jonas-w 4y ago
      Your comment made me curious. I downloaded the complete last month (February). 28 days, every day has a gzipped json file which is on average 86MB. Summarized it is 53GB of gzipped jsons for 28 days. Roughly 700GB gzipped per year. Didn't have the chance to ungzip them all.

Maxious 4y ago

Especially something easily identifiable as a SSH private key, you get emails from a variety of security vendors to the address associated with the commit offering their services

pbhjpbhj 4y ago
I thought github themselves did checks for credentials? Isn't that party of their offered services?
- acdha 4y ago
  I’d bet that’s why it said “briefly” — someone got the alert, realized what it meant and called their boss.

KirillPanov 4y ago

> you can subscribe to a stream of ALL events happening at GitHub by fetching from this endpoint: https://api.github.com/events https://api.github.com/events Wow I am shocked that they allow "firehose" access not only for free, but without even an API key. Given enough disk and bandwidth, does this mean you could keep your own copy of all of github? I'd love to be able to grep the whole thing.

jonas-w 4y ago
https://www.gharchive.org/ https://www.gharchive.org/
- bspammer 4y ago
  The leaked private key should be somewhere in this archive then no?
  - jonas-w 4y ago
    The events stream does not contain files or anything in your git repo. Only stuff like commit messages, issues, comments, etc.
    - 112 4y ago
      i think it does contain code
      - squeaky-clean 4y ago
        You can view it yourself, it doesn't contain the code directly, but it does contain a link to the code blob.
- KirillPanov 4y ago
  Wow thanks. I looked all over that page but couldn't get a rough idea of the dataset size for the event stream (not the commits). Are we talking 1TB/year, 10TB/year, 100TB/year?
  - jonas-w 4y ago
    Your comment made me curious. I downloaded the complete last month (February). 28 days, every day has a gzipped json file which is on average 86MB. Summarized it is 53GB of gzipped jsons for 28 days. Roughly 700GB gzipped per year. Didn't have the chance to ungzip them all.