4 ms·

This highlights why it's so important that any secret that gets committed must be rotated. Simply removing it from the git history isn't enough, because it can

This highlights why it's so important that any secret that gets committed must be rotated. Simply removing it from the git history isn't enough, because it can still linger, it's just harder to find.

belinder 3y ago
While I agree that you should rotate accidentally exposed secrets, it should be noted that you can remove old history from git reflog by expiring it I see git reflog kinda like an OS recycle bin
- Kluggy 3y ago
  How do you do that once it's on Github's servers?
  - latchkey 3y ago
    [deleted]
    - javawizard 3y ago
      That link provides instructions on how to access commits from the reflog of a GitHub repository, not how to expire or otherwise delete them.
  - KTibow 3y ago
    Sounds like it happens automatically: https://stackoverflow.com/questions/56020314/how-frequently-do-github-run-git-gc-on-repositories https://stackoverflow.com/questions/56020314/how-frequently-...
    - mxmlnkn 3y ago
      I think this isn't true anymore since at least the introduction of the unwanted Github activity view: https://docs.github.com/en/repositories/viewing-activity-and-data-for-your-repository/using-the-activity-view-to-see-changes-to-a-repository https://docs.github.com/en/repositories/viewing-activity-and...
- dunham 3y ago
  That's only client side, and you also need to "gc" it to get rid of it, or it will still be in .git/objects and can be retrieved via something like `git cat-file`.
  - coldpie 3y ago
    We're getting a little off-topic, but even git-add will put it in the object store without even committing! I once saved my boss's bacon with that. He had git-added a presentation file he'd been working on to commit it, but accidentally nuked his changes with "git-reset --hard" before comitting. He mentioned his mistake in chat, and we were able to recover the lost object by sorting files in the object directory by last-modified and cat-filing it back out by that ID. He bought me a beer for that after work that day. Good times. Read gitcore-tutorial(7), folks. You too might save someone's bacon, some day.
    - 01HNNWZ0MV43FF 3y ago
      I also know this, because I accidentally added a huge media file and `git add .` took too long to return and then `.git` was huge lol
sammorrowdrums 3y ago
Full disclosure, I work for GitHub, but push protection from Secret Scanning is awesome for this because your nearly leaked secret doesn’t make it to the remote, and it gives you instructions on how to fix your local repo!
- lol768 3y ago
  Why does GitHub provide no way for a repository administrator to self-service a git gc? I seem to recall reading a blog post that suggested GitHub had invested a bunch of engineering resource in making cleaning up unreachable objects much more scalable.
  - g-b-r 3y ago
    That's the actual insane problem. I noticed long ago that unreferenced commits survive on GitHub for long, but I couldn't find a way to discover them. I know that GitHub stores together the objects of many repositories, but they should have implemented and offered a way to gc them when they came up with that optimization. Sure, there would still be the chance that someone already obtained the objects by the time you gc them, but it's a much lesser risk then leaving them there indefinitely (and they could provide a log of the last fetches to better assess the impact of the erroneous push).
    - g-b-r 3y ago
      Ok, there's also the problem of accesses through the web interface, but it probably wouldn't take much to provide a short-lived log of them as well
    - londons_explore 3y ago
      > chance that someone already obtained the objects by the time you gc them I was under the impression that there are various 'mirror github' projects that listen to the GitHub change event API and immediately crawl some/all commits. If so, this isn't a chance - it is certain.
  - sammorrowdrums 3y ago
arein3 3y ago
Is it commited or pushed? If I commit something locally, reset it and push to remote something else does it leave a trace?
- msm_ 3y ago
  It is still in your local repository, but it's not pushed to the remote repository. So a forensics on your local machine may reveal it (probably until you do git gc, but I'm not an expert on git forensics) but it's safe otherwise.

This highlights why it's so important that any secret that gets committed must be rotated. Simply removing it from the git history isn't enough, because it can still linger, it's just harder to find.

belinder 3y ago

While I agree that you should rotate accidentally exposed secrets, it should be noted that you can remove old history from git reflog by expiring it I see git reflog kinda like an OS recycle bin

Kluggy 3y ago
How do you do that once it's on Github's servers?
- latchkey 3y ago
  [deleted]
  - javawizard 3y ago
    That link provides instructions on how to access commits from the reflog of a GitHub repository, not how to expire or otherwise delete them.
- KTibow 3y ago
  Sounds like it happens automatically: https://stackoverflow.com/questions/56020314/how-frequently-do-github-run-git-gc-on-repositories https://stackoverflow.com/questions/56020314/how-frequently-...
  - mxmlnkn 3y ago
    I think this isn't true anymore since at least the introduction of the unwanted Github activity view: https://docs.github.com/en/repositories/viewing-activity-and-data-for-your-repository/using-the-activity-view-to-see-changes-to-a-repository https://docs.github.com/en/repositories/viewing-activity-and...
dunham 3y ago
That's only client side, and you also need to "gc" it to get rid of it, or it will still be in .git/objects and can be retrieved via something like `git cat-file`.
- coldpie 3y ago
  We're getting a little off-topic, but even git-add will put it in the object store without even committing! I once saved my boss's bacon with that. He had git-added a presentation file he'd been working on to commit it, but accidentally nuked his changes with "git-reset --hard" before comitting. He mentioned his mistake in chat, and we were able to recover the lost object by sorting files in the object directory by last-modified and cat-filing it back out by that ID. He bought me a beer for that after work that day. Good times. Read gitcore-tutorial(7), folks. You too might save someone's bacon, some day.
  - 01HNNWZ0MV43FF 3y ago
    I also know this, because I accidentally added a huge media file and `git add .` took too long to return and then `.git` was huge lol

sammorrowdrums 3y ago

Full disclosure, I work for GitHub, but push protection from Secret Scanning is awesome for this because your nearly leaked secret doesn’t make it to the remote, and it gives you instructions on how to fix your local repo!

lol768 3y ago
Why does GitHub provide no way for a repository administrator to self-service a git gc? I seem to recall reading a blog post that suggested GitHub had invested a bunch of engineering resource in making cleaning up unreachable objects much more scalable.
- g-b-r 3y ago
  That's the actual insane problem. I noticed long ago that unreferenced commits survive on GitHub for long, but I couldn't find a way to discover them. I know that GitHub stores together the objects of many repositories, but they should have implemented and offered a way to gc them when they came up with that optimization. Sure, there would still be the chance that someone already obtained the objects by the time you gc them, but it's a much lesser risk then leaving them there indefinitely (and they could provide a log of the last fetches to better assess the impact of the erroneous push).
  - g-b-r 3y ago
    Ok, there's also the problem of accesses through the web interface, but it probably wouldn't take much to provide a short-lived log of them as well
  - londons_explore 3y ago
    > chance that someone already obtained the objects by the time you gc them I was under the impression that there are various 'mirror github' projects that listen to the GitHub change event API and immediately crawl some/all commits. If so, this isn't a chance - it is certain.
- sammorrowdrums 3y ago

arein3 3y ago

Is it commited or pushed? If I commit something locally, reset it and push to remote something else does it leave a trace?

msm_ 3y ago
It is still in your local repository, but it's not pushed to the remote repository. So a forensics on your local machine may reveal it (probably until you do git gc, but I'm not an expert on git forensics) but it's safe otherwise.