11 ms·

Someone at my company generated the keys. They then put them on a network share without any security restrictions. They've been there for 5 years with no rotati

Someone at my company generated the keys. They then put them on a network share without any security restrictions. They've been there for 5 years with no rotation. At least 2 are checked into source control.

dijit 6y ago
If those are the keys used in production, then I'm horrified. If they're dev-keys, I think this is pretty common.
- thieving_magpie 6y ago
  We make no distinction between dev keys and production. Consider them production. Since it's of interest to HN, I am working on educating our very small team on how keys should be protected and used. I am the youngest developer by about 15 years. It's a very rural company and it often feels like all learning and passion for development stalled around 2005. It's a company that gave me a chance to grow into a development role with no previous experience so I feel indebted to try my best to keep the lights on.
  - dijit 6y ago
    aha, sounds fair. I don't judge too harshly- anyone who has black and white principles on these matters has never worked in any other industry most likely... all you can do is your best to steer the ship and convey the downsides. I think it's important too because it helps us understand how much friction people will tolerate. In many cases, even a small amount of friction will cause people to stop functioning completely; I recently tried setting up vault and it was a nightmare, I understand why people avoid picking it up. That doesn't mean we should not try; we have to become the advocates, arbiters and helpers for those systems. Good luck, you're not alone.
    - thieving_magpie 6y ago
      Thanks for the kind words. Good luck to you as well.
    - Ididntdothis 6y ago
      One thing i have noticed is that “security conscious” people are very good at criticizing things and pointing out flaws. But they are not as good at proposing clear and workable solutions that don’t add huge burden to users. It should be no surprise that people do insecure stuff under deadline pressure.
      - a1369209993 6y ago
        > are very good at criticizing things [but] not as good at proposing clear and workable solutions This is definitely true, but not actually surprising. It's much easier to notice that, say, a violin performance or plumbing repair is done very badly, than it is to actually do it correctly yourself. Which also leads to a great deal of exasperation when people (either apparently or actually) don't even notice that what they're doing is insecure. There's a big difference between "yeah, it's broken, but it'd be a huge pain to fix and we'd probably get it wrong anyway, so we'd rather take our chances" versus "there is no problem".
  - uaas 6y ago
    Using different ones for dev and prod still might be good idea. If either one is compromised, there’s a chance the other is safe. You can still rotate them regularly, and/or if either one is compromised.
panpanna 6y ago
If this is a public repo then you are already hacked. There are multiple automated systems that scan public repos for credentials. 5 minutes later you are mining bitcoins for them.
- thieving_magpie 6y ago
  Luckily I was in charge of setting up our remote repositories and made sure that by default they are all private repos, otherwise this would have happened without a doubt.
- toomuchtodo 6y ago
  Github monitors for public commits of service secrets. Not an excuse to commit secrets, but there is a bit of a safety net. > When you push to a public repository, GitHub scans the content of the commits for secrets. If you switch a private repository to public, GitHub scans the entire repository for secrets. > When secret scanning detects a set of credentials, we notify the service provider who issued the secret. The service provider validates the credential and then decides whether they should revoke the secret, issue a new secret, or reach out to you directly, which will depend on the associated risks to you or the service provider. https://help.github.com/en/github/administering-a-repository/about-secret-scanning https://help.github.com/en/github/administering-a-repository...
- ganstyles 6y ago
  In a boneheaded movei I accidentally committed my SendGrid creds to GitHub. Pretty quickly after, GitHub alerted me. However by then my SG account was sending thousands of automated spam messages. Those automated scammer systems are FAST. Not particularly germane to the discussion, but really disappointed in how SendGrid handled things. I notified them immediately, rotated all API tokens, and tey could not turn it off, so the spammer sent messages for days and eventually my SG account got suspended.
  - jabart 6y ago
    My SG API keys, for an account that we terminated still send out emails if I happen to use an old config for a service. IP Rules are super helpful in this case, still need to rotate when exposed but can limit the exposure.
2mol 6y ago
This is an amazing answer, thank you for sharing :) I wish you the energy to keep trying to change this, don't let it get to you too much, I know this stuff can be frustrating as hell.
AlfeG 6y ago
We have very simmiliar issue. All our databases have password Qwerty1234 Android keystore is checked in repository with access key in scripts. Security keys for external services are also checked in into repository. Some external services for production are managed by devs that are long time ago not working in our company
- eitland 6y ago
  Hehe. Less than 8 years ago I asked for help to add a column in a database at a company I helped. This was a few days after they met me for the first time. The company solved this by giving me a root username and password that worked on every single important database in the company, at least every customer database. I had to beg them to create a somewhat restricted account. The same company was however deeply sceptical to all kinds of remote work. The security equivalent of penny wise pound foolish I guess :-]
  - AlfeG 6y ago
    On one my past job there were fingerprint reader system on enter to office. Almost 6 years later, I were still able to enter office with my fingerprint.
    - shoo 6y ago
      At one of my past jobs there was a fingerprint reader system to enter the office. It didn't work reliably to recognise fingerprints of employees, so after a while people settled on the solution of having a large brick next to the door which was used to wedge the door open during the daytime after the first person managed to get the door open in the morning.
      - acruns 6y ago
        I get this same thing with being invited to an Azure instance. years later I still have full access
  - BrandoElFollito 6y ago
mpfundstein 6y ago
like every startup ever in the last startup I worked, all jwt tokens were created from a 10 letter long shared "secret" stored in json config files all over the place :p even dev environments had same key lol
- mister_hn 6y ago
  like enterprise companies too!
mister_hn 6y ago
Our company is so too.
denysvitali 6y ago
That's smart! At least the secret is now versioned, cool!
jbn 6y ago
This would be hilarious if it were not so telling about the state of security in general (not at this company in particular, i'm certain many if not most companies do the same...).
- a1369209993 6y ago
  It's still hilarious; it just takes a certain level of bitterness and cynicism to appreciate.
clintfred 6y ago
I posted elsewhere about https://github.com/IronCoreLabs/ironhide https://github.com/IronCoreLabs/ironhide a tool purpose built for sharing developer/CI secrets.
maest 6y ago
Sounds very convenient to use.

dijit 6y ago

If those are the keys used in production, then I'm horrified. If they're dev-keys, I think this is pretty common.

thieving_magpie 6y ago
We make no distinction between dev keys and production. Consider them production. Since it's of interest to HN, I am working on educating our very small team on how keys should be protected and used. I am the youngest developer by about 15 years. It's a very rural company and it often feels like all learning and passion for development stalled around 2005. It's a company that gave me a chance to grow into a development role with no previous experience so I feel indebted to try my best to keep the lights on.
- dijit 6y ago
  aha, sounds fair. I don't judge too harshly- anyone who has black and white principles on these matters has never worked in any other industry most likely... all you can do is your best to steer the ship and convey the downsides. I think it's important too because it helps us understand how much friction people will tolerate. In many cases, even a small amount of friction will cause people to stop functioning completely; I recently tried setting up vault and it was a nightmare, I understand why people avoid picking it up. That doesn't mean we should not try; we have to become the advocates, arbiters and helpers for those systems. Good luck, you're not alone.
  - thieving_magpie 6y ago
    Thanks for the kind words. Good luck to you as well.
  - Ididntdothis 6y ago
    One thing i have noticed is that “security conscious” people are very good at criticizing things and pointing out flaws. But they are not as good at proposing clear and workable solutions that don’t add huge burden to users. It should be no surprise that people do insecure stuff under deadline pressure.
    - a1369209993 6y ago
      > are very good at criticizing things [but] not as good at proposing clear and workable solutions This is definitely true, but not actually surprising. It's much easier to notice that, say, a violin performance or plumbing repair is done very badly, than it is to actually do it correctly yourself. Which also leads to a great deal of exasperation when people (either apparently or actually) don't even notice that what they're doing is insecure. There's a big difference between "yeah, it's broken, but it'd be a huge pain to fix and we'd probably get it wrong anyway, so we'd rather take our chances" versus "there is no problem".
- uaas 6y ago
  Using different ones for dev and prod still might be good idea. If either one is compromised, there’s a chance the other is safe. You can still rotate them regularly, and/or if either one is compromised.

panpanna 6y ago

If this is a public repo then you are already hacked. There are multiple automated systems that scan public repos for credentials. 5 minutes later you are mining bitcoins for them.

thieving_magpie 6y ago
Luckily I was in charge of setting up our remote repositories and made sure that by default they are all private repos, otherwise this would have happened without a doubt.
toomuchtodo 6y ago
Github monitors for public commits of service secrets. Not an excuse to commit secrets, but there is a bit of a safety net. > When you push to a public repository, GitHub scans the content of the commits for secrets. If you switch a private repository to public, GitHub scans the entire repository for secrets. > When secret scanning detects a set of credentials, we notify the service provider who issued the secret. The service provider validates the credential and then decides whether they should revoke the secret, issue a new secret, or reach out to you directly, which will depend on the associated risks to you or the service provider. https://help.github.com/en/github/administering-a-repository/about-secret-scanning https://help.github.com/en/github/administering-a-repository...
ganstyles 6y ago
In a boneheaded movei I accidentally committed my SendGrid creds to GitHub. Pretty quickly after, GitHub alerted me. However by then my SG account was sending thousands of automated spam messages. Those automated scammer systems are FAST. Not particularly germane to the discussion, but really disappointed in how SendGrid handled things. I notified them immediately, rotated all API tokens, and tey could not turn it off, so the spammer sent messages for days and eventually my SG account got suspended.
- jabart 6y ago
  My SG API keys, for an account that we terminated still send out emails if I happen to use an old config for a service. IP Rules are super helpful in this case, still need to rotate when exposed but can limit the exposure.

2mol 6y ago

This is an amazing answer, thank you for sharing :) I wish you the energy to keep trying to change this, don't let it get to you too much, I know this stuff can be frustrating as hell.

AlfeG 6y ago

We have very simmiliar issue. All our databases have password Qwerty1234 Android keystore is checked in repository with access key in scripts. Security keys for external services are also checked in into repository. Some external services for production are managed by devs that are long time ago not working in our company

eitland 6y ago
Hehe. Less than 8 years ago I asked for help to add a column in a database at a company I helped. This was a few days after they met me for the first time. The company solved this by giving me a root username and password that worked on every single important database in the company, at least every customer database. I had to beg them to create a somewhat restricted account. The same company was however deeply sceptical to all kinds of remote work. The security equivalent of penny wise pound foolish I guess :-]
- AlfeG 6y ago
  On one my past job there were fingerprint reader system on enter to office. Almost 6 years later, I were still able to enter office with my fingerprint.
  - shoo 6y ago
    At one of my past jobs there was a fingerprint reader system to enter the office. It didn't work reliably to recognise fingerprints of employees, so after a while people settled on the solution of having a large brick next to the door which was used to wedge the door open during the daytime after the first person managed to get the door open in the morning.
    - acruns 6y ago
      I get this same thing with being invited to an Azure instance. years later I still have full access
- BrandoElFollito 6y ago

mpfundstein 6y ago

like every startup ever in the last startup I worked, all jwt tokens were created from a 10 letter long shared "secret" stored in json config files all over the place :p even dev environments had same key lol

mister_hn 6y ago
like enterprise companies too!

mister_hn 6y ago

Our company is so too.

denysvitali 6y ago

That's smart! At least the secret is now versioned, cool!

jbn 6y ago

This would be hilarious if it were not so telling about the state of security in general (not at this company in particular, i'm certain many if not most companies do the same...).

a1369209993 6y ago
It's still hilarious; it just takes a certain level of bitterness and cynicism to appreciate.

clintfred 6y ago

I posted elsewhere about https://github.com/IronCoreLabs/ironhide https://github.com/IronCoreLabs/ironhide a tool purpose built for sharing developer/CI secrets.

maest 6y ago

Sounds very convenient to use.