4 ms·

> Leakage of credentials sure has led to quite a few compromises. No. As per my post above, they "led to compromise" because lazy syadmins have been too inept

> Leakage of credentials sure has led to quite a few compromises. No. As per my post above, they "led to compromise" because lazy syadmins have been too inept to RTFM and make full use of IAM. I mean, I could post my IAM credentials right here. I've even been lazy and haven't rotated them for 5 years (yes I've already slapped myself on the wrist !). But they'll be naff all use to you because: - My credentials are deactivated when I'm not using them. - The STS policies they are linked to are IP-restricted - The Role policies they are linked to are also IP-restricted - The Role policies have minimal necessary privilege - The Role policies can only be invoked be an assumed role - The Roles have externalID attribute set using a UUID as value Thus even if I copy/pasted my credentials right here, you wouldn't even get past step 1 let alone any further. And that's just a simple example. You can go much more layered and granular than that. Its not rocket science, its not difficult, its just a case of the old PPPPPP (Prior Planning Prevents P* Poor Performance).

iampims 4y ago
Share them then.
philsnow 4y ago
We can't expect everybody to get to the level of understanding you have about the fine points of AWS IAM. It's a scalpel shotgun, an extremely sharp tool that can easily blow your company's entire leg off. Companies generally totter along until about 50 engineers / 100 employees before hiring a single security-focused engineer. When those first few security engineers come on board, you can bet they'll have a backlog of at least several months' if not a couple years' worth of research and remediation before they get anywhere near taming the company's IAM setup. The vast majority of companies are small and probably aren't where they need to be in terms of IAM practices. AWS adoption has become a lot more mainstream among companies. 10 years ago it was almost a secret weapon for forward-thinking (and VC-money-burning) companies that let them out-scale competitors. Unfortunately their defaults, documentation, guard rails, etc are still set up for those bleeding-edge, hyper-competent, top-0.01% companies.
- throw_nbvc1234 4y ago
  Who is supposed to understand this then if not everybody? If you think of this as a "security focused engineer" problem then you've already lost. If an engineer doesn't understand how to write secure software then they will continue to write un-secure software and there will be leaks and compromises. You're just externalizing risk and costs to your customers.
jlund-molfese 4y ago
I don’t know why you’re getting downvoted, because you’re completely right. Perhaps people are reading your comment as “credential leakage doesn’t decrease security at all”, but that doesn’t seem to be what you’re saying. A public/leaked set of IAM credentials, using reasonable AWS access policies, should be considered more secure than a private set of credentials that just has unrestricted access to everything.
- traceroute66 4y ago
  > but that doesn’t seem to be what you’re saying Indeed I'm not. To spell it out, what I'm saying is.... People should be working on the assumption that their IAM credentials WILL be leaked and work back from there. That leak may be accidental on Github, it may be via a hack, you might have accidentally pasted it into an email, it doesn't matter, the mechanism of leak is irrelevant. Blast radius / Attack surface / Layered security ... whatever your choice of words, that's what you need to do and that's what AWS IAM gives you the tools to do. As I've previously said, its really not that much effort to add the extra layers of security (e.g. you can start with easy stuff, low-hanging fruit like IP/Time constraints and then move onto more advanced stuff later). Infact I'd argue the detractors here who have been busy downvoting me and arguing against me could have put a lot of extra security on their IAM credentials in the same timeframe. ;-)
  - bagels 4y ago
    It takes a few seconds to respond to your comments. I takes days to implement all of the items that someone may have never even heard of on your list.
nigma1337 4y ago
> My credentials are deactivated when I'm not using them. This sounds rather interesting, mind elaborating how its done? How would they activate when you use them, but not if anyone else does?
- vegardx 4y ago
  I imagine they just log in to the console and activate/deactivate them. But that seems like more steps than necessary. Ideally you'd just use aws sts. In short you request a temporary set of access keys, and authenticate the request with both your access keys and mfa as part of the request. This returns a set of short-lived credentials that you then use. By using a policy attached to the user/role you make sure that the only thing they can request is a set of temporary access keys, with all kinds of conditionals, like ttl, source ip and mfa. You can quite easily monitor cloudtrail for malicious activity and use cloudwatch to notify you when someone uses your credentials from, say, an unknown IP, outside of working hours, etc.
supermatt 4y ago
How do you disable your credentials when not using them? Do you mean using roles (which aren’t credentials) or temporary credentials (which can’t be revoked)? Or do you use other credentials to disable them (and how do you disable those)? I’m not an AWS expert by any stretch, so genuinely interested as I wasn’t aware of any other mechanism you could be using.
simonw 4y ago
I'm going to disagree with you on "it's not difficult". I've been using AWS for fifteen years (since S3 came out) and I don't know how to configure any of the things you list there - at least not without doing deep research into every single one of them. Not saying they're impossible, but they are definitely difficult to figure out!
dap 4y ago
>> Leakage of credentials sure has led to quite a few compromises. > No. > As per my post above, they "led to compromise" because lazy syadmins have been too inept to RTFM and make full use of IAM. You're contradicting yourself: "No, it hasn't led to compromises" but "they led to compromise because ..." It's possible that (as you argue) AWS IAM provides strong security controls and, simultaneously, a lot of people use it in a way where credential leak _does_ represent compromise. My going-in expectation is that both of these are true. But this post isn't about whether AWS IAM is any good. Nor does the post talk about solutions -- I don't see a call to action to buy Datadog (aside from the banner in the footer that doesn't look connected to the post). So your criticism of "this is a non-problem -- everybody's just using it wrong" sounds pretty lame.
- traceroute66 4y ago
  > So your criticism of "this is a non-problem -- everybody's just using it wrong" sounds pretty lame. I don't follow your argument. Perhaps let me try with simpler example. I'll tell you my credit card PIN number is 7341. Following your line of argument, you'll stand there screaming until you're blue in the face: "that's a secret credential leak, your card is compromised". To which I say: "well, my card is in my wallet, and my wallet is in my pocket, so watcha' gonna' do with that PIN number chum ?" So its the same with AWS. I could tell you one of my access keys is AKIA3CWKZKKGZLHN and its secret access key is QOF0yG/lvqqAcklAHCDzPKRtk9D5oPnY. But watcha gonna do with it ? Try logging in ? To what ? And even if you knew which AWS service, even with just low-hanging fruit security layers of IP range restrictions and time-gating you still won't get anywhere. Once we start adding extra layers such as roles and STS on top, then frankly you're more likely to win the jackpot on the lotto twice in a row. If its not usable, its not compromise.
  - dap 4y ago
    I do get what you're saying. And yes, a credential leak is not _equivalent_ to a compromise. And yes, if you're doing things well, it won't be. (Although I think most people would say one of your layers of defense is breached and one ought to rotate the credentials, right? Otherwise, why have the credentials at all?) I think that's an important, somewhat tangential point. Maybe we came to this with different assumptions? My going-in assumption is that most people have not set up those extra layers that you're talking about. For them, a credential compromise is tantamount to an account compromise. Thus, a post like this is relevant to raise awareness of credential compromise (and sure, maybe a missed opportunity to talk about those other layers one could add). Is your going-in assumption that most people _are_ using those extra layers and so the post is pointless because it _erroneously_ implies that a lot of folks might be more exposed than they think? That's not what I thought you were saying. I thought you were saying: "if a credential leak compromises your account, then you're doing it wrong". That might be true, but if there are lots of people doing it wrong, then it doesn't matter. (I don't like your PIN example because in real life, the initial conditions set by the bank are that you know your PIN and you have your card. You have to go out of your way to expose both. By contrast, with the AWS credentials, you have to have taken the extra steps you mention to establish the extra layers that you're talking about (IP range restrictions, time gating, etc.))

iampims 4y ago

Share them then.

philsnow 4y ago

We can't expect everybody to get to the level of understanding you have about the fine points of AWS IAM. It's a scalpel shotgun, an extremely sharp tool that can easily blow your company's entire leg off. Companies generally totter along until about 50 engineers / 100 employees before hiring a single security-focused engineer. When those first few security engineers come on board, you can bet they'll have a backlog of at least several months' if not a couple years' worth of research and remediation before they get anywhere near taming the company's IAM setup. The vast majority of companies are small and probably aren't where they need to be in terms of IAM practices. AWS adoption has become a lot more mainstream among companies. 10 years ago it was almost a secret weapon for forward-thinking (and VC-money-burning) companies that let them out-scale competitors. Unfortunately their defaults, documentation, guard rails, etc are still set up for those bleeding-edge, hyper-competent, top-0.01% companies.

throw_nbvc1234 4y ago
Who is supposed to understand this then if not everybody? If you think of this as a "security focused engineer" problem then you've already lost. If an engineer doesn't understand how to write secure software then they will continue to write un-secure software and there will be leaks and compromises. You're just externalizing risk and costs to your customers.

jlund-molfese 4y ago

I don’t know why you’re getting downvoted, because you’re completely right. Perhaps people are reading your comment as “credential leakage doesn’t decrease security at all”, but that doesn’t seem to be what you’re saying. A public/leaked set of IAM credentials, using reasonable AWS access policies, should be considered more secure than a private set of credentials that just has unrestricted access to everything.

traceroute66 4y ago
> but that doesn’t seem to be what you’re saying Indeed I'm not. To spell it out, what I'm saying is.... People should be working on the assumption that their IAM credentials WILL be leaked and work back from there. That leak may be accidental on Github, it may be via a hack, you might have accidentally pasted it into an email, it doesn't matter, the mechanism of leak is irrelevant. Blast radius / Attack surface / Layered security ... whatever your choice of words, that's what you need to do and that's what AWS IAM gives you the tools to do. As I've previously said, its really not that much effort to add the extra layers of security (e.g. you can start with easy stuff, low-hanging fruit like IP/Time constraints and then move onto more advanced stuff later). Infact I'd argue the detractors here who have been busy downvoting me and arguing against me could have put a lot of extra security on their IAM credentials in the same timeframe. ;-)
- bagels 4y ago
  It takes a few seconds to respond to your comments. I takes days to implement all of the items that someone may have never even heard of on your list.

nigma1337 4y ago

> My credentials are deactivated when I'm not using them. This sounds rather interesting, mind elaborating how its done? How would they activate when you use them, but not if anyone else does?

vegardx 4y ago
I imagine they just log in to the console and activate/deactivate them. But that seems like more steps than necessary. Ideally you'd just use aws sts. In short you request a temporary set of access keys, and authenticate the request with both your access keys and mfa as part of the request. This returns a set of short-lived credentials that you then use. By using a policy attached to the user/role you make sure that the only thing they can request is a set of temporary access keys, with all kinds of conditionals, like ttl, source ip and mfa. You can quite easily monitor cloudtrail for malicious activity and use cloudwatch to notify you when someone uses your credentials from, say, an unknown IP, outside of working hours, etc.

supermatt 4y ago

How do you disable your credentials when not using them? Do you mean using roles (which aren’t credentials) or temporary credentials (which can’t be revoked)? Or do you use other credentials to disable them (and how do you disable those)? I’m not an AWS expert by any stretch, so genuinely interested as I wasn’t aware of any other mechanism you could be using.

simonw 4y ago

I'm going to disagree with you on "it's not difficult". I've been using AWS for fifteen years (since S3 came out) and I don't know how to configure any of the things you list there - at least not without doing deep research into every single one of them. Not saying they're impossible, but they are definitely difficult to figure out!

dap 4y ago

>> Leakage of credentials sure has led to quite a few compromises. > No. > As per my post above, they "led to compromise" because lazy syadmins have been too inept to RTFM and make full use of IAM. You're contradicting yourself: "No, it hasn't led to compromises" but "they led to compromise because ..." It's possible that (as you argue) AWS IAM provides strong security controls and, simultaneously, a lot of people use it in a way where credential leak _does_ represent compromise. My going-in expectation is that both of these are true. But this post isn't about whether AWS IAM is any good. Nor does the post talk about solutions -- I don't see a call to action to buy Datadog (aside from the banner in the footer that doesn't look connected to the post). So your criticism of "this is a non-problem -- everybody's just using it wrong" sounds pretty lame.

traceroute66 4y ago
> So your criticism of "this is a non-problem -- everybody's just using it wrong" sounds pretty lame. I don't follow your argument. Perhaps let me try with simpler example. I'll tell you my credit card PIN number is 7341. Following your line of argument, you'll stand there screaming until you're blue in the face: "that's a secret credential leak, your card is compromised". To which I say: "well, my card is in my wallet, and my wallet is in my pocket, so watcha' gonna' do with that PIN number chum ?" So its the same with AWS. I could tell you one of my access keys is AKIA3CWKZKKGZLHN and its secret access key is QOF0yG/lvqqAcklAHCDzPKRtk9D5oPnY. But watcha gonna do with it ? Try logging in ? To what ? And even if you knew which AWS service, even with just low-hanging fruit security layers of IP range restrictions and time-gating you still won't get anywhere. Once we start adding extra layers such as roles and STS on top, then frankly you're more likely to win the jackpot on the lotto twice in a row. If its not usable, its not compromise.
- dap 4y ago
  I do get what you're saying. And yes, a credential leak is not _equivalent_ to a compromise. And yes, if you're doing things well, it won't be. (Although I think most people would say one of your layers of defense is breached and one ought to rotate the credentials, right? Otherwise, why have the credentials at all?) I think that's an important, somewhat tangential point. Maybe we came to this with different assumptions? My going-in assumption is that most people have not set up those extra layers that you're talking about. For them, a credential compromise is tantamount to an account compromise. Thus, a post like this is relevant to raise awareness of credential compromise (and sure, maybe a missed opportunity to talk about those other layers one could add). Is your going-in assumption that most people _are_ using those extra layers and so the post is pointless because it _erroneously_ implies that a lot of folks might be more exposed than they think? That's not what I thought you were saying. I thought you were saying: "if a credential leak compromises your account, then you're doing it wrong". That might be true, but if there are lots of people doing it wrong, then it doesn't matter. (I don't like your PIN example because in real life, the initial conditions set by the bank are that you know your PIN and you have your card. You have to go out of your way to expose both. By contrast, with the AWS credentials, you have to have taken the extra steps you mention to establish the extra layers that you're talking about (IP range restrictions, time gating, etc.))