5 ms·

This looks like an AWS equivalent of the Amazon-internal secret management tool called Odin. Which is very nice because Odin was pretty much universally loved f

This looks like an AWS equivalent of the Amazon-internal secret management tool called Odin. Which is very nice because Odin was pretty much universally loved from what I saw.

josefdlange 9y ago
I miss having ODIN.
- deleted 9y ago
  [deleted]
valar_m 9y ago
Odin works well, it's just a hassle to share credentials with non-developers who don't have a dev desktop and therefore can't access Odin. It's an issue I've encountered several times recently with no clear solution, it seems.
qxi 9y ago
Odin works really well but but only because it integrates so tightly with with the rest of the internal tooling. Without Apollo (another internal tool) to manage and orchestrate the deployment of packages that need Odin's secrets, it's a lot clumsier to use.
ryanianian 9y ago
There's still a bootstrap issue with AWS Secrets Manager - you have to set up enough tooling to be able to call the API. The killer-app of odin imho was/is the on-machine http server that let all manner of applications very easily get credentials. You could just do something like MY_API_KEY="$(curl http://localhost:5000?key=my-api)" and boom your shell script or whatever was very easily using secrets. The fact that Amazon systems bootstrapped EC2 instances meant they could easily enable this org-wide. The odin back-end then owned all the lifecycles around those secrets including what kinds of hosts they would go to, if/how they would rotate, etc etc. It has its annoyances including the fact you have to use http to localhost which can get saturated if you're not doing things right, but overall it really made secrets-management a non-issue.
- mdeeks 9y ago
  That is actually pretty scary from a security perspective because of SSRF attacks. Basically if I can get your service to make an HTTP request on my behalf then I can get at your secrets. Suppose you have a service that fetches snippets from user pasted links (like Slack does). All I have to do is paste a link of "http://localhost:5000?key=my-api" http://localhost:5000?key=my-api" and your server would return a snippet containing the secret. Thats why Google Cloud and others require a special header to be set. Hopefully Odin does the same? https://cloud.google.com/compute/docs/storing-retrieving-metadata#querying https://cloud.google.com/compute/docs/storing-retrieving-met...
  - phamilton 9y ago
    This is already an issue on EC2. https://ionize.com.au/stealing-amazon-ec2-keys-via-xss-vulnerability/ https://ionize.com.au/stealing-amazon-ec2-keys-via-xss-vulne...
- phamilton 9y ago
  This is exactly how things already work on EC2. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-role... TL;DR; curl http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access will get you the credentials for s3 access if you define a role named s3access and grant that role the associated permissions.
ronnier 9y ago
After leaving Amazon you really go on to understand how good the internal tools are at Amazon and how they probably have the best tools in the industry.
- WaxProlix 9y ago
  Except SIM.
  - Zombieball 9y ago
    +1 Hahaha! This made my day.
  - cheeze 9y ago
    The Lotus Notes of Amazon
- harryh 9y ago
  As a former Googler, I know a lot of Googlers that feel exactly the same way. It would be interesting for someone who has deeply experienced both to weigh in.
- mabbo 9y ago
  Brazil + Apollo + Pipelines. I left (and later boomeranged) and was surprised that the outside world hadn't solved these problems nearly as well. Don't get me wrong, Brazil and Apollo probably are ready for a rethink and rewrite, but they do a great job.
  - mrep 9y ago
    What would you change about them?
    - mabbo 9y ago
      Hard to say (especially in as public forum like this). Brazil's dependency model is, I think, very much worth rethinking. Just try to use it for NodeJS. Apollo is fine but slow and dated and needs love. Pipelines works for specific software models and not others (ever seen a pipeline with 500 envs?). Great design for building custom approval steps though (long ago I wrote a now-popular one).
jcims 9y ago
Surprised at all the folks openly discussing internal tooling.
- mabbo 9y ago
  There are a specific set of internal things at Amazon that public discussion of would be detrimental to your career. Saying what you think of Odin isn't quite the same as mentioning the release date of the next Kindle.
  - jcims 9y ago
    Of course, and I don't think anyone would or should get reprimanded for it. That said, I still think it's poor form and should be avoided unless absolutely necessary.

This looks like an AWS equivalent of the Amazon-internal secret management tool called Odin. Which is very nice because Odin was pretty much universally loved from what I saw.

josefdlange 9y ago

I miss having ODIN.

deleted 9y ago
[deleted]

valar_m 9y ago

Odin works well, it's just a hassle to share credentials with non-developers who don't have a dev desktop and therefore can't access Odin. It's an issue I've encountered several times recently with no clear solution, it seems.

qxi 9y ago

Odin works really well but but only because it integrates so tightly with with the rest of the internal tooling. Without Apollo (another internal tool) to manage and orchestrate the deployment of packages that need Odin's secrets, it's a lot clumsier to use.

ryanianian 9y ago

There's still a bootstrap issue with AWS Secrets Manager - you have to set up enough tooling to be able to call the API. The killer-app of odin imho was/is the on-machine http server that let all manner of applications very easily get credentials. You could just do something like MY_API_KEY="$(curl http://localhost:5000?key=my-api)" and boom your shell script or whatever was very easily using secrets. The fact that Amazon systems bootstrapped EC2 instances meant they could easily enable this org-wide. The odin back-end then owned all the lifecycles around those secrets including what kinds of hosts they would go to, if/how they would rotate, etc etc. It has its annoyances including the fact you have to use http to localhost which can get saturated if you're not doing things right, but overall it really made secrets-management a non-issue.

mdeeks 9y ago
That is actually pretty scary from a security perspective because of SSRF attacks. Basically if I can get your service to make an HTTP request on my behalf then I can get at your secrets. Suppose you have a service that fetches snippets from user pasted links (like Slack does). All I have to do is paste a link of "http://localhost:5000?key=my-api" http://localhost:5000?key=my-api" and your server would return a snippet containing the secret. Thats why Google Cloud and others require a special header to be set. Hopefully Odin does the same? https://cloud.google.com/compute/docs/storing-retrieving-metadata#querying https://cloud.google.com/compute/docs/storing-retrieving-met...
- phamilton 9y ago
  This is already an issue on EC2. https://ionize.com.au/stealing-amazon-ec2-keys-via-xss-vulnerability/ https://ionize.com.au/stealing-amazon-ec2-keys-via-xss-vulne...
phamilton 9y ago
This is exactly how things already work on EC2. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-role... TL;DR; curl http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access will get you the credentials for s3 access if you define a role named s3access and grant that role the associated permissions.

ronnier 9y ago

After leaving Amazon you really go on to understand how good the internal tools are at Amazon and how they probably have the best tools in the industry.

WaxProlix 9y ago
Except SIM.
- Zombieball 9y ago
  +1 Hahaha! This made my day.
- cheeze 9y ago
  The Lotus Notes of Amazon
harryh 9y ago
As a former Googler, I know a lot of Googlers that feel exactly the same way. It would be interesting for someone who has deeply experienced both to weigh in.
mabbo 9y ago
Brazil + Apollo + Pipelines. I left (and later boomeranged) and was surprised that the outside world hadn't solved these problems nearly as well. Don't get me wrong, Brazil and Apollo probably are ready for a rethink and rewrite, but they do a great job.
- mrep 9y ago
  What would you change about them?
  - mabbo 9y ago
    Hard to say (especially in as public forum like this). Brazil's dependency model is, I think, very much worth rethinking. Just try to use it for NodeJS. Apollo is fine but slow and dated and needs love. Pipelines works for specific software models and not others (ever seen a pipeline with 500 envs?). Great design for building custom approval steps though (long ago I wrote a now-popular one).

jcims 9y ago

Surprised at all the folks openly discussing internal tooling.

mabbo 9y ago
There are a specific set of internal things at Amazon that public discussion of would be detrimental to your career. Saying what you think of Odin isn't quite the same as mentioning the release date of the next Kindle.
- jcims 9y ago
  Of course, and I don't think anyone would or should get reprimanded for it. That said, I still think it's poor form and should be avoided unless absolutely necessary.