5 ms·

Please don't require static AWS credentials: https://github.com/Qovery/replibyte/blob/v0.4.4/replibyte/src/bridge/s3.rs#L36-L37 https://github.com/Qovery/replib

Please don't require static AWS credentials: https://github.com/Qovery/replibyte/blob/v0.4.4/replibyte/src/bridge/s3.rs#L36-L37 https://github.com/Qovery/replibyte/blob/v0.4.4/replibyte/sr... or at least either include "AWS_SESSION_TOKEN" in that setup (if it is present) in order to allow "aws sts assume-role" to work, or allow `AWS_PROFILE`, or just use the aws-sdk's normal credential discovery mechanism which at least on their "main" SDKs is a fallback list of them, but I couldn't follow the docs.rs soup in order to know if their rust sdk is up to speed or what

herpderperator 5y ago
Why?
- jensenbox 5y ago
  Static tokens are terrible. They are far too easy to egress and are downright evil.
- alphabettsy 5y ago
  I presume so that one can use instance roles or automatic credential discovery that is common within AWS and using AWS applications.
- macksd 5y ago
  Some organizations use this feature to enforce security policy and ensure regular rotation of credentials. You can't get a static access key ID / secret access key pair, but you can get one with a session token, and so you run everything inside that context. If you can't specify the session token to an Amazon client, you just can't use Amazon STS.
- kodah 5y ago
  Credentials in a config file can be mistakenly checked into a repository. They're easy to exfiltrate from files, say I write a script with well known configuration locations for thousands of applications and just dumbly pull them all from a compromised system. I now have little bits of access to the wider system where I can now jump from system to system. The best way to store ephemeral secrets is in an environment variable or /dev/shm. This locks the secret behind the scope of the parent process (shell instance) and the user.
  - osrec 5y ago
    How do you automate setting that environment variable?
    - toomuchtodo 5y ago
      Injection into the environment (container config, Docker, k8s, etc) or the execution context. The actual mechanics are highly dependent on where your secrets are and where they’re going for use. On a dev workstation, could just be some bash or Python using the AWS cli, for example.
    - dboreham 5y ago
      For a specific example of one way how : https://github.com/99designs/aws-vault https://github.com/99designs/aws-vault
  - dvt 5y ago
    I don't get this logic, that's what .gitignore is for. I've been using .env files for years and never mistakenly checked one into a repo.
    - kristjansson 5y ago
      It’s not just to protect secrets, it’s better architecture. Rely on an ACCESS_ID/SECRET_KEY pair and it’s easy to bind too tightly to that authentication mechanism. Then providing credentials in production is a pain. Use the standard credential provider chain and the transition to production settings is trivial.
    - dboreham 5y ago
      Still a bad idea though.
    - kodah 5y ago
      idk what to tell you. I work on a security team, one of the tools the team built finds and identifies secrets already checked into VCS or ones at the pre-commit stage. It's certainly not a seldomly used tool.
- Aeolun 5y ago
  One big reason is it’s insecure. Another big reason is it’s much nicer to deploy on any AWS service and have the SDK use the metadata host, which will automatically provide you with a temporary access token with the permissions of the role you set for it.
moltar 5y ago
Second this. I would NEVER hardcode credentials into a config file. I run all AWS commands through an assumed role (STS) via aws-vault.
atonse 5y ago
Should work out of the box if they’re using the rust AWS library right?
- mdaniel 5y ago
  That was the "I can't grok the docs.rs in order to tell you" part; in boto and the sane SDKs, there's a bunch of "Credential Providers" and they're in a list, meaning some are tried before others I have no idea what implementations there are for this: https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/struct.Credentials.html#impl-ProvideCredentials https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/struct.Credenti... and its official page is even worse: https://docs.aws.amazon.com/sdk-for-rust/latest/dg/credentials.html https://docs.aws.amazon.com/sdk-for-rust/latest/dg/credentia... Going all the way down to the GH readme seems to back up the investigation that, no, they really seem to have forgotten about "AWS_SESSION_TOKEN": https://github.com/awslabs/aws-sdk-rust#getting-started-with-the-sdk https://github.com/awslabs/aws-sdk-rust#getting-started-with...
  - simtel20 5y ago
    Not sure if this is any better than the docs (which are raw, at best) - I'm a middling beginner rust programmer, but I did do this test[1] for myself and it matches most of the other SDKs. [1] https://github.com/pcn/check-assume-role-s3/blob/master/src/main.rs https://github.com/pcn/check-assume-role-s3/blob/master/src/...
    - mdaniel 5y ago
      That's excellent news, thanks so much for taking the time to put that together!
evoxmusic 5y ago
You can use env vars in the conf yaml file -> https://github.com/Qovery/replibyte/blob/main/examples/source-postgres-with-env-vars.yaml https://github.com/Qovery/replibyte/blob/main/examples/sourc...
- mdaniel 5y ago
  That is better, but apologies that I overloaded the word "static" -- I meant the non-assume-role flavor, "aws iam create-access-key", static style credentials. Static in that from AWS's perspective, once issued they don't expire, unlike session credentials that have a finite lifespan I can now see how my wording was confusing, I'll try to be more precise in the future

herpderperator 5y ago

Why?

jensenbox 5y ago
Static tokens are terrible. They are far too easy to egress and are downright evil.
alphabettsy 5y ago
I presume so that one can use instance roles or automatic credential discovery that is common within AWS and using AWS applications.
macksd 5y ago
Some organizations use this feature to enforce security policy and ensure regular rotation of credentials. You can't get a static access key ID / secret access key pair, but you can get one with a session token, and so you run everything inside that context. If you can't specify the session token to an Amazon client, you just can't use Amazon STS.
kodah 5y ago
Credentials in a config file can be mistakenly checked into a repository. They're easy to exfiltrate from files, say I write a script with well known configuration locations for thousands of applications and just dumbly pull them all from a compromised system. I now have little bits of access to the wider system where I can now jump from system to system. The best way to store ephemeral secrets is in an environment variable or /dev/shm. This locks the secret behind the scope of the parent process (shell instance) and the user.
- osrec 5y ago
  How do you automate setting that environment variable?
  - toomuchtodo 5y ago
    Injection into the environment (container config, Docker, k8s, etc) or the execution context. The actual mechanics are highly dependent on where your secrets are and where they’re going for use. On a dev workstation, could just be some bash or Python using the AWS cli, for example.
  - dboreham 5y ago
    For a specific example of one way how : https://github.com/99designs/aws-vault https://github.com/99designs/aws-vault
- dvt 5y ago
  I don't get this logic, that's what .gitignore is for. I've been using .env files for years and never mistakenly checked one into a repo.
  - kristjansson 5y ago
    It’s not just to protect secrets, it’s better architecture. Rely on an ACCESS_ID/SECRET_KEY pair and it’s easy to bind too tightly to that authentication mechanism. Then providing credentials in production is a pain. Use the standard credential provider chain and the transition to production settings is trivial.
  - dboreham 5y ago
    Still a bad idea though.
  - kodah 5y ago
    idk what to tell you. I work on a security team, one of the tools the team built finds and identifies secrets already checked into VCS or ones at the pre-commit stage. It's certainly not a seldomly used tool.
Aeolun 5y ago
One big reason is it’s insecure. Another big reason is it’s much nicer to deploy on any AWS service and have the SDK use the metadata host, which will automatically provide you with a temporary access token with the permissions of the role you set for it.

moltar 5y ago

Second this. I would NEVER hardcode credentials into a config file. I run all AWS commands through an assumed role (STS) via aws-vault.

atonse 5y ago

Should work out of the box if they’re using the rust AWS library right?

mdaniel 5y ago
That was the "I can't grok the docs.rs in order to tell you" part; in boto and the sane SDKs, there's a bunch of "Credential Providers" and they're in a list, meaning some are tried before others I have no idea what implementations there are for this: https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/struct.Credentials.html#impl-ProvideCredentials https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/struct.Credenti... and its official page is even worse: https://docs.aws.amazon.com/sdk-for-rust/latest/dg/credentials.html https://docs.aws.amazon.com/sdk-for-rust/latest/dg/credentia... Going all the way down to the GH readme seems to back up the investigation that, no, they really seem to have forgotten about "AWS_SESSION_TOKEN": https://github.com/awslabs/aws-sdk-rust#getting-started-with-the-sdk https://github.com/awslabs/aws-sdk-rust#getting-started-with...
- simtel20 5y ago
  Not sure if this is any better than the docs (which are raw, at best) - I'm a middling beginner rust programmer, but I did do this test[1] for myself and it matches most of the other SDKs. [1] https://github.com/pcn/check-assume-role-s3/blob/master/src/main.rs https://github.com/pcn/check-assume-role-s3/blob/master/src/...
  - mdaniel 5y ago
    That's excellent news, thanks so much for taking the time to put that together!

evoxmusic 5y ago

You can use env vars in the conf yaml file -> https://github.com/Qovery/replibyte/blob/main/examples/source-postgres-with-env-vars.yaml https://github.com/Qovery/replibyte/blob/main/examples/sourc...

mdaniel 5y ago
That is better, but apologies that I overloaded the word "static" -- I meant the non-assume-role flavor, "aws iam create-access-key", static style credentials. Static in that from AWS's perspective, once issued they don't expire, unlike session credentials that have a finite lifespan I can now see how my wording was confusing, I'll try to be more precise in the future