5 ms·

Setting up AWS so you can try it via Amazon Bedrock API is a hassle, so I made a step-by-step guide: https://ndurner.github.io/amazon-nova https://ndurner.githu

by ndr_ 2y ago

Setting up AWS so you can try it via Amazon Bedrock API is a hassle, so I made a step-by-step guide: https://ndurner.github.io/amazon-nova https://ndurner.github.io/amazon-nova. It's 14+ steps!

simonw 2y ago
Thank you!
teruakohatu 2y ago
Thanks for that. Are there any proxies that can communicate with bedrock and serve it via a OpenAI style api?
- moduspol 2y ago
  You'd have to deploy it yourself, but there's this: https://github.com/aws-samples/bedrock-access-gateway https://github.com/aws-samples/bedrock-access-gateway
  - teruakohatu 2y ago
    Thanks. That is quite a heavy stack!
- popinman322 2y ago
  Try LiteLLM; their core LLM proxy is open source. As an added bonus it also supports other major providers.
OJFord 2y ago
Your 14 steps appear to be 'create an IAM user'..?
- Spivak 2y ago
  If you're already in the AWS ecosystem or have worked in it, it's no problem. If you're used to "make OpenAI account, add credit card, copy/paste API key" it can be a bit daunting.
- scosman 2y ago
  find to supported region, request model access, wait for model access, create policy, create user, attach policy... it's not comparable
- weitendorf 2y ago
  AWS does not use the exact same authn/authz/identity model or terminology as other providers, and for people familiar with other models, it's pretty non-trivial to adapt to. I recently posted a rant about this to https://www.reddit.com/r/aws/comments/1geczoz/the_aws_iam_identity_center_is_decadent_and/ https://www.reddit.com/r/aws/comments/1geczoz/the_aws_iam_id... Personally I am more familiar with directly using API keys or auth tokens than AWS's IAM users (which are more similar to what I'd call "service accounts").
  - OJFord 2y ago
    If you're looking for a generative AI model API only, I think Nova is not for you. If you want to build that capability into your cloud application, it uses exactly the model you expect and have, and you just add a new policy/role/whatever for whatever piece of it's going to use Nova.
SaggyDoomSr 2y ago
Nice! FWIW, The only nova model I see on the HuggingFace user space page is us.amazon.nova-pro-v1:0. I cloned the repo and added the other nova options in my clone, but you might want to add them to yours. (I would do a PR, but... I'm lazy and it's a trivial PR :-)).
- ndr_ 2y ago
  OK! I only add what people are interested in, so noted with thanks - will do! :-)
tootie 2y ago
I'm so confused on the value prop of Bedrock. It's seems like it wants to be guardrails for implementing RAG with popular models but it's not the least but intuitive. Is it actually better than setting up a custom pipeline?
- ndr_ 2y ago
  The value I get is: 1) one platform, largely one API, several models, 2) includes Claude 3.5 "unlimited" pay-as-you-go, 3) part of our corporate infra (SSO, billing, ... corporate discussions are easier to have) I'm using none to very little of the functionality they have added recently: not interested in RAG, not interested in Guardrails. Just Claude access, basically.
fumeux_fume 2y ago
This appears to be a way to steal and harvest aws credentials. No one should be following any of these steps.
- ndr_ 2y ago
  Do you have any evidence for this accusation? This is a guide for the casual observer who wants to try things out, given that getting started with other AI platforms is so much more straightforward. It's all open source, with transparent hosting, catering to any remaining concerns someone interested in exactly that may have.
  - jdmg94 2y ago
    I work at Amazon, you don't need to follow this steps to use bedrock. You. Are. Shameless
  - placardloop 2y ago
    The most common way for an AWS account to be hacked, by far, is mishandling of AWS IAM user credentials. AWS has even gone so far as to provide multiple warnings in the AWS console that you should never create long-lived IAM user credentials unless you really need to do so and really know what you are doing (aka not a “casual observer who wants to try things out”). This blog post encourages you to do this known dangerous thing, instructs you to bypass these warnings, and then paste these credentials into an untrusted app that is made up of 1000+ lines of code. Yes, the 1000+ lines of code are available for a security audit, but let’s be real: the “casual observer who wants to try things out” is not going to actually review all (if any) of the code, and likely not even realize they should review it. I give kudos to you for wanting to be helpful, but the instructions in this blog (“do this dangerous thing, but trust me it’s okay, and then do this other dangerous thing, but trust me it’s okay”) is exactly what nefarious actors would ask of unsuspecting victims, too, and following such blog posts is a practice that should not be generally encouraged.
  - xendo 2y ago
    Sharing your IAM credentials is like sharing your password. Just don't do it, regardless of the intentions. Even if this one doesn't steal anything it creates a precedence that will let people think it's ok and make them easier targets in the future. Besides, bedrock already has a console, so what's the point of using your UI?
d4rkp4ttern 2y ago
Setting up Azure LLM access is a similar hellish process. I learned after several days that I had to look at the actual endpoint URL to determine how to set the “deployment name” and “version” etc.
metzpapa 2y ago
You can set up AWS in less than 5 minutes...

by ndr_ 2y ago

Setting up AWS so you can try it via Amazon Bedrock API is a hassle, so I made a step-by-step guide: https://ndurner.github.io/amazon-nova https://ndurner.github.io/amazon-nova. It's 14+ steps!

simonw 2y ago

Thank you!

teruakohatu 2y ago

Thanks for that. Are there any proxies that can communicate with bedrock and serve it via a OpenAI style api?

moduspol 2y ago
You'd have to deploy it yourself, but there's this: https://github.com/aws-samples/bedrock-access-gateway https://github.com/aws-samples/bedrock-access-gateway
- teruakohatu 2y ago
  Thanks. That is quite a heavy stack!
popinman322 2y ago
Try LiteLLM; their core LLM proxy is open source. As an added bonus it also supports other major providers.

OJFord 2y ago

Your 14 steps appear to be 'create an IAM user'..?

Spivak 2y ago
If you're already in the AWS ecosystem or have worked in it, it's no problem. If you're used to "make OpenAI account, add credit card, copy/paste API key" it can be a bit daunting.
scosman 2y ago
find to supported region, request model access, wait for model access, create policy, create user, attach policy... it's not comparable
weitendorf 2y ago
AWS does not use the exact same authn/authz/identity model or terminology as other providers, and for people familiar with other models, it's pretty non-trivial to adapt to. I recently posted a rant about this to https://www.reddit.com/r/aws/comments/1geczoz/the_aws_iam_identity_center_is_decadent_and/ https://www.reddit.com/r/aws/comments/1geczoz/the_aws_iam_id... Personally I am more familiar with directly using API keys or auth tokens than AWS's IAM users (which are more similar to what I'd call "service accounts").
- OJFord 2y ago
  If you're looking for a generative AI model API only, I think Nova is not for you. If you want to build that capability into your cloud application, it uses exactly the model you expect and have, and you just add a new policy/role/whatever for whatever piece of it's going to use Nova.

SaggyDoomSr 2y ago

Nice! FWIW, The only nova model I see on the HuggingFace user space page is us.amazon.nova-pro-v1:0. I cloned the repo and added the other nova options in my clone, but you might want to add them to yours. (I would do a PR, but... I'm lazy and it's a trivial PR :-)).

ndr_ 2y ago
OK! I only add what people are interested in, so noted with thanks - will do! :-)

tootie 2y ago

I'm so confused on the value prop of Bedrock. It's seems like it wants to be guardrails for implementing RAG with popular models but it's not the least but intuitive. Is it actually better than setting up a custom pipeline?

ndr_ 2y ago
The value I get is: 1) one platform, largely one API, several models, 2) includes Claude 3.5 "unlimited" pay-as-you-go, 3) part of our corporate infra (SSO, billing, ... corporate discussions are easier to have) I'm using none to very little of the functionality they have added recently: not interested in RAG, not interested in Guardrails. Just Claude access, basically.

fumeux_fume 2y ago

This appears to be a way to steal and harvest aws credentials. No one should be following any of these steps.

ndr_ 2y ago
Do you have any evidence for this accusation? This is a guide for the casual observer who wants to try things out, given that getting started with other AI platforms is so much more straightforward. It's all open source, with transparent hosting, catering to any remaining concerns someone interested in exactly that may have.
- jdmg94 2y ago
  I work at Amazon, you don't need to follow this steps to use bedrock. You. Are. Shameless
- placardloop 2y ago
  The most common way for an AWS account to be hacked, by far, is mishandling of AWS IAM user credentials. AWS has even gone so far as to provide multiple warnings in the AWS console that you should never create long-lived IAM user credentials unless you really need to do so and really know what you are doing (aka not a “casual observer who wants to try things out”). This blog post encourages you to do this known dangerous thing, instructs you to bypass these warnings, and then paste these credentials into an untrusted app that is made up of 1000+ lines of code. Yes, the 1000+ lines of code are available for a security audit, but let’s be real: the “casual observer who wants to try things out” is not going to actually review all (if any) of the code, and likely not even realize they should review it. I give kudos to you for wanting to be helpful, but the instructions in this blog (“do this dangerous thing, but trust me it’s okay, and then do this other dangerous thing, but trust me it’s okay”) is exactly what nefarious actors would ask of unsuspecting victims, too, and following such blog posts is a practice that should not be generally encouraged.
- xendo 2y ago
  Sharing your IAM credentials is like sharing your password. Just don't do it, regardless of the intentions. Even if this one doesn't steal anything it creates a precedence that will let people think it's ok and make them easier targets in the future. Besides, bedrock already has a console, so what's the point of using your UI?

d4rkp4ttern 2y ago

Setting up Azure LLM access is a similar hellish process. I learned after several days that I had to look at the actual endpoint URL to determine how to set the “deployment name” and “version” etc.

metzpapa 2y ago

You can set up AWS in less than 5 minutes...