12 ms·
Does anyone else finds AWS and other Amazon services overly complicated?
I think AWS and its signature system is making things more complicated than it should be, like this is a normal signing in process to API:
1. you request client credentials, which is normal.
2. construct request URL, normal.
3. add headers, eh, normal.
4. signature... fuck.
First you need to convert the URL you have from step 2, mash it with headers from step 3, add header keys to signed headers, then sum256 hash the payload and hex encode it.
Then you create a sign, add algorithm, request date time that is formatted with ISO8601 but all special characters stripped apart, add credential scopes, hash the canonical request you created at the first step.
Then, you calculate this abomination:
HMAC(HMAC(HMAC(HMAC("AWS4" + kSecret,"20150830"),"us-east-1"),"iam"),"aws4_request")
after that you calculate this:
signature = HexEncode(HMAC(derived signing key, string to sign))
after that you create an authorization header and add signature to it:
Authorization: AWS4-HMAC-SHA256 Credential=AKIAIHV6HIXXXXXXX/20201022/us-east-1/execute-api/aws4_request, SignedHeaders=host;user-agent;x-amz-access-token;x-amz-date, Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924aEXAMPLE
...I mean what the fuck? I can understand why people choose Azure over AWS for the sake of freaking simplicity just by looking at this sign and request process. It feels overly-complicated. Does anyone feels the same while working with this abomination?
- Mathildebuxton 4y ago
- gregjor 4y agoLots of APIs do HMAC signing, not really something I get worked up over. I use AWS and Azure. Each has their advantages and disadvantages. To answer if something seems "overly complicated" I would have to know, compared to what? AWS certainly has lots of complexity, but so does any services with so many features and moving parts and potentially catastrophic misconfigurations and security threats. Compared to managing servers in a rack I prefer AWS.
- xyzzy123 4y agoIt hardly matters in practice because nobody does this, they use the AWS SDK.
- tluyben2 4y agoBut there are libraries for this with wrappers; don’t think many people do this manually or are bothered by it.
- carabiner 4y agoI did AWS training at the Amazon offices in Seattle for data science. I was blown away by the configuration... I have recompiled linux kernels and configured iptables as a teenager, and this was an entire galaxy of more complexity. It took us 6 hours to the point where some of us had a Jupyter Notebook running. Many people didn't make it though.
- goodpoint 4y agoArtificial complexity to justify lock-in and artificial salaries.
- wombatpm 4y agoOracle DBA’s would like to have a word with you in the back alley
- themoonisachees 4y agoThe microsoft model of "if sysadmins are spending much more time debugging windows, then they put windows first on their CV and gradually management forgets linux exists"
- Traubenfuchs 4y agoObviously! Just like devops as a whole. Why do we have this current mess instead of just pushing stuff to heroku?
- zeroxp 4y agoBecause heroku has limitations.
- rippercushions 4y agoWere you doing things the hard way on purpose? It's literally one click to spin up a Jupyter notebook on GCP, arguably less if you use Colab (instant access to shared instances), and I'm sure AWS has a similar service. https://cloud.google.com/vertex-ai-workbench https://cloud.google.com/vertex-ai-workbench
- kureikain 4y agoI though its just me who find it overly complicated. With people who use AWS SDK its all abstracted out. But there is time I just want to send a damn `curl` to download a S3 file and yes, doing the dance in bash isn't easy. There is time I wrote a Lua plugin for openresty to fetch s3 and. I have to trial and error with lot of debugging. The ordering. the timestamp format...all of that...
- gw98 4y agoIt's even worse. We use presigned URLs (from the SDK) and S3 for file uploads. That's one fresh hell you don't want to get involved with. One of those seemed like a good idea at the time things...
- alserio 4y agoMay I ask you why? I use and have used both PUT presigned urls and POST signed policies for uploading file to S3 without too many problems, but your remark worries me a bit. Am I missing something?
- gw98 4y agoWe get our customers to upload to S3 directly so we don't have to handle their ingress traffic through our infra. Problems involve mostly when the client cocks up as it's near impossible to distinguish between broken files and good ones. This leads to the assumption that the clients have uploaded a file and they haven't.
- gw98 4y agoAh that's nothing. Simple problem from a high level: static web site on apex domain. What you should be able to do: Click click done. Upload files to S3. Point CNAME at AWS. What you have to do: Create an S3 bucket and stick the files in it. Create a zone in Route 53 and import your old zone file. Change your nameservers at the registrar. Wait a bit. Go to ACM in the correct region and create a cert. Tell it to add the DNS entries to R53. Wait a bit. Create a CloudFront distribution making sure that you get all the options right which is quite difficult. While that's deploying, copy the IAM policy it generates back to the S3 bucket. Jump back to R53 and add a new A record pointing at the cloudfront distribution alias. Cross fingers and start praying that the whole stack works. If it doesn't spend several hours working out which thing you forgot to click. What devops culture would have you do: Play with CloudFormation for 2 days writing oodles of mind numbing YAML and realise that you have to create the ACM cert and CF distribution in a different region to your bucket but that's impossible. Try and work around this with recommended StackSets but realise they are so horrible that they are unusable. Spend an hour googling. Download terraform because everyone is gushing over it. Spend several hours learning the above and writing it in HCL constantly tearing down and creating resources until the whole thing limps along. Eventually realise you need to share this is someone and the thing is stateful so pay Hashicorp for TF Cloud. After the PO is approved with takes 2 weeks, check it in to git, cheer loudly, hand it over to a colleague who has a different version of terraform and then discover about tfenv and terraform upgrades. I only do this because I'm paid by the hour. I'm not even a cloud or ops guy. I'm an electronics engineer who needs to eat. I eat well due to this mess but I know it's all so so so wrong.
- zoover2020 4y agoAre you aware of the AWS CDK? Doing that in Infra as Code shouldn't take two days IMO.
- gw98 4y agoYes. CDK is just another way of expressing the same things as terraform and cloudfront but using a framework which is opaque. Plus it's slow as fuck, buggy and difficult to debug when it goes wrong. Every attempt keeps trying to solve the same problems with a new abstraction but the problem is the underlying abstraction not the tools.
- lmz 4y agoUse the SDK? Think about what this protects against - it's fair to think you may not need that much protection, but it's hardly pointless.
- ejb999 4y agoThat is why they give you the SDK - if you choose to not use it, and unnecessarily make your work more difficult - thats on you. Most people don't build their signatures by themselves - they use the SDK provided.
- outime 4y agoNo, I use the SDK or the CLI tools available for everyone.
- neilv 4y agoYou probably want to be using one of the popular libraries to do that. AWS documentation isn't consistently good, and there are lots of decoys. So, if you wound up in some local minima that made it look like you had to implement the signing and such yourself, try a Web search that includes the name of your chosen programming language. (A long time ago, I had to implement the AWS client API from scratch, but that's because I was using a fringe language that didn't have such libraries.)
- adaml_623 4y agoI like your language of decoys and local minima. It's very easy to follow out of date documentation and find yourself wasting time in a cul-de-sac
- brunooliv 4y agoIf you don't use the SDK, how can you judge anything as being "overly complicated"? I mean, I don't know about you, but, last time I checked, signatures, certificates, security and all that stuff IS SUPPOSED to be super complicated because it's a subject with a very high inherent complexity in and of itself. The SDK exists and is well designed to precisely shield you from said complexity. If you deliberately choose not to use it or can't for some reasons then yes... The complexity will be laid bare
- Gigachad 4y agoHN users routinely try to do things the obtuse way and then complain when its hard. Throw in something about the SDK being spyware or not following the unix philosophy.
- asah 4y agoCurl is my preferred SDK.
- numbsafari 4y agoWell, here you go then… https://curl.se/docs/manpage.html#--aws-sigv4 https://curl.se/docs/manpage.html#--aws-sigv4
- deleted 4y ago[deleted]
- naasking 4y ago> I mean, I don't know about you, but, last time I checked, signatures, certificates, security and all that stuff IS SUPPOSED to be super complicated because it's a subject with a very high inherent complexity in and of itself. Actually, security is not supposed to be complicated. "Complicated" is the anti-thesis of "secure".
- BackBlast 4y agoHave you ever tried to import the AWS SDK into a front end client? It's huge. Last time I tried it, it added multi MB of JS to my SPA, so I could do a relatively "simple" call using it. Yuck. It did not tree shake cleanly with my build system and I eventually ended up just yanking AWS from the stack entirely.
- batmanturkey 4y ago
- benjaminwootton 4y agoIAM is the one for me. I can stumble through the basics, but trying to keep everything least privilege adds a layer of pain to everything relatively complex that I try to do.
- Simon_O_Rourke 4y agoAbsolutely, there's so many services and permissions that getting them to all work together happily can be complex. I work at a company where we have both AWS certified engineers and some very smart people in general, and there's been times when we spend ages trying to figure out why one particular service can't read data from an S3 bucket (or a particular folder in bucket).
- perryizgr8 4y agoCan confirm this was a intense 12 hours of work to get it working properly. Their example and explanation in the docs is also very vague and borderline misleading at points. Another example is Google cloud run vs elastic container service. Cloud run is dead easy to get up and running. ECS is a mess of confusing terminology and unnecessary complexity. I literally do not care what class of machine you will use to run my container. If I cared that much I would have used a proper EC2 instance.
- throwaway894345 4y agoHave you looked at ECS Fargate? That should manage the instances for you transparently. It’s been around for probably five years or so.
- cube00 4y agoI think I'd rather take the 12 hour hit and get it working on EC2 then paying the ongoing convenience fee charged for Fargate.
- acdha 4y agoUnless your time is free, you have to run a ton of containers to break even, especially because Fargate avoids paying for EC2 capacity which can’t fit a container. It’s certainly possible to save money but my experience has been that people save less than expected once they account for ops time.
- deleted 4y ago[deleted]
- KaiserPro 4y agoYeah this is normally why you'd use the SDK, because its just horrific when you're outside. However, having said that, compared to FAANG internal tools, its actually not that bad. It's at least vaguely consistent
- PartiallyTyped 4y ago> compared to FAANG internal tools, its actually not that bad It's because it is an "internal tool". All services are built on top of aws native.
- iLoveOncall 4y agoIt's the contrary. An internal service is really good and then it becomes an AWS service. I've even been in 2 teams that have tried to do that and one may succeed in a few years.
- PartiallyTyped 4y agoIn my team, all of our services are built on top of AWS.
- damacaner 4y agoI am trying to work with MWS, and AWS SDK of GoLang horribly fails when tried to use standalone (I only need the signer part), but oh my oh my. https://github.com/canercetin-randomguy/club-noira/blob/main/main.go https://github.com/canercetin-randomguy/club-noira/blob/main... This is only the auth&sign part. and it doesnt work. and it is already 150 LoC. god.
- iLoveOncall 4y ago> However, having said that, compared to FAANG internal tools, its actually not that bad. It's at least vaguely consistent Actually I find a lot of Amazon's internal tools to be a breeze to use compared to AWS. Basically all the painful Amazon-specific stuff is done for you so it makes everything very easy. Some stuff is bad obviously but what we use the most is pretty freaking good.
- mcqueenjordan 4y agoYou HMAC the region so that in case a region is compromised, other regions aren't as well. You HMAC the service so that in case a service is compromised, other services aren't as well, you HMAC the timestamp for obvious reasons (time bound the signature), the outer "aws4_request" HMAC, I'm sure there's a good reason for. Maybe just versioning? Not sure. Also: All of this is handled in the SDKs. Anyone implementing this themselves either isn't using the right libraries or has a very special use case.
- dividuum 4y agoSounds like OP is having a bit of a Chesterton's Fence moment. It's clearly complicated, but you describe why this is probably implemented the way it is. The scheme reminds me of Macaroons: https://blog.gtank.cc/macaroons-reading-list/ https://blog.gtank.cc/macaroons-reading-list/
- naasking 4y agoI'm sure there are "reasons" why the HMACs are layered, the question is, does using HMACs actually add useful security properties here, or is this layered HMAC really just a way to generate a cryptographically secure id? If the latter, then can't you just generate that directly rather than needing to gather all of the right information and HMAC it in just the right way?
- mcqueenjordan 4y agoSorry, how do you propose to transmit a signature over the wire such that if it were compromised, the blast radius is limited to only the called service within the called region within a finite time window?
- naasking 4y agoIs the signature even necessary? As I said, an opaque bearer token is considerably simpler. Generate as many as you need for whatever services you're running to limit the damage of a leak. Set server-side policies for expiration or whatever else.
- stevage 4y agoThis title could be improved to clarify it's the API signature process OP has an issue with.
- orf 4y agoYour complaint isn’t about AWS, it’s about the authentication scheme. I find it to be pretty neat, especially when it’s flexible enough to create signed URLs for any method and send them to third parties. We use that as a basis for service-to-service auth. It’s cool and flexible. But overall this complaint seems pretty shallow. You’d just use the client SDKs they publish, or if you want to really go off the beaten path you could just use the signing methods from those SDKs with your own request/response calls. And if you’re building your own SDK (why?), well there is a lot more complexity down the line once you get past authentication.
- omnibrain 4y ago> And if you’re building your own SDK (why?) There are more languages than C++, Go, Java, JavaScript, Kotlin, .NET, Node.js, PHP, Python, Ruby, Rust & Swift.
- orf 4y agoOf course there are. And there are also unofficial SDKs for many other languages. But if you’re building a feature to interact with AWS then ignoring the availability of SDKs is stupid. If you want to write your service in brainfuck then go for it, but don’t blame AWS for that decision.
- deleted 4y ago[deleted]
- md_shakib 4y agoWhen you're outside, it's really horrific, so you'd normally use the SDK. In spite of that, it's actually not all that bad compared to FAANG's internal tools. I think it's at least vaguely consistent
- jen20 4y agoNot remotely. Every aspect of request signing has a purpose, and I appreciate that security is actually a concern at AWS. Plus, signing is implemented as a library almost everywhere you can think of.
- damacaner 4y agoYeah, at least getting used to signings might come useful in the future.
- marginalia_nu 4y agoWell, if you bill by the resources used, you really have no incentive to run an operation where building clean and resource-effective applications is easy, or where migrating to another platform is painless.
- throwaway894345 4y agoI mean, Amazon does a pretty good job of helping you write reasonably resource-efficient applications, and I’ve migrated from AWS to GCP and there wasn’t any particular lock-in that one wouldn’t have experienced in moving between a cloud provider and on prem or vice versa. Moreover, as it pertains to this post, AWS provides dozens of high-quality SDKs so you never actually have to know about this stuff. I’ve been using AWS for a decade and I’ve never known about how requests are formed.
- arielcostas 4y agoMy main problem with AWS has always been the console UI being horrendous and inconsistent, the poor documentation and having far too many services. Also, AWS not having something like Azure's resource groups or GCP's projects and instead telling you to create accounts for different projects and environments, and using AWS Organizations or however they call it sounds like a huge PITA.
- alrlroipsp 4y agoJust use aws sdk.
- pluc 4y agoIt has to be complicated if you want to spawn an industry of experts.
- Lapsa 4y agoyes
- BrS96bVxXBLzf5B 4y agoEntered the thread for a different reason. No, I don't really find building the API request complicated. I mean, it is but they provide a python example that I was easily able to get working in lua without major issues. And you spend half a day wondering why it's returning forbidden until you get it right and then it's done. What is complicated is receiving the response, that different services sometimes operate in regions and sometimes operate globally (but have a us-east-1 endpoint) and that some return XML, some return JSON, some return a gzipped response and some not, and much of it I haven't found documentation for and have had to figure out how to wrap calls to different services to be able to interact with them in a sensible manner.
- rco8786 4y agoI find basically all of AWS to be a nightmare of complexity. Probably great if you're in devops or infrastructure or something, but the story of "write code and deploy it" is horrific.
- jsemrau 4y agoTerraform is a godsend in my opinion. It makes architecture as a service as simply as a python script.
- flybrand 4y agoGiven the abundance of fake reviews, search results being infiltrated w advertising, and many shipping/vending options (new/used/Prime etc) - simply buying a book from Amazon has become very complicated.
- mabbo 4y agoFirst, most users are using the SDK that Amazon provides. But second, most of this complexity is in the name of security. I have plenty of criticism of my former employer (Amazon), but I have never once felt that AWS didn't take security seriously. Use the SDK. Be happy that it will keep your calls very secure.
- cpach 4y agoMakes a lot of sense IMHO. Because of their size they are a very interesting target for cyber-criminals. If Amazon wouldn’t make security a top priority it would all tumble down pretty quickly.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- joshuanapoli 4y agoThe outline of the signing algorithm is fine, and the details serve obvious purposes. However the spec is very loose. Normalization when forming the canonical request actually varies subtly between AWS services. This is super frustrating. Of course, you’re already in a really weird place if you need to write your own signer.
- ninefathom 4y agoI don't necessarily think that AWS is overly complicated. What I do think is that the entire ecosystem of tools is very powerful, and often times finding the "easy button" for a given task (and there usually is one) is far more time-consuming than it should be. Many a time* I wished that Amazon would have a single easy-to-find documentation page listing common use cases and the de facto standard third-party tool (and there usually is one) to make a given case easier. Fun side note: as a security professional, I cannot even begin to tell you how many times I've heard app owners claim that "AWS handles security for us" or some variation thereof. *It's been a few years since I messed with AWS in depth; for all I know such a page may exist now.
- philliphaydon 4y agoNope. When I did windows RT development back when the surface RT first came out I wanted to use SQS with a windows store app I was building. Using the documentation from AWS website I implemented a mini SQS Library. This was with 6 months AWS exp at the time. So I don’t think it’s complicated especially compared to Azure or GC where the documentation is inferior.
- MarcoSanto 4y agoYes, I do, but that's probably because I am not the target audience of their product. My gut feeling is that the target customer are devops teams in scaleups
- tqwhite 4y agoYes it's too complicated. It's horrible. It's the Windows of cloud services, surpassed in obscurity and nastiness only by Microsoft's own Azure, which is a nightmare. In both cases, the UI and UX designers should be shot. I understand that they are supporting a lot of complicated things. It's a huge fail, unusable by anyone who hasn't dedicated their lives to learning.
- dan-robertson 4y agoWhether or not AWS is overly complicated, I don’t think this is a good example. Any decent api should have some kind of signatures which work roughly like: signature = hmac_sha256(secret, url+payload+time+etc) set_header(…, auth_header(signature)) And the point is to prove that you know the key without sending it to AWS / your logs, and to prove that the request came from someone with the key ;and not eg someone replaying a message from logs which was either old or modified). Most of the mess seems to be due to AWS wanting to limit the power of the secrets that they store, presumably in case they’re compromised. You can get a reasonably good idea of their architecture by looking at how you construct the derived secret – each inner layer will be more tightly controlled than the one around it.
- dennis_jeeves1 4y agoI think, hidden within this question is another question: are we getting paid enough for this kind of complexity? Complexity takes it's toll on the brain. I'll gladly jump through all hoops that a sdk/frameworks/platform want me to if I'm paid sufficiently.
- dawnerd 4y agoI’ve been using aws for years and still don’t get it. Anytime I need to find logs it takes way too long. Simple things they use their own jargon for instead of just making it straight forward. They just need a “simple” mode that cuts out all of the verbose stuff
- jjgreen 4y ago> their own jargon Halleluja, it's not a console, it's a sodding web-page
- antonvs 4y agoWhat does Azure’s security system for object storage look like? Because if it’s not doing something like this, then I have questions about their security. The reality is is that what AWS is doing here is all fully justified and normal, and that’s why SDKs exist. Like many developers, you’re being a bit clueless about security here. Instead of complaining, I recommend educating yourself - look into capability security and signed tokens in general, and you’ll understand what AWS is doing and it’ll make you a better developer.
- jasonhansel 4y agoLots of people here are saying "just use the SDK." But a good API shouldn't be tightly coupled to a particular client; it should be possible to use the API easily using ordinary REST calls. If such a signature mechanism is necessary for security, it should be vendor-neutral, so that it can be shared by other providers and built into general-purpose HTTP clients.
- taeric 4y agoThis is somewhat silly. First, it is perfectly straight forward how to do the signature stuff, such that if you really want to, you can. However, this is akin to asking why it is so hard to write a binary against stdlibs on the machine. It is hard because that isn't really an easy ask. And the solutions we made to make the problem tractable are all nicely encapsulated in the SDK tools that we use. Why skip out on them?
- fhfuewidxjhe 4y agohmac is meh. the real issue with aws is the marketing alienation. router? switch? proxy? rack? forget all the sane terms for your infrastructure. you will call your router a amazon purple capybara 5 and you will have to like it!
- nuker 4y ago> Does anyone feels the same while working with this abomination? Have you tried Boto3? Libraries are supposed to do this, not people.
- commandlinefan 4y agoHave you ever looked at how X.509 certificate signatures are calculated? It’s wild, but it’s also observably secure. The quadruple HMAC looks… weird, but it seems like something you could just code and test in a few minutes and then forget about.
- kokizzu2 4y agoyes
- DangitBobby 4y agoAWS is an absolute nightmare of complexity. Try out GCP. With minimal experience, you'll find you can actually accomplish things in the cloud console without consulting a single piece of outside documentation! It's wild. I like this article which echos some of my experience. https://nandovillalba.medium.com/why-i-think-gcp-is-better-than-aws-ea78f9975bda https://nandovillalba.medium.com/why-i-think-gcp-is-better-t...
- appleflaxen 4y agoAbsolutely yes.
- twodave 4y agoThere are way better examples than this, such as the checkbox on Elastic Beanstalk that makes you decide whether to give your application a public IP. This checkbox would default to false, and in most sensible configurations would cause Elastic Beanstalk to be unable to reach the application VM. The amount of similar things on AWS (options that literally make the thing unusable if set a certain way) is astronomical, and often for simple things the documentation is non-existent or requires you to custom-configure IAM via xml. Oof.
- MauroIksem 4y agoLots of Apis do this both Microsoft and Google's Apis require signed jwt tokens. I think this is the norm now.
- moltar 4y agoI’ve been developing for the web since 1998. I also held this belief until about 3 years ago. Then I actually tried to learn AWS and now I just can’t imagine going back to anything before. And I’ve used everything from basic shell scripts to ansible to Docker to K8s to Heroku and everything in between. In the end AWS is a bunch of legos that I love to snap together to create powerful, robust solutions. I encourage you to take a look at AWS CDK.
- tomcam 4y agoJust wait until you have to figure out your AWS billing