10 ms·

The good parts of AWS: a visual summary

325 points by hac96 7y ago

OJFord 7y ago
I really like this 'tl;dr, consider this, start with this rather than that until/unless you really need it' style, was left disappointed it didn't cover more services!
- VectorLock 7y ago
  I would have really loved to see this for some of the more obscure services in the AWS panoply. The ones listed here are pretty well understood.
- WaxProlix 7y ago
  It's also unfortunately completely incorrect.
- mth003 7y ago
  Agreed. Very much inclined to read the straight-to-the point material.
jbergknoff 7y ago
This appears pretty superficial and it gets many things wrong. > Choose between [DynamoDB] on-demand option (no capacity management) or provisioned option (cheaper). It's very, very easy for provisioned capacity to be more expensive than on-demand. You need to overprovision because, if you exceed the provisioned throughput, you will get throttled and your application will suffer. Scaling up and down is a slow, opaque process, and you're only allowed to do it something like 5 times per day. If your workload has any idle periods, you're just burning money. > Avoid using S3 for static web hosting (No HTTPS) I won't argue that S3 is ideal for static web hosting, but "avoid" is pretty strong and IMO not warranted. > Do not use AWS Lambda as a general EC2 host What does this mean? As general compute? This is too simplistic a judgment. Lambda is a versatile service and can be useful in many situations. > Kinesis: unlimited consumers No, Kinesis is an extremely limited service. 5 reads per second per shard. In my experience, if you put even two consumers on a stream, you will start to see throttling. The official solution is to fan out to multiple streams. Hacky and super expensive. > Kinesis 30x cheaper than SQS This is hilarious. Maybe it's true for some workloads. In my experience, Kinesis is incredibly expensive and SQS is not.
- supdatecron 7y ago
  > Avoid using S3 for static web hosting (No HTTPS) Ya, "avoid" is a strong word. I use S3 for tons of static web hosting, because it is exceptionally cheap and I don't have to worry about patching CentOS/Apache/MySQL/PHP and all the other binaries/daemons that I used to. I know how to, I just would rather not when S3 is a couple pence per GB.
  - nerdjon 7y ago
    That part stuck out to me as well, especially since the lack of https was their only reason. You can stick Cloudfront in front of it and get HTTPS very easily (with the added benefit of being on edge servers). Personally I have found S3+Cloudfront to be perfect for static content for that reason.
- unethical_ban 7y ago
  > Do not use AWS Lambda as a general EC2 host. It seems like they're saying "Don't use this for generic services, only use it as an integration with other AWS services" to which I would say, "What?" If I recall, https://acloud.guru https://acloud.guru for example is hosted entirely using Lambda, API GW, S3 and some other AWS services.
- pensatoio 7y ago
  THIS! I too was taken aback by the comparison of SQS and Kinesis. In my experience kinesis is significantly more expensive and significantly harder to scale. The only reason you should use it is if you need the throughput and delivery guarantees.
- calderarrow 7y ago
  What would you recommend for site hosting instead of S3?
  - xenophonf 7y ago
    I'm not sure where the article author is going with this, but the official way to host a static web site on S3 with HTTPS is to use ACM and CloudFront: https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-https-requests-s3/ https://aws.amazon.com/premiumsupport/knowledge-center/cloud...
sk5t 7y ago
Eh. The ALB/ELB/NLB section is off the mark for failing to consider inter-AZ costs, perplexingly indicates NLB has something to do with TLS, even more perplexingly suggests NLB as the default choice. Dynamo isn't something I'd suggest at all unless one has well studied the costs and benefits. Kinesis is not much like SQS, and new users might be more interested in EventBridge...
- dastbe 7y ago
  Conflating TLS and HTTPS makes that row weird, but NLB supports TLS termination at the load balancer. https://aws.amazon.com/blogs/aws/new-tls-termination-for-network-load-balancers/ https://aws.amazon.com/blogs/aws/new-tls-termination-for-net...
- cthalupa 7y ago
  >Kinesis is not much like SQS, and new users might be more interested in EventBridge... As someone who has done a lot of work with AWS, I have at times in the past picked the wrong one of Kinesis vs SQS for different workloads - largely due to my inexperience with Kafka and similar at the time, and misunderstanding the point of some of the "benefits" that I saw for using Kinesis vs SQS. Most of those times, I would have been better suited to a combo SQS/SNS for fanning out (something a helpful HN reader actually suggested to me!). Anyway, EventBridge steps in perfectly for most of those use cases, my new projects are using it, and I am considering going back to some old services and reworking them to use EventBridge instead. The rules/filtering parts in particular are going to let me decom several lambdas and save money. Definitely worth people investigating.
technics256 7y ago
I liked the book, but it fell apart when it got to cloudformation. It hand waved a ton of the actual IaaC part, so instead you end up copying from book, which had some differences from the git repo they shared. Good to get the basics of what to do and best practices for architecting, but incredibly frustrating on actually following and building with cloudformation.
- DVassallo 7y ago
  We had a couple of frustrating typos in the first version. Very sorry about that. We fixed them now, and the latest book version is in sync with the GitHub repo.
markphip 7y ago
Not sure where they came up with that SQS pricing? It is free for first million requests per month. I have been using SQS as a message queue in my app and based on AWS Cost Explorer I have never reached that in a month .. always a $0 charge. Looking at Cloudwatch I never even got to 100K requests in a month and the app is moderately active. It does not generate a ton of messages, maybe hundreds a day and the workers receiving the messages use long poll. Anyway, the costs in the doc seem way off.
zemo 7y ago
> DynamoDB is a non-relational database that has two main features : it’s immediately consistent and ... that's ... only true if you're exclusively using strongly consistent reads, but if you're exclusively using strongly consistent reads, why are you using DynamoDB to begin with?
Scarbutt 7y ago
What are EC2 "saving plans"?
- WrtCdEvrydy 7y ago
  A way to save some cash, you set aside your reserve amount and some EC2 magic makes it happen.
  - deleted 7y ago
    [deleted]
- zokier 7y ago
  You borrow money to AWS and get a discount for their services.
- NikolaeVarius 7y ago
  https://lmgtfy.com/?q=+EC2+saving+plans https://lmgtfy.com/?q=+EC2+saving+plans
Scarbutt 7y ago
I have seen some projects use DynamoDB for relational data, with the "single table design" approach, anyone who did this, how was your experience?
- kbanman 7y ago
  I've seen this done for new projects, and it works really well. If your data access patterns are truly relational (varied lookup paths) then it is probably not the right tool, but many apps can be modeled in a way that DDB handles well. Highly recommended viewing: https://www.youtube.com/watch?v=HaEPXoXVf2k https://www.youtube.com/watch?v=HaEPXoXVf2k this talk explains how relational data can be efficiently modeled for key-value stores.
  - deleted 7y ago
    [deleted]
deanCommie 7y ago
The SQS vs Kinesis summary is particularly bad Kinesis has strict ordering but SQS does not? No. Kinesis has strict ordering PER SHARD. Which dilutes your throughput, unlike SQS. And it has limitations on throughput in terms of total event size, which are different from SQS's 300 TPS for FIFO. It's apples and oranges. More significant, this comparison really makes it seem like you should pick Kinesis for a simple starting project rather than SQS, which is the exact opposite of the truth. SQS is the simpler choice for 90% of the cases, and Kinesis is an advanced powerful tool for the other 10%
- naragon 7y ago
  Agreed, this has been my experience as well.
- mooreds 7y ago
  Right, and also SQS can have more than one consumer; I daresay N consumers is the most common scenario.
- snorkel 7y ago
  Also Kinesis records are write-once read-many, where SQS messages are meant to be consumed (receive a message, do the task, delete the message) Kinesis is for streaming data, and SQS is for job queues.
bowmessage 7y ago
Touching only the surface of some AWS services and calling them "the good parts" is really misleading. This does not run the gamut of all AWS services available, skips over a number of other popular databases and developer productivity services, and makes broad generalizations as another commenter points out...
ggregoire 7y ago
> AVOID using S3 for static web hosting (No HTTPS). I would absolutely recommend S3 for static web hosting. Just add CloudFront on top if you need HTTPS, it takes like 2 clicks. If anyone interested, I answered on StackOverflow how to deploy a front end app (React, Vue, etc) to S3: https://stackoverflow.com/q/41250087/652669 https://stackoverflow.com/q/41250087/652669
- ddoolin 7y ago
  We run almost all of our UIs on S3 static hosting behind CloudFront. Works like a champ. I'm a bit stumped on why they say to avoid using local indices in DynamoDB, though.
  - DVassallo 7y ago
    The entire chapter on DynanoDB is free in the sample. The answer to your question is there: https://gumroad.com/l/aws-good-parts https://gumroad.com/l/aws-good-parts
  - harrisonjackson 7y ago
    In case you don't feel like clicking through to gumroad and opening the sample PDF... https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/LSI.html#LSI.Creating https://docs.amazonaws.cn/en_us/amazondynamodb/latest/develo... Here is a link to the AWS docs referencing the limitation of local secondary indices. Any table with a local secondary index is limited to 10GB per partition key. The authors may have misinterpreted this limitation. >Once you create a local index on a table, the property that allows a table to keep growing indefinitely goes away. It limits the value for that partition key but does not limit the overall size of the table...
- tobilg 7y ago
  You can have a look at https://github.com/tobilg/serverless-aws-static-websites https://github.com/tobilg/serverless-aws-static-websites if you want to set this up with one command
  - 7y ago
chickenpotpie 7y ago
Does anyone else get the feeling that the author has not used any of these services before and is just writing this from reading AWS documentation and pricing?
- DVassallo 7y ago
  It’s a summary of a book.
  - jessaustin 7y ago
    So, was the book written that way?
mmckelvy 7y ago
I'd agree with a few of the other comments here. I read it and can't say that I found it particularly helpful. I've deployed a lot of Node.js apps to Heroku and Digital Ocean, and was looking for something to help me "level up" to AWS. I was left thinking that maybe I should just stick to Heroku.
- bowmessage 7y ago
  Beanstalk & CodeDeploy & X-Ray if you need to level up your deployment & webapp management.
deleted 7y ago
[deleted]
Androider 7y ago
Can't agree on the NLB part. Everyone should be using ALB by default, and only reach for NLB in very specific cases (you'll know when), in my humble opinion having worked with AWS for 8 years and being a certified architect. Enable the dropping of invalid headers, enable HTTP/2, let the ALB terminate the TLS, and you'll see benefits even if your full backend isn't HTTP/2 enabled and you'll have eliminated a whole range of other headaches you no longer need to manage. It's one of the most reliable services AWS has.
- DVassallo 7y ago
  Book co-author here. (Not article author.) The recommendation is based on the multi-tenant behavior of NLBs (no need to warm up the LB to handle traffic spikes). If you need any ALB features, use ALB. Otherwise, NLBs give you one less thing to worry about (and slightly cheaper and faster too.)
enahs-sf 7y ago
I would strongly advocate that Aurora RDS be added here. Easy to manage, SQL data storage. Not cheap, but cheaper than a DBA for sure.
Raphomet 7y ago
Anyone know what tool the author used to make those nice animated SVGs?
- mrlatinos 7y ago
  I found this on his Github: https://github.com/hassenc/svgAnimator https://github.com/hassenc/svgAnimator
- mkw5053 7y ago
  I'm wondering the same thing!
- ANaimi 7y ago
  Had the same question. Apparently he created them manually! https://twitter.com/hassenchaieb/status/1240346726822752262?s=20 https://twitter.com/hassenchaieb/status/1240346726822752262?...
bob1029 7y ago
No mention of Route53? Either I am missing out on something big or the author is. Route53 is probably the biggest value-add on top of raw compute & storage that exists in Amazon's entire cloud product portfolio. Who else remembers requiring 3 different vendors to run 1 https website?
whalesalad 7y ago
There is a tremendous amount of incorrect and misleading information on this post. The first sentence re: Dynamo discusses the inability to do filtering or sorting which is flat out wrong: “DynamoDB requires data operations (aggregations, filtering, sorting ..) to be done by your application. → All data needs to be sent over the network.”
- redisman 7y ago
  Sorting - really? Isn't the first thing to learn about Dynamo the primary key and the sort key? Filtering you can do on the database but you still pay for the reads as if it wasn't filtered. To me Dynamos biggest con is that it's propriety technology and aggregations it's just not really any good for.
pojzon 7y ago
There are so many plain wrong points here I simply dont know where to start. It feels like the person who wrote it has no real commercial experience or very little experience with described services.
corp_muppet 7y ago
This opinion piece has too many inaccurate statements. - S3 for static assests - SQS V Kinesis - Lambda ('Small code that doesn't change....what). - ELB completely overlooked, if you need session persistence then you need an ELB so for more traditional software you've completely omitted that solution because it's "Legacy". Missing - No discussion on when to pick Lambda, Containers or EC2 - No Cloudfront addresses S3 and any API GW or ALB fronted Lambda. - Where is the use of SNS? - What if I want to Fan Out on a message/event? complete missed the why SQS V SNS V Kinesis and when to use which. - No RDS....so if i've got relational data are you advising i should put it in DynamoDB? - No container context in any way. Really poor quality of content and advisories needs an urgent update.
- SideburnsOfDoom 7y ago
  > Where is the use of SNS? Right, the SQS part is very misleading: "SQS has 1 consumer" is technically true (although it's actually one group of competing consumers), but in practice, you get fan-out to zero or more consumers easily, by wiring SNS and SQS together (one SNS topic, multiple attached SQS queues). > No RDS... Without also mentioning RDS with DynamoDB, you can't make meaningful choices between them. It also omits the one big benefit of DynamoDB: single-digit number of milliseconds to read a record by primary key, regardless of scale, due to sharding on that key.
- intsunny 7y ago
  > - ELB completely overlooked, if you need session persistence then you need an ELB so for more traditional software you've completely omitted that solution because it's "Legacy". Additionally the legacy ELB is great for when you want to do you own TLS/SSL termination with Lets Encrypt. (Especially with kubernetes, ingress, and cert-manager.) Also with the legacy feature, you can enable ProxyProtcol which passes on the public IP info after the NAT.
rb808 7y ago
Its nice to have some proper opinions and recommendations. Seems the tech world is so full of articles and blogs that show you how to do something, with very little saying don't use this because...
reilly3000 7y ago
This article is wildly inaccurate. My corrections: - DynamoDb allows for a lot of filtering, and Lambda + DDB Streams can do aggregations that are written as state changes and provide incredible read speeds. Use it if its a good fit, and its only a good fit if you're ready to learn its query language and quirks. When it is a good fit, its a world-class database. - SQS pricing is $0.0000004 per request [1] with 1 Million monthly messages free. - SQS + SNS is far more flexible and reliable than Kinesis, plus you're paying for and managing shards by the hour. Its great for streaming analytics data processing and distribution (within the Kinesis suite there is stream processing ala Spark and streaming ingest to Elastic, S3, and Redshift). If you're not doing data engineering on a streaming set of events that can't fit in a spreadsheet, its just not the right fit. - Lambda is not expensive to use with DynamoDB at all, the pipes are so fast that that 50-100ms is typical for a call and response. Lambda does get gnarly with SQL databases that require connection management and pooling, but they have improved that situation on multiple fronts in the past 6-9 months. - Also, Lambda ought to be considered the first choice for runtimes for greenfield apps, not just for AWS plumbing. This is strictly opinion and definitely depends on your team and existing codebase, but it makes sense from a cost and scaling standpoint for all kinds of projects. - NLB should be the first choice only for specialized use cases like UDP services or those that require extraordinary throughput. ALB gives you an incredible feature set with things like HTTPS, Auth, access logging, routing requests to different machines or Lambdas based on url path, canary deployments, and more. Use as much or little as you like. The cost difference between ALB and NLB is negligible for most use cases. If you're new to AWS and cloud stuff: welcome, the water's fine. Sure its hard at first to grok all the breadth and depth of the offerings, but you get to build with the same tools the largest services on the web depend on. Start small and pick a service, learn it from the console, then cli. Hit up the docs a lot and ask questions, there are lots of people that can help. Do NOT get lead astray by this article or opinion pieces on Medium. Something like 75% of AWS services are geared at serving enterprises with on-premise data centers that want to move some or all of their workloads to the cloud. Stick with the basics like S3, Lambda, SNS, SQS, and whatever servers you need. Pick a managed service vs servers when you can unless you have solid sysadmin resources and a good reason. Avoid hourly billed services whenever you can until you can get cost management figured out. VPC is daunting unless you're good with networking, but you can avoid it entirely by using the primitives mentioned above. IAM takes some real grokking, but there is almost nothing you can't do with it to scope permissions both internally and externally. AWS YouTube Channel is full of quality material for all levels. Try to start forming your opinions with those and your own experience instead of unsubstantial articles or random HN comments :)
reese_john 7y ago
Why does it say that SQS has duplicates, while Kinesis does not ? AWS documentation explicitly says that Kinesis has at least once semantics due to consumer/producer retries.
fraserharris 7y ago
A lesser known limitation of S3 is that it does not have Read-After-Write consistency for overwriting PUTS & DELETE operations. Netflix has reported observing consistency taking hours for outliers. https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction...
- pojzon 7y ago
  Because its a distributed system and it takes time to propagate ?? I think its very well known, ppl just understand its normal and in 99.999% of cases does not matter.
skywhopper 7y ago
I really like the summary format and the visual design here. It feels slightly weird to specifically call out Redis in a negative way in the Dynamo summary when no other tools listed give a similar callout to a specific product.

325 points by hac96 7y ago

I really like this 'tl;dr, consider this, start with this rather than that until/unless you really need it' style, was left disappointed it didn't cover more services!

VectorLock 7y ago
I would have really loved to see this for some of the more obscure services in the AWS panoply. The ones listed here are pretty well understood.
WaxProlix 7y ago
It's also unfortunately completely incorrect.
mth003 7y ago
Agreed. Very much inclined to read the straight-to-the point material.

jbergknoff 7y ago

This appears pretty superficial and it gets many things wrong. > Choose between [DynamoDB] on-demand option (no capacity management) or provisioned option (cheaper). It's very, very easy for provisioned capacity to be more expensive than on-demand. You need to overprovision because, if you exceed the provisioned throughput, you will get throttled and your application will suffer. Scaling up and down is a slow, opaque process, and you're only allowed to do it something like 5 times per day. If your workload has any idle periods, you're just burning money. > Avoid using S3 for static web hosting (No HTTPS) I won't argue that S3 is ideal for static web hosting, but "avoid" is pretty strong and IMO not warranted. > Do not use AWS Lambda as a general EC2 host What does this mean? As general compute? This is too simplistic a judgment. Lambda is a versatile service and can be useful in many situations. > Kinesis: unlimited consumers No, Kinesis is an extremely limited service. 5 reads per second per shard. In my experience, if you put even two consumers on a stream, you will start to see throttling. The official solution is to fan out to multiple streams. Hacky and super expensive. > Kinesis 30x cheaper than SQS This is hilarious. Maybe it's true for some workloads. In my experience, Kinesis is incredibly expensive and SQS is not.

supdatecron 7y ago
> Avoid using S3 for static web hosting (No HTTPS) Ya, "avoid" is a strong word. I use S3 for tons of static web hosting, because it is exceptionally cheap and I don't have to worry about patching CentOS/Apache/MySQL/PHP and all the other binaries/daemons that I used to. I know how to, I just would rather not when S3 is a couple pence per GB.
- nerdjon 7y ago
  That part stuck out to me as well, especially since the lack of https was their only reason. You can stick Cloudfront in front of it and get HTTPS very easily (with the added benefit of being on edge servers). Personally I have found S3+Cloudfront to be perfect for static content for that reason.
unethical_ban 7y ago
> Do not use AWS Lambda as a general EC2 host. It seems like they're saying "Don't use this for generic services, only use it as an integration with other AWS services" to which I would say, "What?" If I recall, https://acloud.guru https://acloud.guru for example is hosted entirely using Lambda, API GW, S3 and some other AWS services.
pensatoio 7y ago
THIS! I too was taken aback by the comparison of SQS and Kinesis. In my experience kinesis is significantly more expensive and significantly harder to scale. The only reason you should use it is if you need the throughput and delivery guarantees.
calderarrow 7y ago
What would you recommend for site hosting instead of S3?
- xenophonf 7y ago
  I'm not sure where the article author is going with this, but the official way to host a static web site on S3 with HTTPS is to use ACM and CloudFront: https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-https-requests-s3/ https://aws.amazon.com/premiumsupport/knowledge-center/cloud...

sk5t 7y ago

Eh. The ALB/ELB/NLB section is off the mark for failing to consider inter-AZ costs, perplexingly indicates NLB has something to do with TLS, even more perplexingly suggests NLB as the default choice. Dynamo isn't something I'd suggest at all unless one has well studied the costs and benefits. Kinesis is not much like SQS, and new users might be more interested in EventBridge...

dastbe 7y ago
Conflating TLS and HTTPS makes that row weird, but NLB supports TLS termination at the load balancer. https://aws.amazon.com/blogs/aws/new-tls-termination-for-network-load-balancers/ https://aws.amazon.com/blogs/aws/new-tls-termination-for-net...
cthalupa 7y ago
>Kinesis is not much like SQS, and new users might be more interested in EventBridge... As someone who has done a lot of work with AWS, I have at times in the past picked the wrong one of Kinesis vs SQS for different workloads - largely due to my inexperience with Kafka and similar at the time, and misunderstanding the point of some of the "benefits" that I saw for using Kinesis vs SQS. Most of those times, I would have been better suited to a combo SQS/SNS for fanning out (something a helpful HN reader actually suggested to me!). Anyway, EventBridge steps in perfectly for most of those use cases, my new projects are using it, and I am considering going back to some old services and reworking them to use EventBridge instead. The rules/filtering parts in particular are going to let me decom several lambdas and save money. Definitely worth people investigating.

technics256 7y ago

I liked the book, but it fell apart when it got to cloudformation. It hand waved a ton of the actual IaaC part, so instead you end up copying from book, which had some differences from the git repo they shared. Good to get the basics of what to do and best practices for architecting, but incredibly frustrating on actually following and building with cloudformation.

DVassallo 7y ago
We had a couple of frustrating typos in the first version. Very sorry about that. We fixed them now, and the latest book version is in sync with the GitHub repo.

markphip 7y ago

Not sure where they came up with that SQS pricing? It is free for first million requests per month. I have been using SQS as a message queue in my app and based on AWS Cost Explorer I have never reached that in a month .. always a $0 charge. Looking at Cloudwatch I never even got to 100K requests in a month and the app is moderately active. It does not generate a ton of messages, maybe hundreds a day and the workers receiving the messages use long poll. Anyway, the costs in the doc seem way off.

zemo 7y ago

> DynamoDB is a non-relational database that has two main features : it’s immediately consistent and ... that's ... only true if you're exclusively using strongly consistent reads, but if you're exclusively using strongly consistent reads, why are you using DynamoDB to begin with?

Scarbutt 7y ago

What are EC2 "saving plans"?

WrtCdEvrydy 7y ago
A way to save some cash, you set aside your reserve amount and some EC2 magic makes it happen.
- deleted 7y ago
  [deleted]
zokier 7y ago
You borrow money to AWS and get a discount for their services.
NikolaeVarius 7y ago
https://lmgtfy.com/?q=+EC2+saving+plans https://lmgtfy.com/?q=+EC2+saving+plans

Scarbutt 7y ago

I have seen some projects use DynamoDB for relational data, with the "single table design" approach, anyone who did this, how was your experience?

kbanman 7y ago
I've seen this done for new projects, and it works really well. If your data access patterns are truly relational (varied lookup paths) then it is probably not the right tool, but many apps can be modeled in a way that DDB handles well. Highly recommended viewing: https://www.youtube.com/watch?v=HaEPXoXVf2k https://www.youtube.com/watch?v=HaEPXoXVf2k this talk explains how relational data can be efficiently modeled for key-value stores.
- deleted 7y ago
  [deleted]

deanCommie 7y ago

The SQS vs Kinesis summary is particularly bad Kinesis has strict ordering but SQS does not? No. Kinesis has strict ordering PER SHARD. Which dilutes your throughput, unlike SQS. And it has limitations on throughput in terms of total event size, which are different from SQS's 300 TPS for FIFO. It's apples and oranges. More significant, this comparison really makes it seem like you should pick Kinesis for a simple starting project rather than SQS, which is the exact opposite of the truth. SQS is the simpler choice for 90% of the cases, and Kinesis is an advanced powerful tool for the other 10%

naragon 7y ago
Agreed, this has been my experience as well.
mooreds 7y ago
Right, and also SQS can have more than one consumer; I daresay N consumers is the most common scenario.
snorkel 7y ago
Also Kinesis records are write-once read-many, where SQS messages are meant to be consumed (receive a message, do the task, delete the message) Kinesis is for streaming data, and SQS is for job queues.

bowmessage 7y ago

Touching only the surface of some AWS services and calling them "the good parts" is really misleading. This does not run the gamut of all AWS services available, skips over a number of other popular databases and developer productivity services, and makes broad generalizations as another commenter points out...

ggregoire 7y ago

> AVOID using S3 for static web hosting (No HTTPS). I would absolutely recommend S3 for static web hosting. Just add CloudFront on top if you need HTTPS, it takes like 2 clicks. If anyone interested, I answered on StackOverflow how to deploy a front end app (React, Vue, etc) to S3: https://stackoverflow.com/q/41250087/652669 https://stackoverflow.com/q/41250087/652669

ddoolin 7y ago
We run almost all of our UIs on S3 static hosting behind CloudFront. Works like a champ. I'm a bit stumped on why they say to avoid using local indices in DynamoDB, though.
- DVassallo 7y ago
  The entire chapter on DynanoDB is free in the sample. The answer to your question is there: https://gumroad.com/l/aws-good-parts https://gumroad.com/l/aws-good-parts
- harrisonjackson 7y ago
  In case you don't feel like clicking through to gumroad and opening the sample PDF... https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/LSI.html#LSI.Creating https://docs.amazonaws.cn/en_us/amazondynamodb/latest/develo... Here is a link to the AWS docs referencing the limitation of local secondary indices. Any table with a local secondary index is limited to 10GB per partition key. The authors may have misinterpreted this limitation. >Once you create a local index on a table, the property that allows a table to keep growing indefinitely goes away. It limits the value for that partition key but does not limit the overall size of the table...
tobilg 7y ago
You can have a look at https://github.com/tobilg/serverless-aws-static-websites https://github.com/tobilg/serverless-aws-static-websites if you want to set this up with one command
- 7y ago

chickenpotpie 7y ago

Does anyone else get the feeling that the author has not used any of these services before and is just writing this from reading AWS documentation and pricing?

DVassallo 7y ago
It’s a summary of a book.
- jessaustin 7y ago
  So, was the book written that way?

mmckelvy 7y ago

I'd agree with a few of the other comments here. I read it and can't say that I found it particularly helpful. I've deployed a lot of Node.js apps to Heroku and Digital Ocean, and was looking for something to help me "level up" to AWS. I was left thinking that maybe I should just stick to Heroku.

bowmessage 7y ago
Beanstalk & CodeDeploy & X-Ray if you need to level up your deployment & webapp management.

deleted 7y ago

[deleted]

Androider 7y ago

Can't agree on the NLB part. Everyone should be using ALB by default, and only reach for NLB in very specific cases (you'll know when), in my humble opinion having worked with AWS for 8 years and being a certified architect. Enable the dropping of invalid headers, enable HTTP/2, let the ALB terminate the TLS, and you'll see benefits even if your full backend isn't HTTP/2 enabled and you'll have eliminated a whole range of other headaches you no longer need to manage. It's one of the most reliable services AWS has.

DVassallo 7y ago
Book co-author here. (Not article author.) The recommendation is based on the multi-tenant behavior of NLBs (no need to warm up the LB to handle traffic spikes). If you need any ALB features, use ALB. Otherwise, NLBs give you one less thing to worry about (and slightly cheaper and faster too.)

enahs-sf 7y ago

I would strongly advocate that Aurora RDS be added here. Easy to manage, SQL data storage. Not cheap, but cheaper than a DBA for sure.

Raphomet 7y ago

Anyone know what tool the author used to make those nice animated SVGs?

mrlatinos 7y ago
I found this on his Github: https://github.com/hassenc/svgAnimator https://github.com/hassenc/svgAnimator
mkw5053 7y ago
I'm wondering the same thing!
ANaimi 7y ago
Had the same question. Apparently he created them manually! https://twitter.com/hassenchaieb/status/1240346726822752262?s=20 https://twitter.com/hassenchaieb/status/1240346726822752262?...

bob1029 7y ago

No mention of Route53? Either I am missing out on something big or the author is. Route53 is probably the biggest value-add on top of raw compute & storage that exists in Amazon's entire cloud product portfolio. Who else remembers requiring 3 different vendors to run 1 https website?

whalesalad 7y ago

There is a tremendous amount of incorrect and misleading information on this post. The first sentence re: Dynamo discusses the inability to do filtering or sorting which is flat out wrong: “DynamoDB requires data operations (aggregations, filtering, sorting ..) to be done by your application. → All data needs to be sent over the network.”

redisman 7y ago
Sorting - really? Isn't the first thing to learn about Dynamo the primary key and the sort key? Filtering you can do on the database but you still pay for the reads as if it wasn't filtered. To me Dynamos biggest con is that it's propriety technology and aggregations it's just not really any good for.

pojzon 7y ago

There are so many plain wrong points here I simply dont know where to start. It feels like the person who wrote it has no real commercial experience or very little experience with described services.

corp_muppet 7y ago

This opinion piece has too many inaccurate statements. - S3 for static assests - SQS V Kinesis - Lambda ('Small code that doesn't change....what). - ELB completely overlooked, if you need session persistence then you need an ELB so for more traditional software you've completely omitted that solution because it's "Legacy". Missing - No discussion on when to pick Lambda, Containers or EC2 - No Cloudfront addresses S3 and any API GW or ALB fronted Lambda. - Where is the use of SNS? - What if I want to Fan Out on a message/event? complete missed the why SQS V SNS V Kinesis and when to use which. - No RDS....so if i've got relational data are you advising i should put it in DynamoDB? - No container context in any way. Really poor quality of content and advisories needs an urgent update.

SideburnsOfDoom 7y ago
> Where is the use of SNS? Right, the SQS part is very misleading: "SQS has 1 consumer" is technically true (although it's actually one group of competing consumers), but in practice, you get fan-out to zero or more consumers easily, by wiring SNS and SQS together (one SNS topic, multiple attached SQS queues). > No RDS... Without also mentioning RDS with DynamoDB, you can't make meaningful choices between them. It also omits the one big benefit of DynamoDB: single-digit number of milliseconds to read a record by primary key, regardless of scale, due to sharding on that key.
intsunny 7y ago
> - ELB completely overlooked, if you need session persistence then you need an ELB so for more traditional software you've completely omitted that solution because it's "Legacy". Additionally the legacy ELB is great for when you want to do you own TLS/SSL termination with Lets Encrypt. (Especially with kubernetes, ingress, and cert-manager.) Also with the legacy feature, you can enable ProxyProtcol which passes on the public IP info after the NAT.

rb808 7y ago

Its nice to have some proper opinions and recommendations. Seems the tech world is so full of articles and blogs that show you how to do something, with very little saying don't use this because...

reilly3000 7y ago

This article is wildly inaccurate. My corrections: - DynamoDb allows for a lot of filtering, and Lambda + DDB Streams can do aggregations that are written as state changes and provide incredible read speeds. Use it if its a good fit, and its only a good fit if you're ready to learn its query language and quirks. When it is a good fit, its a world-class database. - SQS pricing is $0.0000004 per request [1] with 1 Million monthly messages free. - SQS + SNS is far more flexible and reliable than Kinesis, plus you're paying for and managing shards by the hour. Its great for streaming analytics data processing and distribution (within the Kinesis suite there is stream processing ala Spark and streaming ingest to Elastic, S3, and Redshift). If you're not doing data engineering on a streaming set of events that can't fit in a spreadsheet, its just not the right fit. - Lambda is not expensive to use with DynamoDB at all, the pipes are so fast that that 50-100ms is typical for a call and response. Lambda does get gnarly with SQL databases that require connection management and pooling, but they have improved that situation on multiple fronts in the past 6-9 months. - Also, Lambda ought to be considered the first choice for runtimes for greenfield apps, not just for AWS plumbing. This is strictly opinion and definitely depends on your team and existing codebase, but it makes sense from a cost and scaling standpoint for all kinds of projects. - NLB should be the first choice only for specialized use cases like UDP services or those that require extraordinary throughput. ALB gives you an incredible feature set with things like HTTPS, Auth, access logging, routing requests to different machines or Lambdas based on url path, canary deployments, and more. Use as much or little as you like. The cost difference between ALB and NLB is negligible for most use cases. If you're new to AWS and cloud stuff: welcome, the water's fine. Sure its hard at first to grok all the breadth and depth of the offerings, but you get to build with the same tools the largest services on the web depend on. Start small and pick a service, learn it from the console, then cli. Hit up the docs a lot and ask questions, there are lots of people that can help. Do NOT get lead astray by this article or opinion pieces on Medium. Something like 75% of AWS services are geared at serving enterprises with on-premise data centers that want to move some or all of their workloads to the cloud. Stick with the basics like S3, Lambda, SNS, SQS, and whatever servers you need. Pick a managed service vs servers when you can unless you have solid sysadmin resources and a good reason. Avoid hourly billed services whenever you can until you can get cost management figured out. VPC is daunting unless you're good with networking, but you can avoid it entirely by using the primitives mentioned above. IAM takes some real grokking, but there is almost nothing you can't do with it to scope permissions both internally and externally. AWS YouTube Channel is full of quality material for all levels. Try to start forming your opinions with those and your own experience instead of unsubstantial articles or random HN comments :)

reese_john 7y ago

Why does it say that SQS has duplicates, while Kinesis does not ? AWS documentation explicitly says that Kinesis has at least once semantics due to consumer/producer retries.

fraserharris 7y ago

A lesser known limitation of S3 is that it does not have Read-After-Write consistency for overwriting PUTS & DELETE operations. Netflix has reported observing consistency taking hours for outliers. https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction...

pojzon 7y ago
Because its a distributed system and it takes time to propagate ?? I think its very well known, ppl just understand its normal and in 99.999% of cases does not matter.

skywhopper 7y ago

I really like the summary format and the visual design here. It feels slightly weird to specifically call out Redis in a negative way in the Dynamo summary when no other tools listed give a similar callout to a specific product.