9 ms·
Prime Day 2017: 12.9M DynamoDB requests per second
- tcarn 9y agoAmazing metrics, the website performed flawlessly and their entire organization should get a giant round of applause
- delecti 9y agoAfter hearing pagers going off throughout that day, it's reassuring to hear someone on the outside say everything went flawlessly.
- rajathagasthya 9y agoI’m curious, how do you prep for things like the Prime Day? I imagine almost all parts of Amazon.com and AWS operate at their limits, especially resource-wise. What precautions do you take?
- lxmcneill 9y agoJeff covered this in the post (GameDays, excessive amounts of auditing, etc.). Regarding the resource limits you suggest, they mention metrics of 50+ pB of data movement and 3.34 trillion DynamoDB queries in 30 hours, all of which they elastically scale down after the event... so I'd say resource limitations are more in terms of humans on-deck and crisis management, rather than physical limitation of hardware. (edited to correct size - was 52 pB not 520 pB...)
- jlgaddis 9y agoYou might be interested to read the white paper that was linked in TFA.
- LiteskinKanye 9y agoWhat's the DevOps duty look like? Anytime you just see a page and root cause it and just tell your higher-up they are SOL
- mabbo 9y agoI can't speak for the whole company, but I've done 5 peaks (holiday seasons) as an SDE in the Warehouse/Delivery orgs. The key to remember about Amazon DevOps is that Developers are also DevOps. My team would usually share a dedicated DevOps team with 3 to 6 other teams- usually rotating front-line on-call duty between 2 or 3 people in the US and 2-3 in India. That DevOps person has the job of: what is the problem? Do my teams own this problem (if not, redirect to the right place)? Do I know how to immediately fix this? If not, for which of my teams do I page the on-call SDE?" When you have a great DevOps team, SDE oncall duty is a walk in the park. But DevOps people take time to become great- and many of them are also applying for transfers to SDE roles. Overall, each org decides how on-call/DevOps is. If time and effort and spent investing in stable software, it's easy. If other priorities get in the way, things can get bad.
- engi_nerd 9y agoSure, and for the second year in a row, the Amazon website told me a price for a deal, told me the deal was still available, and then wouldn't let me purchase it. Overall things went well for Amazon, but flawless? I think not.
- brndnmtthws 9y ago"Prime Day" is a rip-off, and the pricing schemes border on fraudulent. Give yourselves a pat on the back for tricking people into spending money they'd be better off saving.
- Flammy 9y agoNot the point of the submission - keeping up with the interest, regardless of the contents, still is interesting and useful to know more about.
- brndnmtthws 9y agoWould you applaud a criminal for inventing a novel way to commit a crime?
- azhenley 9y agoIt seems like most of the documentaries I watch now a days are basically that! They're very interesting and entertaining.
- dvt 9y agoIf you're going to say stuff like this, at least provide a modicum of evidence. What pricing scheme, in particular, is borderline fraudulent? Does it differ fundamentally from events like Black Friday? Why should people save money instead of spending it?
- brndnmtthws 9y agoJust do an internet search for "prime day scam", or something to that effect. The FTC has been investigating Amazon for their price deception tactics: https://www.reuters.com/article/us-whole-foods-m-a-amazon-ftc/ftc-probing-allegations-of-amazons-deceptive-discounting-idUSKBN1A52R5 https://www.reuters.com/article/us-whole-foods-m-a-amazon-ft... Here's more on the report: http://www.consumerwatchdog.org/story/your-amazon-sale-item-truly-discounted-here%E2%80%99s-how-find-out http://www.consumerwatchdog.org/story/your-amazon-sale-item-...
- h4pless 9y agoSo what? I still got a 500 error while trying to process an order and when I refreshed - sold out. "60% of the time - it works every time" is only good for Panther scented cologne. 12,900,000 / second - WOW. Too bad the request rate was probably closer to 15,000,000 / second and you just abandoned everyone else.
- plandis 9y agoOnly 60% of your requests to Amazon went through? That is really really bad. I don't remember hearing of an outage.
- scrollaway 9y agoI don't know what is blowing my mind more: Those numbers, or reading someone on this site reacting to them with a "so what?".
- sandGorgon 9y agoWhat is Dynamodb used for internally? Looks like it's being used for fulfillment and checkout as well. Is Amazon entirely based out of Dynamodb nosql ?
- plandis 9y agoAccording to this Amazon job listing [1], DynamoDB isn't even the biggest (in terms of throughput) data store that Amazon uses. [1] https://www.amazon.jobs/en/jobs/447605/systems-engineer https://www.amazon.jobs/en/jobs/447605/systems-engineer
- jonhohle 9y agoI miss SABLE. I like it better than DDB, but I'm probably in the minority there. It made building high throughput services easy. I'm somewhat surprised it mentioned publicly.
- deleted 9y ago[deleted]
- rprime 9y ago> I'm somewhat surprised it mentioned publicly. I guess it explains why I am not able to find any information related to SABLE. Are you able to release anything related to it? It seems really interesting.
- ra7 9y ago> SABLE is at the core of Amazon.com retail infrastructure. It more than 400 billion requests per day from Amazon.com websites, Web Services, and internal Amazon systems. Wow!
- puzzle 9y agoThree years ago or more, Bigtable served that much traffic in less than fifteen minutes: https://twitter.com/akpurtell/status/463747917782589441 https://twitter.com/akpurtell/status/463747917782589441
- mattdennewitz 9y agowhat'd that cost?
- jlgaddis 9y agoGood thing they aren't paying $0.09/GB for data transfer!
- uji 9y agoIs Amazon comfortable enough to create single point of failure. What if DDB goes down assuming they run same stack for both retail and AWS?
- yeukhon 9y agoThey introduce change one region at a time usually. DDB did have outage many times and brought half of AWS operations down. But good question how they kept Amazon.com alive if some part of their systems rely on DDB. [1]: https://aws.amazon.com/message/5467D2/ https://aws.amazon.com/message/5467D2/
- phamilton 9y ago> What if DDB goes down It's worth clarifying that question. What does "Go Down" mean? One region is down? Is it a bad software bug? The database itself is fault tolerant. Nodes can go down all day without any real consequence. Each region is isolated, so a failure in one region won't affect other regions. I guess I'm at a loss for the type of failure with DDB that would make it a SPOF, especially more than other datastores.
- dx034 9y agoHow do they structure regions? Is amazon.co.uk running on different regions than amazon.com? Or everything distributed globally?
- pentelian 9y agoWhen I read that I thought holy heck... on DynamoBD, if it's not a case study about Amazon then that company is in serious financial trouble.
- usaphp 9y agoI would want to have a "trouble" of having 12.9M customer requests per second ))
- dx034 9y agoI'm sure that any external client using >1bn queries per day would get a very different pricing from the one advertised.
- jjeaff 9y agoI have heard that the price is the price for AWS and that the small guys pay the same as the biggest ones. But I have no evidence to back that up.
- hobofan 9y agoIf the AWS cost estimator is not broken, it wouldn't even be _crazy_ expensive at ~8mio/month. Snap with 1 billion committed to AWS over the next 5 years could definitely come close to the same order of magnitude on a consistent basis.
- lkrubner 9y agoThis is a good example of the stupidity of the normal rules that Hacker News enforces about article titles. I am glad they made an exception in this case. I never would have read this if the headline text had been "Prime Day 2017 – Powered by AWS". I did read the article, because in this case, Hacker News allowed the title to highlight what I would find interesting: the incredible performance of DynamoDB.
- dastbe 9y agoI don't think the rules actually say you can't edit titles. At best, they ask that you please use the original title unless misleading or linkbait. You could argue that the original title is misleading, because you would've thought that it was some advertisement (though isn't anything touting large numbers a form of advertising?).
- julianhouston12 9y agoHi My name is Julian Houston, am from upper island cove , Canada.. I want to Use this opportunity to thank God for using this Great prophet to solve my Marriage problem. This Great Prophet of God brought my husband back to me. 3 years ago, i and my husband has been into one quarrel or the other until he finally left me for one lady. I felt my life was over and my kids thought they would never see their father again. I tried to be strong just for the kids but i could not control the pains that was tormenting my heart. My heart was filled with sorrows and pains, because i was really in love with my husband. Every day and night i think of him and always wish he would come back to me. Until one day i melt a good friend of mine that was once in my situation, but her problem was different a little bite, her ex-boyfriend who she had an unwanted pregnancy for refused to take responsibility and dumped her. She told me that mine issue was a minor case and that i shouldn't worry about it at all. So, i asked her what was the solution to my problems and she gave me this Great Prophet of God phone number and his email address. I was doubting if this Great Prophet of God could actually solve my problem. So, I contacted this Great Prophet of God and he told me what to do and i did it. He told me to wait for just four days and that my husband will come crawling on his kneels just for forgiveness. So, I faithfully did what this Great Prophet of God asked me to do and for sure after four days i heard a knock on the door, in a great Surprise i saw him on his knees and i was speechless, when he saw me, all he did was crying and asking me for forgiveness, from that day, all the pains and sorrows in my heart flew away, since then i and my husband and our lovely kids are happy. That why i want to say thank you to God for using Prophet Ikehedu .to solve my marriage problem. This Great Prophet of God made me to understand that there’s no problem on earth that does not Have solution. So, if you are having same problem, any problem that is Similar, i will advise you to a contact This Great Prophet of God straight At prophetikehedu@gmail.com
- Artemis2 9y agoSomething I appreciate very much with AWS is how much dogfooding Amazon does. In contrast, Google has said in the past that GCP is not used by Google engineers (and it shows, occasionally GCP goes down but actual Google products do not!). There is also no clear indication that Microsoft is actively using the public deployment of Azure.
- tehlike 9y agoPart of is because gcp became a product much later in google's history. As a result, most of the products that were used were tied to internal google infra, and was hard to put on gcp.
- sitepodmatt 9y agoI think this will gradually change. Kubernetes came out of all the internal learning after all, I would be surprised if they aren't putting new greenfield stuff on GCP and other Google Cloud products.
- deleted 9y ago[deleted]
- londons_explore 9y agoThis is true for smaller projects. Big projects are still built out of GCP mostly for cost reasons - google doesn't give itself a discount on gcp
- sitepodmatt 9y agoDid you mean to say 'still not built out of'?
- ElectronShak 9y ago>dogfooding The term "dogfooding" is a tech slang for the use of one's own products. In some uses, it implies that developers or companies are using their own products to work out bugs, as in beta testing. One benefit of dogfooding is that it shows that a company is confident about its products. I didnt know what the term meant so had to look it up. Yeah, I agree.
- yeukhon 9y agoThe elasticity of EBS sizing on demand from this year really is a shocking and exciting news.
- deleted 9y ago[deleted]
- vegbrasil 9y agoI wonder how many of that were made by bots looking for the best prices...
- patrickxb 9y agoMust be nice if you don't have to pay for it...
- jjeaff 9y agoIf they operate like lots of other big corps, I would think the retail arm actually would have to pay AWS for the service. Obviously zero sum, but you want can't know your true profit and loss for individual business units if you don't.