3 ms·

This is not a discussion of monolith vs serverless. This is some terrible engineering all over that was "fixed". Some excerpts: > This eliminated the need for

by LASR 3y ago

This is not a discussion of monolith vs serverless. This is some terrible engineering all over that was "fixed". Some excerpts: > This eliminated the need for the S3 bucket as the intermediate storage for video frames because our data transfer now happened in the memory. My candid reaction: Seriously? WTF? I am honestly surprised that someone thought it was a good idea to shuffle video frames over the wire to S3 and then back down to run some buffer computations. Fixing the problem and then calling it a win? But I think I understand what might have lead to this. At AWS, there is an emphasis on using their own services. So when use cases that don't fit well on top of AWS services come up, there is internal pressure to shoehorn it anyway. Hence these sorts of decisions.

chank 3y ago
> Fixing the problem and then calling it a win? It is a win. Just not the win they're aluding to.
ripper1138 3y ago
This is what L6 and L7 are building at Amazon, meanwhile in sys design interviews I’m being asked to design solutions for a gaming platform with 50M concurrent users.
adql 3y ago
> This is not a discussion of monolith vs serverless. This is some terrible engineering all over that was "fixed". I feel that's like 95% of the "we migrated from X to Y and now it is better"; most of improvements coming from rewriting app/infrastructure after learning the lessons with only small part sometimes being the change in tech
tylerdurden91 3y ago
To the contrary, from my time at Amazon, I felt that developers want to use more high level AWS services. Unfortunately, the landscape of AWS services is so rapidly evolving that Amazon engineers themselves cant keep up and end up using the wrong service. As mentioned in other comments, there are options such as Fargate, that would still technically be "serverless" and still yield similar cost reductions. Not to mention that AWS also has Step functions express for "on host orchestration" use cases. This seems like a case where the original architecture wasn't very well researched and nor was the new one.
munchbunny 3y ago
I wouldn't be surprised if the actual story underneath was that they got to a "works well enough" implementation and then forgot about the inefficiencies until someone looked at costs, connected the dots, and went "ok yeah we need to optimize this architecture." I've seen some staggering cost savings realized because someone happened to notice that an inefficient implementation that wasn't a problem two years ago at the scale it was running at back then did not age well to the 10x volume it was handling two years later. The reason it hadn't fallen over was that horizontal scaling features built into the cloud products were able to keep it running with minimal attention from the SRE's.

by LASR 3y ago

chank 3y ago

> Fixing the problem and then calling it a win? It is a win. Just not the win they're aluding to.

ripper1138 3y ago

This is what L6 and L7 are building at Amazon, meanwhile in sys design interviews I’m being asked to design solutions for a gaming platform with 50M concurrent users.

adql 3y ago

> This is not a discussion of monolith vs serverless. This is some terrible engineering all over that was "fixed". I feel that's like 95% of the "we migrated from X to Y and now it is better"; most of improvements coming from rewriting app/infrastructure after learning the lessons with only small part sometimes being the change in tech

tylerdurden91 3y ago

To the contrary, from my time at Amazon, I felt that developers want to use more high level AWS services. Unfortunately, the landscape of AWS services is so rapidly evolving that Amazon engineers themselves cant keep up and end up using the wrong service. As mentioned in other comments, there are options such as Fargate, that would still technically be "serverless" and still yield similar cost reductions. Not to mention that AWS also has Step functions express for "on host orchestration" use cases. This seems like a case where the original architecture wasn't very well researched and nor was the new one.

munchbunny 3y ago

I wouldn't be surprised if the actual story underneath was that they got to a "works well enough" implementation and then forgot about the inefficiencies until someone looked at costs, connected the dots, and went "ok yeah we need to optimize this architecture." I've seen some staggering cost savings realized because someone happened to notice that an inefficient implementation that wasn't a problem two years ago at the scale it was running at back then did not age well to the 10x volume it was handling two years later. The reason it hadn't fallen over was that horizontal scaling features built into the cloud products were able to keep it running with minimal attention from the SRE's.