2 ms·
From what I recall about that situation, they had a really stupid architecture that was using S3 as intermediate storage and processing video multiple times on
by dijit 2y ago
From what I recall about that situation, they had a really stupid architecture that was using S3 as intermediate storage and processing video multiple times on multiple stages.
In fact, the solution still used serverless afaik: https://www.youtube.com/watch?v=BcMm0aaqnnI https://www.youtube.com/watch?v=BcMm0aaqnnI
(take that u/UltraSane! https://news.ycombinator.com/item?id=42506205 https://news.ycombinator.com/item?id=42506205)
It likely could have been solved by serverless too, by using local storage and having the pipeline condensed into a single action...
FD: I'm not a fan of serverless for production anything.
- qaq 2y agoFor the same amount of compute Lambda is priced far higher than Fargate which itself is priced higher than ec2. People run large workloads on k8s with base workload compute fully covered by dedicated ec2 instances not because it's fun, but because it saves you a lot of $.
- dijit 2y agoTotally agree. But it's the python argument. Python is super slow, inefficient and let's say that the build environment is not so nice. However, it's so easy to write functioning software in python that the alternative sometimes is not more efficient code, it's no code. Lambda, in theory, follows a similar paradigm, if you can click a button and have a service that scales to zero (with logging and monitoring) then you're more likely to make toy webhooks and tiny services. If I have to make a build pipeline and a docker container and wrangle some yaml, configure service accounts and a service definition with the right labels and annotations. Well, that's a decent chunk of work that means I'm probably going to think a bit longer about even deploying my little toy service that might see one request a day. Going to reiterate though: I do not advocate for serverless in production. If you seriously think you're building something that will scale, it's fiscally illiterate to use a managed serverless provider.
- qaq 2y agoIf you under the impression that CI/CD and observability is easy with lambda I have a bridge to sell you. I worked on a large scale pure serverless project we wrote more CDK code than application code.