Baseten serves models as a service, at scale. There’s quite a lot of interesting engineering both for inference and infrastructure perf. This is a pretty good d

Baseten serves models as a service, at scale. There’s quite a lot of interesting engineering both for inference and infrastructure perf. This is a pretty good deep dive into the tricks they employ: https://www.baseten.co/resources/guide/the-baseten-inference-stack/#the-challenge-of-building-fast-cost-efficient-and-reliable-inference https://www.baseten.co/resources/guide/the-baseten-inference...