> some secret cost cutting measure That’s not the point — it’s just a day in the life of ops to tweak your system to improve resource utilization and performan

> some secret cost cutting measure That’s not the point — it’s just a day in the life of ops to tweak your system to improve resource utilization and performance. Which can cause bugs you don’t expect in LLMs. it’s a lot easier to monitor performance in a deterministic system, but harder to see the true impact a change has to the LLM