4 ms·
It astounds me that a company valued in the hundreds-of-billions-of-dollars has written this. One of the following must be true: 1. They actually believed late
by tadfisher 6mo ago
It astounds me that a company valued in the hundreds-of-billions-of-dollars has written this. One of the following must be true:
1. They actually believed latency reduction was worth compromising output quality for sessions that have already been long idle. Moreover, they thought doing so was better than showing a loading indicator or some other means of communicating to the user that context is being loaded.
2. What I suspect actually happened: they wanted to cost-reduce idle sessions to the bare minimum, and "latency" is a convenient-enough excuse to pass muster in a blog post explaining a resulting bug.
- retinaros 6mo agothey just vibecoded a fix and didnt think about the tradeoff they were making and their always yes-man of a model just went with it
- someguyiguess 6mo agoIt’s definitely a cost / resource saving strategy on their end.
- billywhizz 6mo agowhat's even more amazing is it took them two weeks to fix what must have been a pretty obvious bug, especially given who they are and what they are selling.
- raincole 6mo agoIt's very weird that they frame caching as "latency reduction" when it comes to a cloud service. I mean, yes, technically it reduces latency, but more importantly it reduces cost. Sometimes it's more than 80% of the total cost. I'm sure most companies and customers will consider compromising quality for 80% cost reduction. If they just be honest they'll be fine.
- sekai 6mo agoThe same company that claims they have models that are too "dangerous" to release btw.
- adam_patarino 6mo agoIt’s certainly #2. They have shown over dozens of decisions they move very quickly, break stuff, then have to both figure out what broke and how to explain it.