4 ms·
Why don't they monitor average prompt and response token length(both cached and uncached) per interaction. Seems this could have solved all their previous unnot
by YetAnotherNick 6mo ago
Why don't they monitor average prompt and response token length(both cached and uncached) per interaction. Seems this could have solved all their previous unnoticed degradation.
Also bit surprised they don't have any automated quality check. They can run something like swe bench before each release. Both of these seem like a basic thing even for startup, let alone some product generating billions in revenue.