5 ms·
I've long wondered if the answer for this and other issues like tax evasion wouldn't be to often change the metrics. If you start cheating the metrics, or opti
by MayeulC 5y ago
I've long wondered if the answer for this and other issues like tax evasion wouldn't be to often change the metrics.
If you start cheating the metrics, or optimizing a lot towards them, it becomes counter-productive when they change. As such, the most efficient way forward would be to work without trying to optimize for a temporary metric.
On the flip side, it would be troublesome to convince people to different, complex forms each time.
What first gave me the idea was the concept of "lubricating" headers (submitting some with random values) for future http protocols, to combat "ossification", where middle boxes start to meddle with them and become obsolete when they don't recognize the new fields, instead of transmitting them.
- tchalla 5y agoA slightly different take from me would be to have multiple metrics that kind of cross each other out making the tradeoffs explicit instead of implicit. It's fine to temporarily optimise towards anyone but at least we know which trade off was made.
- ByteJockey 5y agoI always thought having a pool of 20 metrics and randomly picking 4 every quarter/year for bonuses/promotions/whatever might get people to stop optimizing for single thing.
- bongoman37 5y agoThis is fine until someone sues because they had similar or better metrics than someone else but the other person got in in one year and they were refused. A lot of the reliance on metrics is CYA by administrators.
- MaxBarraclough 5y agoA more deterministic alternative: a weighted sum of the pool of metrics, where the weight can be gradually adjusted over time.
- Beldin 5y agoIf you have 20 independent metrics, this could work. Typically though, the metrics aren't independent as they all try to measure "doing your job well". In that case, if you gamed the "wrong" metrics, you still come out ahead. Not as much as for gaming the right metrics, but still.
- telchar 5y agoThe Joint Commission (healthcare field) seems to do this. There is at least some churn in metrics for each triennial inspection of healthcare facilities and operations. Aside from reducing gaming, it's a good way to make sure BI teams always have work to do.