3 ms·
This will always happen and will happen much more as we move forward. People specifically tune their models to sound human. People specifically use detectors to
by godelski 3y ago
This will always happen and will happen much more as we move forward. People specifically tune their models to sound human. People specifically use detectors to turn models, adversarially.
I keep proclaiming that we live in Goodhart's Hell, and this is yet another great example.
First, you have to remember that metrics are never a complete measure, but they are guides. This doesn't matter if you're cutting lumber or training a ML model. Metrics are __always__ proxies. For the vast majority of things you are intending to evaluate, you don't actually have a measurable function to express that. Every fucking measurement has error/noise, and this is why you measure twice and cut once. It is absolutely insane that in the age of advanced algorithms we've just tossed caution to the wind and we're instead doing a half assed measurement and then getting upset when our cuts are bad, blaming the ruler or blaming the saw.
Second, you have to ask how closely your metric aligns with the thing you're actually trying to measure. Even cutting lumber you have to check that your measurement aligns. Accounting for angles and grain. We should be doing the same in algorithms, but with __much__ more care, but we don't for some reason. All metrics must be continually scrutinized. This is why we measure twice (or more), why we double check our calculations, and we do frequent sanity checks and build in redundancy, because you can't fucking trust your metrics.
For some reason we seem to have this belief that empirical measurements are absolute and generalized. We seem to have this belief that noise is a thing that can be decoupled from the system and precisely removed. We seem to have this belief that measurements are unhackable. What the fuck is going on. This isn't just ML, it is academia, it is bureaucracy, it is the workplace, it is governments, it is economics. So I gotta ask, what the fuck is going on? In the past we treated things as hard, so there was an inherent uncertainty built into mental models but now we treat everything like they are simple to understand and a quick reading or video can make you an expert. We may have never had nuance from non-experts, but now everyone acts like an expert and that math is unbiased and objective. Why do we keep shooting ourselves in the foot and deciding the next best action is to reload the gun?