3 ms·
Bias is an overloaded word. To be clear, there is bias (there's always bias). Every model always has bias, we just want to understand that to understand the lim
by godelski 5y ago
Bias is an overloaded word. To be clear, there is bias (there's always bias). Every model always has bias, we just want to understand that to understand the limitations of it. But yes, I agree that the reconstruction is decently faithful. I'm sure an L2 loss isn't that high when comparing the images.
Part of what I was trying to say is that there is a clear human bias when evaluating the reconstruction. The problem here is our prior. We humans already know that the compressed image is Obama because he is a well known face. So we're judging based on that prior. So there is some irony when talking about model bias when you don't recognize the prior bias in evaluation. We should analyze the bias of the model (I mean there's problems with the reconstruction other than the race issue, like the teeth), but we have to also be careful when analyzing because we also have a bias (which we may or may not want to impart upon the model).
The thing that gets me about these bias people in ML is that it is easy to recognize and identify bias. It is much harder to solve them. There seems to be two camps. Those that just want to identify and those that work on resolving them. The former typically say they work in bias and the latter typically says they work on long tail data (or sometimes few/low shot learning, or even metalearning). Yeah, we should recognize biases and discuss them, but just yelling at people doesn't help develop the necessary mathematical frameworks to resolve the issues.