3 ms·
> Ultimately the engineers and other humans behind the scenes are the ultimate arbiters of success, loss functions are chosen by humans, and swapping training d
by m00x 6y ago
> Ultimately the engineers and other humans behind the scenes are the ultimate arbiters of success, loss functions are chosen by humans, and swapping training data on the same model won't change those early decisions
How would changing loss functions alter this? This makes no sense.
Hyperparameter tuning is done iteratively, and used get the best score on the test dataset. Do you really believe the engineers hand picked examples and purposefully trained it so that looked more "white" and disregarded the test scores?
Training/test data is 100% the cause for this bias.
- 6gvONxR4sf7o 6y agoLearning conditional mean vs conditional median for superresolution might be differently affected by a large chunk of outgroup. That was what I remember folks talking about during this twitter feud. Or, like with word embeddings, where people have added some balance penalties to the loss function to make it unbiased in the presence of biased data. Dataset bias caused this, but dataset bias can sometimes be well mitigated.
- ianbicking 6y agoThe goal is to get pleasing face reconstruction, everything follows from that. There's no objectively correct loss function, it's something that is selected because it has a good effect. A simple example could be sensitivity to color differences: with lighter skin tones there is a fairly large difference between eyebrows and eye features and the person's forehead or cheek. Someone with a pretty dark complexion might have features that are distinguished in a very compressed set of colors. Depending on the loss function a dark complexion face could be essentially flat and featureless. This could be fixed of course... but it requires changing the loss function.