An evaluation over reliability does not just come from test results: it also results from structural assessments. Yes, but it's actually remarkable how bad hum

An evaluation over reliability does not just come from test results: it also results from structural assessments. Yes, but it's actually remarkable how bad humans are at seemingly simple tasks like recalling a short scene that they witnessed in person, or even just estimating how good their recollection is. Humans will routinely get even the most basic facts such as how many people were involved, who did what, in what order events occurred... completely wrong, while often being adamant that their account is correct. There are studies about that in the context of crime witnesses and the type of biases also mentioned in the article. A NN network might give a different answer if we apply some practically invisible Gaussian noise to an image, a human will have a different memory recollection depending on the lighting conditions of the room they're in, whether they're asked before or after lunch break... Again, I'm on board with saying that our standards for machines should be higher than that, but I think we're underestimating how hard those seemingly simple tasks are even for humans.