6 ms·
One of the problems we have to be conscious about with self supervised learning is that the already existing problem of transparency and understandability gets
by montenegrohugo 6y ago
One of the problems we have to be conscious about with self supervised learning is that the already existing problem of transparency and understandability gets worsened by orders of magnitude. If current billion parameter models are opaque, the result of SSL is, or will be, the equivalent of a black hole. How do you understand what the model has learned? Why has it learned that way? What things could have possible gone completely, horribly wrong in its quest to optimize its loss function?
This somewhat humorous but also worrying paper comes to mind:
https://youtu.be/Lu56xVlZ40M https://youtu.be/Lu56xVlZ40M
I think we really need to consider how we can mitigate the risk of raising completely ununderstandable yet scarily capable machines.
- iujjkfjdkkdkf 6y agoI dont agree with this, either we know what the pretext task was, or we fine tune with labels, but in either case, it's no different to analyze than supervised models. I do agree that more parameters and a bigger training set makes the models more opaque (or at least more expensive to understand), but the self supervision is not the reason.
- deleted 6y ago[deleted]
- Forge36 6y agoI have this same problem with people. Just because we don't understand perfectly doesn't mean the output is not useful.
- montenegrohugo 6y agoSure, but humans are a relatively known entity. They exist with a level of variance that is not too extreme, and you can more or less, in a general way, predict their behavior to a tolerable risk level. We've seen billions of humans, we know what to expect of them. We might not understand how other humans work internally, but we understand their actionspace. For AI models, that isn't really the case. We don't know how each individual model will act in the real world, and certainly not after having spent millions of hours training against itself or in a simulated environment. And as these models get more and more powerful, this growing uncertainty is something that I believe is worrisome.
- tasogare 6y ago> Sure, but humans are a relatively known entity. We don’t know how brain works, why sleep exist, most of humans cultures and languages are not documented, hundreds if not thousands of psychological and medical things are unknown, etc. Machines and their applications are a few more magnitudes more understood than any human matter, by the sheer fact we created them. Their complexity is ridiculously low in comparison to human, their biology, psychology, etc.
- epidemian 6y agoon one hand i agree with what you're saying. we humans have done terrible stuff like wars, genocides, famines, destruction of ecosystems, extinction of entire species, etc. and that's only the things we did more or less deliberately. we might cause our own extermination or a mass extinction event "by mistake", and we don't even understand how basic parts of our minds work. but all of that being said, i think it's also worth considering what a non-human entity with more power than us might be able to do. after all, even the worst examples of humanity are confined to at most "only" causing millions of deaths, but not the eradication of all life on Earth for example. it's not difficult to imagine that a non-human entity, with non-human goals, and with super-human powers, might act in a way that is contrary to our interests, or even to our existence.
- Voloskaya 6y agoNot sure I see your point. How is it harder to understand a model that was trained to predict which two images crops come from the same source image (a la contrastive loss for example), than a model that was trained to predict which class an image belongs to?
- loopz 6y agoWhat's the algorithm?
- visarga 6y ago> How do you understand what the model has learned? How do you do that with humans? You watch their behavior, language and you test them.
- vincnetas 6y agoYes. Humans make mistakes, humans with computers make mistakes faster. Computers without humans make mistakes even faster.
- uoaei 6y agoHumans can explain their reasoning when you ask them. The best we have with NNs is a fancy version of first-order sensitivity analysis.
- elcomet 6y agoHumans cannot really explain their reasoning. It has been shown that they usually invent an explanation that matches what has already been decided in their black-box.They mostly use post-hoc explanation, but cannot explain the true decision mechanism. If you make them believe that they choose another decision, they will create another explanation on the fly.
- ReDeiPirati 6y agoThis! Decision making is mostly "irrational". I wish it would be different.
- visarga 6y agoDecision making in humans is a mix of slow and fast thinking. Fast thinking - like reading, writing, physical reflexes - requires little effort and is best suited for repetitive situations. Slow thinking - such as reasoning - is more expensive so we only use it when we are faced with a new situation for which we have not 'cached' the answer. It entails the creation of a mental model and imagining outcomes. The exact mix of the two modes is tuned by evolution. If all we had was the slow system we'd be much less efficient at survival.
- knuthsat 6y agoTools for exploring the models are already being developed. Language models that are not given concepts of part-of-speech, dependency trees, grammars, develop representations of their own and people have found ways to inspect them.
- Schiphol 6y agoCould you please provide pointers to relevant papers? I'm very interested.
- desku 6y agoHere's a paper on how BERT (a large Transformer model trained using self-supervised learning) implicitly learns the traditional NLP pipeline: https://arxiv.org/abs/1905.05950 https://arxiv.org/abs/1905.05950