3 ms·
> I'm not sure how you could ever 100% trust a model someone else trains without you being able to train the model yourself Are sure you can even trust the mod
by sangnoir 5y ago
> I'm not sure how you could ever 100% trust a model someone else trains without you being able to train the model yourself
Are sure you can even trust the models you train yourself? It's possible that a model you trained is defective in a way you don't realize, e.g. will not recognize speeding cars with a sticker that has a picture of a bicycle[1]. It's likely that someone will discover that vulnerability before the publisher does, given the current state of ML model observability; ML model exploits are going to be wild, and inexplicable.
1. or the letters "GTEHQ"
- sponaugle 5y agoYea, I didn't think much about that, but you are right. Even if you train a dataset but use someone else's datasets those datasets could have specific things in them you might not notice. The hardest part of this problem is the difficulty of auditing. If you use someones open source code you at least have to potential of reading the code to look for something... with a large model that is difficult on a different scale!
- hackerfromthefu 5y agoIs it really difficult to audit, or practically impossible?
- yencabulator 5y agoEven if you train with your own data, what if the model learns "bicycles are always too slow to break speed limit" and "car with sticker picture of a bicycle is a bicycle". It's unlikely your test data would contain such a picture. Somebody else can notice this loophole and abuse it.