3 ms·
Data ordering attacks on deep neural networks
- tartakovsky 4y agoCan we learn permutation matrices? Something like this but applied to learning order that maximizes some objective. Off the cuff it sounds like an NP problem but seems that people have tried before with some success. https://arxiv.org/abs/2205.00242v1 https://arxiv.org/abs/2205.00242v1 It also reminds me of tuning random seeds (can't find the arxiv link) Or things like this: https://arxiv.org/abs/2104.06338 https://arxiv.org/abs/2104.06338 Overall this stuff seems highly intractable in today's world but the humans are trying to make headway. Such is research.
- melony 4y agoIs there a generating function for it?
- gillesjacobs 4y agoIf someone controls the ordering of samples in training code, you have a compromised pipeline. Owning a codebase and underlying dataset to steer towards likely but uncertain model predictions seems like a far-fetched threat model. Still interesting research in its own right and goes to show it is worth testing for randomness of all system components.
- NonNefarious 4y agoWhy are the data ordering attacks? Do they hate neural networks or something?