3 ms·
Where is the security model in this? What stops malicious attackers from uploading updates that are constructed to destory the model?
by legulere 10y ago
Where is the security model in this? What stops malicious attackers from uploading updates that are constructed to destory the model?
- willvarfar 10y agoNo different from a centralized approach? When the rail company in Sweden first offered voice bookings people would hoax it by calling and saying "I want to book a ticket from Göteborg to Stockholm", and a robotic voice would reply "Do you want to book a ticket from Göteborg to Stockholm?"; at this point the hoaxer says "No, I want to book a ticket from Stockholm to Göteborg". And back and forth for hours and hours until the system could no longer distinguish between Stockholm and Göteborg and the system was basically useless. A system that tries to learn corrections, e.g. the keyboard example in the article, are all vulnerable to poisoning. As anyone who plays with poisoning Google search results delights in.
- anonymousDan 10y agoThat's hilarious! Do you have a link to an article about it by any chance?
- willvarfar 10y agoHeard it first-hand from a dev working on the project. Railway company is called SJ. My Google-fu is weak.
- legulere 10y agoI guess with the centralized approach you still have the possibility to detect malicious content. With the decentralized updates it is harder as you need to detect malicious updates.
- stingraycharles 10y agoI suppose that is where the Consensus part comes into play, in which an attack as you describe would simply be an outlier and not included in the model.