4 ms·
I did not read the paper fully, just some general commentary, hopefully useful. First, fantastic that you're doing academic work so early. I believe most stude
by darkmighty 7y ago
I did not read the paper fully, just some general commentary, hopefully useful.
First, fantastic that you're doing academic work so early. I believe most students wait far too long to be exposed to this aspect of academia (and their lives), which is far more about asking good questions, achieving deep understanding, and getting good results, than memorizing some procedure that isn't necessarily useful (but happens often in school). Keep creative and keep working on creating a good toolset (find math tools you find interesting/useful and own them!).
Now on to the paper. I'd reiterate the importance of asking good questions almost above results. For example, the result of adversarial sticks and sinks looks good -- but is it asking the right question? If you think realistically, adversarial attacks can occur in a number of ways.
One of them is that a human classifies a dataset one way while a machine another. In this case you would also want the human not to be able to tell you data is weird or there's something funky going on -- that is clearly the case with sticks (and sinks to a lesser extent). A human could be easily trained to spot them, and generally tell something weird is going on.
Another attack scenario is where you can modify some object, like a picture, but have some restriction on how much you can modify it. For example, you can manipulate only some bits of an image, or only perturb a small part of a real-world object that is under classification (say by putting a sticker on a car and fooling a system into thinking it is a dog, or something). If there is no restriction on your perturbation, this problem would be trivial (just replace the data with intended object data). The justification behind sticks and sinks does not look very well fundamented.
So sticks/sinks do not fare too well in either case, despite looking very good in terms of success vs defenses (although there's a chance they could inspire more practical attacks).
The commentary on Haussdorf distance is relevant here, but only on the first case (fooling human judgement), and it is of course an imperfect proxy (the true metric is human perception) -- another hint that fundamentals (and applications) are important to keep in mind.
Overall the paper seems well written and I specially like the numerous illustrations.
Keep the good work and don't forget to always look for the inspiring, beautiful and impactful, and seeking understanding. With a little of this in mind I have no doubt you can achieve very much. Good luck!
- c0deb0t 7y agoThanks for your comments and encouragements! The success rates of the attacks are not really emphasized---we only show that they work. They provide other benefits like robustness against point removal defenses. The attacks are optimized for different criterion. The sticks attack is supposed to be easy to construct, at the cost of perceptibility. The distributional attack is more geared towards imperceptibility. Indeed, we do bound the perceptibility of the sticks and sinks attacks, just with different metrics. You can even argue that the number of sticks we generate is a measure of perceptibility. Compared to other papers in terms of visual perceptibility, our attacks are not that crazy. Of course, human perception is the true metric, and I think more work must be done on quantifying perceptibility in 3D. This paper is the first step, and I mainly wanted to show that there are factors other than perceptibility that we care about.