3 ms·

There is no feedback loop to support the development of the attack. Apple in this case would just be filtering and ignoring false positives without notifying th

There is no feedback loop to support the development of the attack. Apple in this case would just be filtering and ignoring false positives without notifying the attacker if they were successful or not. I still don't quite see how the 'worm' would work in practice. It's not just a matter of sharing a link to an image, but you have to add it to your photo album. Maybe I'm getting old, but I don't have a single meme-type image in my album. I don't think this can be classed as a security by obscurity problem. Security by obscurity fails because they 'key' (the obscure bit) can't be easily changed if it leaks. But given Apple has both the target and candidate images available to it, it can in effect generate a new key at the expense of having to do additional computation.

mirker 5y ago
The way I think about it is that the attacker has plenty of time (years) to either guess or discover the viability of an attack. For example, inspecting packets and reverse engineering, correlating photos with police visits, creating a hash database using black market data, etc. will all be used to determine if an attack is working. Same thing goes with the worm. Posting infected memes is one way and that’ll passively get some downloads. Another way is to text the pictures via bots to people and hope their SMS is iCloud backed up. The point is if someone figures out how to make an attack, they (or entity they sell it to) will almost surely spend great effort in social engineering the distribution. I agree that it’s probably not the right term. My main concern though is that the class of attacks that is effective against one model will likely be effective against the entire class of similar models. So Apple’s main defense is the limited feedback loop caused by keeping part of the modeling private rather than a traditional computational complexity defense, which would allow disclosure of the entire implementation. It’s taken one week for the proof of concept to be demonstrated and most of that was likely boilerplate API code; it would take perhaps even less time to retarget if Apple disclosed the entire implementation with a different model. The demonstrated attack was textbook material, so it’s not unreasonable to believe that all attacks are textbook material.

mirker 5y ago

The way I think about it is that the attacker has plenty of time (years) to either guess or discover the viability of an attack. For example, inspecting packets and reverse engineering, correlating photos with police visits, creating a hash database using black market data, etc. will all be used to determine if an attack is working. Same thing goes with the worm. Posting infected memes is one way and that’ll passively get some downloads. Another way is to text the pictures via bots to people and hope their SMS is iCloud backed up. The point is if someone figures out how to make an attack, they (or entity they sell it to) will almost surely spend great effort in social engineering the distribution. I agree that it’s probably not the right term. My main concern though is that the class of attacks that is effective against one model will likely be effective against the entire class of similar models. So Apple’s main defense is the limited feedback loop caused by keeping part of the modeling private rather than a traditional computational complexity defense, which would allow disclosure of the entire implementation. It’s taken one week for the proof of concept to be demonstrated and most of that was likely boilerplate API code; it would take perhaps even less time to retarget if Apple disclosed the entire implementation with a different model. The demonstrated attack was textbook material, so it’s not unreasonable to believe that all attacks are textbook material.