3 ms·

> E.g. just knowing some inputs and outputs of the secret model allows you to train your own classifier which behaves similarly enough to let you find perceptu

> E.g. just knowing some inputs and outputs of the secret model allows you to train your own classifier which behaves similarly enough to let you find perceptual hash collisions. This seems highly unlikely. You could train a model to find those exact known hashes, but I highly doubt you could get it to accurately find any other unknown hash. > You'd have to keep all of the following secret: the model, all its inputs, all its outputs. These are all, in fact, secret.

saithound 5y ago
> This seems highly unlikely. You could train a model to find those exact known hashes, but I highly doubt you could get it to accurately find any other unknown hash. Your "highly doubt" is baseless. Black box attacks (where you create adversarial examples only using some inputs and outputs, but not the model) on machine learning models are not new. They have been demonstrated countless times [1]. You don't need to know the network at all. > These are all, in fact, secret. This is not the case, since regular, unprivileged Apple employees can and will look at the inputs and outputs of the model (the visual derivatives and their hashes). It's also irrelevant. You insist that there is some kind of analogy between "keeping the model secret" and keeping a "cryptographic key" secret. There is no such analogy. It makes no sense. It is simply not there. Keeping a cryptographic key secret keeps confidentiality and integrity. Keeping your model secret accomplishes neither of these. [1] https://towardsdatascience.com/adversarial-attacks-in-machine-learning-and-how-to-defend-against-them-a2beed95f49c https://towardsdatascience.com/adversarial-attacks-in-machin...
- user-the-name 5y ago
  > Your "highly doubt" is baseless. Black box attacks (where you create adversarial examples only using some inputs and outputs, but not the model) on machine learning models are not new. They have been demonstrated countless times [1]. You don't need to know the network at all. This is not a machine learning model as such, though, and is used differently than they are. > This is not the case, since regular, unprivileged Apple employees can and will look at the inputs and outputs of the model Can they?

saithound 5y ago

> This seems highly unlikely. You could train a model to find those exact known hashes, but I highly doubt you could get it to accurately find any other unknown hash. Your "highly doubt" is baseless. Black box attacks (where you create adversarial examples only using some inputs and outputs, but not the model) on machine learning models are not new. They have been demonstrated countless times [1]. You don't need to know the network at all. > These are all, in fact, secret. This is not the case, since regular, unprivileged Apple employees can and will look at the inputs and outputs of the model (the visual derivatives and their hashes). It's also irrelevant. You insist that there is some kind of analogy between "keeping the model secret" and keeping a "cryptographic key" secret. There is no such analogy. It makes no sense. It is simply not there. Keeping a cryptographic key secret keeps confidentiality and integrity. Keeping your model secret accomplishes neither of these. [1] https://towardsdatascience.com/adversarial-attacks-in-machine-learning-and-how-to-defend-against-them-a2beed95f49c https://towardsdatascience.com/adversarial-attacks-in-machin...

user-the-name 5y ago
> Your "highly doubt" is baseless. Black box attacks (where you create adversarial examples only using some inputs and outputs, but not the model) on machine learning models are not new. They have been demonstrated countless times [1]. You don't need to know the network at all. This is not a machine learning model as such, though, and is used differently than they are. > This is not the case, since regular, unprivileged Apple employees can and will look at the inputs and outputs of the model Can they?