3 ms·

Not what the paper discusses, but the title gives me a thought: can you train a neural net with a specific, predictable behavior that would generally go undetec

by blunte 5y ago

Not what the paper discusses, but the title gives me a thought: can you train a neural net with a specific, predictable behavior that would generally go undetected but which you could use to your advantage? I suspect you could. Imagine building an apparently useful trained model for finance which you have a special way to predict the behavior of and profit from. (This could also be likened to formal human finance education, which certain elite groups take advantage of, but I don't mean to go into conspiracy theory territory...)

xyzzy21 5y ago
You mean like a "mole" or "sleeper cell"? :-) Just give it the right input and it turns on its masters.
- blunte 5y ago
  Yes, basically. Imagine in financial markets if you trained a system to identify trends in currency pairs (for example). Generally they would be benign and useful, but if the a certain input were present, they would advise an action which the designers would expect and profit from. The users of the system would follow the advice as normal and probably not even know that someone essentially had inside information.
  - kiwih 5y ago
    There is a name for this in machine learning, it's called a 'poisoning attack'. Essentially you add false data into a training set. An example: https://arxiv.org/abs/1703.01340 https://arxiv.org/abs/1703.01340
123pie123 5y ago
Neuralnet/ ML version of "Order 66"
- blunte 5y ago
  If I read the back story on this correctly, then no - it's not what I mean. I don't mean to change an outcome, but rather to create a situation where you can be certain of what the oucome will be. Then you can profit. Granted, I wouldn't do this myself as I do find ethical problems. But its a fascinating thought experiment. And if you believe in human legacies, then you don't need AI. You just build a system and manage the players so that your offspring or their offspring can take advantage of the game you have setup. Maybe this falls into Rothschild-like territory (which I frankly don't believe humans are meticulous, lucky, and coordinated enough to pull off).

by blunte 5y ago

xyzzy21 5y ago

You mean like a "mole" or "sleeper cell"? :-) Just give it the right input and it turns on its masters.

blunte 5y ago
Yes, basically. Imagine in financial markets if you trained a system to identify trends in currency pairs (for example). Generally they would be benign and useful, but if the a certain input were present, they would advise an action which the designers would expect and profit from. The users of the system would follow the advice as normal and probably not even know that someone essentially had inside information.
- kiwih 5y ago
  There is a name for this in machine learning, it's called a 'poisoning attack'. Essentially you add false data into a training set. An example: https://arxiv.org/abs/1703.01340 https://arxiv.org/abs/1703.01340

123pie123 5y ago

Neuralnet/ ML version of "Order 66"

blunte 5y ago
If I read the back story on this correctly, then no - it's not what I mean. I don't mean to change an outcome, but rather to create a situation where you can be certain of what the oucome will be. Then you can profit. Granted, I wouldn't do this myself as I do find ethical problems. But its a fascinating thought experiment. And if you believe in human legacies, then you don't need AI. You just build a system and manage the players so that your offspring or their offspring can take advantage of the game you have setup. Maybe this falls into Rothschild-like territory (which I frankly don't believe humans are meticulous, lucky, and coordinated enough to pull off).