AI works exactly that way. Reinforcement learning involves training a model using some input features, some outputs, and a value function (usually, how closely

AI works exactly that way. Reinforcement learning involves training a model using some input features, some outputs, and a value function (usually, how closely the output(s) match a known value in the training set). Consider a model whose inputs are market prices, outputs are market transactions, and which has a naive value function that simply optimises for the amount of profit is trades result in in a relative short space of time. It's plausible that it could detect trading patterns that correspond to market crashes that can be exploited for profit. These trading patterns might actually be causing or supporting political / economic shocks, such as bubbles, corrupt or unethical behaviour or inter-state conflict that then cause these crashes, but, to the model, this would be invisible or irrelevant. It would learn to perform trades that encourage these same shocks (e.g. undermining a large employer, causing job losses that cause political anger), and profit from the consequences. All without ever "understanding" what is happening outside of its model or having malevolent intent.