4 ms·
> Moreover, it's a neural network with well-defined input and output channels, not some kind of self-modifying executable. Elsewhere upthread someone posted a
by TeMPOraL 4y ago
> Moreover, it's a neural network with well-defined input and output channels, not some kind of self-modifying executable.
Elsewhere upthread someone posted a link to this article:
https://www.alignmentforum.org/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.alignmentforum.org/posts/kpPnReyBC54KESiSn/optim...
which constructs a scenario in which (spoiler alert) a GPT-based model could accidentally trick you into bootstrapping a self-modifying runtime, consisting of unconstrained, recursive execution of the AI's own model.
> But ok, instead of sending you the coins, it could just tell/promise you a wallet address and private key. How did it obtain those in the first place and how did remember them if state is reset for each thread?
Don't focus on cryptocurrencies here. The thesis is a sufficiently smart AI can talk its way out of the box somehow. There is no one good answer here, because it's trying to manipulate the human operator.