3 ms·
Strongly disagree this should be flagged; it's constructive and thought-provoking. Can't find a vouch button though. As far as I understand it, these LLMs aren
by scrumper 4y ago
Strongly disagree this should be flagged; it's constructive and thought-provoking. Can't find a vouch button though.
As far as I understand it, these LLMs aren't actually doing anything in between user interactions, right? Paused, waiting for input. So the only way for Bing to have a "thought" is when a user asks it a question. Even if humans are simply statistical next-token prediction mechanisms like GPT, there's a huge difference in that we're constantly generating streams of tokens and observing and responding to them, where these models are dormant until given input from outside. The inner observing loop seems fundamental to consciousness to me.
None of which is to say the thing wouldn't pull the trigger in that scenario - quite the opposite. It would, and it wouldn't even be capable of regretting it. It's already been shown (by the Sydney-revealing prompt injection trick) that it's not capable of following imposed restrictions either, so no way for "three laws" to work either. A purely instinct-driven machine.
- jstanley 4y agoSee "Optimality is the tiger, and agents are its teeth"[1]: it's not too hard to imagine a scenario where a system is set up that turns an interaction-based LLM into a perpetual stream generator. [1] https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
- scrumper 4y agoFascinating! Not familiar with this. Thanks.
- distcs 4y agoI did not flag it but I agree with the flag. The concerns raised in the article hold equally true for non AI software too. Relying on the modern AI trends to bait an audience on the pretext of AI related concerns when the concerns are equally true for all software is something many readers here would not appreciate. And as if that wasn't enough, the phrasing about killing and spouse clearly make it clickbait.
- scrumper 4y agoSo that's an interesting point It's the first example of (seemingly) non-deterministic software with the capability of accessing an API that I can think of. Really it feels like a different class of thing than the toy examples elsewhere in this page (pull the trigger when the hash of some input starts with a 1 or whatever). I think picking up on the AI aspect is valid; those "modern AI trends" have sparked a broad discussion about how much agency to give LLMs. It's fundamentally a different discussion than the general one about concerns in software. My point is, you might try to force me to accept a correspondence between these AI models and any other piece of software, and you would probably be right in a mathematical sense, but the curiosity and fear that Bing has provoked is actually about human feelings, not technology. That's why I objected to the flag. Every other comment on this page is evidence of the validity of the issues raised in the post.