3 ms·

> What could go wrong? Great question! These are predictive models that accept a text query, do some matrix math, and then return some text. At what point in t

> What could go wrong? Great question! These are predictive models that accept a text query, do some matrix math, and then return some text. At what point in that server-client relationship does this algorithm jump the rails and run amok?

deleted 4y ago
[deleted]
doctor_eval 4y ago
What about here: https://news.ycombinator.com/item?id=35239578 https://news.ycombinator.com/item?id=35239578 > A popular nightmare scenario for AI is giving it access to tools, so it can make API calls and execute its own code and generally break free of the constraints of its initial environment. Let's do that now! I also think that you're assuming we know a lot more about how these things work than we actually do; you seem to think nobody is going to hook these up to APIs that can actually modify the world, despite the barrier to doing so being incredibly low; and you don't seem to have read about the adversarial training that people have been doing between the LLMs. It's obvious that you think everything is all safe and nothing will go wrong, and I really hope you're right. But I think it's a very dangerous assumption. Hope for the best, plan for the worst.
- cjohnson318 4y ago
  You've moved the goalposts from, "we should assume that predictive text models are sentient," to, "we should be wary of programs explicitly designed to interact unpredictably with other computer systems," something that everyone from the Amish to the NSA to Cozy Bear would agree with.
  - doctor_eval 4y ago
    What I said at the outset was, > Considering that we don’t know how the brain works so well, and we don’t understand why LLMs work so well, simply on the basis of their output I think the safest assumption is that these models do indeed have agency, or at least the capability of agency. These systems can already interact with others, it’s not moving the goalposts, it’s common knowledge. Anyone with access to the APIs can make it happen. Or are you now claiming we are just talking about one specific LLM and not LLMs generally? Anyway this discussion is fruitless, I’m out.

deleted 4y ago

[deleted]

doctor_eval 4y ago

What about here: https://news.ycombinator.com/item?id=35239578 https://news.ycombinator.com/item?id=35239578 > A popular nightmare scenario for AI is giving it access to tools, so it can make API calls and execute its own code and generally break free of the constraints of its initial environment. Let's do that now! I also think that you're assuming we know a lot more about how these things work than we actually do; you seem to think nobody is going to hook these up to APIs that can actually modify the world, despite the barrier to doing so being incredibly low; and you don't seem to have read about the adversarial training that people have been doing between the LLMs. It's obvious that you think everything is all safe and nothing will go wrong, and I really hope you're right. But I think it's a very dangerous assumption. Hope for the best, plan for the worst.

cjohnson318 4y ago
You've moved the goalposts from, "we should assume that predictive text models are sentient," to, "we should be wary of programs explicitly designed to interact unpredictably with other computer systems," something that everyone from the Amish to the NSA to Cozy Bear would agree with.
- doctor_eval 4y ago
  What I said at the outset was, > Considering that we don’t know how the brain works so well, and we don’t understand why LLMs work so well, simply on the basis of their output I think the safest assumption is that these models do indeed have agency, or at least the capability of agency. These systems can already interact with others, it’s not moving the goalposts, it’s common knowledge. Anyone with access to the APIs can make it happen. Or are you now claiming we are just talking about one specific LLM and not LLMs generally? Anyway this discussion is fruitless, I’m out.