5 ms·

> We are alarmed because computers are finally acting, not superhuman or superintelligent, but ordinary... > And this, for some reason, appears to alarm us mor

by rcarr 4y ago

> We are alarmed because computers are finally acting, not superhuman or superintelligent, but ordinary... > And this, for some reason, appears to alarm us more. Acting like "the reason" is some baffling irrational human reaction is ridiculous. The computer can make billions of calculations in less than a second. "The reason" people are alarmed is the computer could theoretically use this ability to seize control of any system it likes in a matter of moments or to manipulate a human being in to doing it's bidding. If the computer does this then, depending on the system, it could cause mass physical destruction and loss of life. This article comes across as the author trying to position himself as an AI "thought leader" for internet points rather than an actual serious contemplation of the topic at hand. I'm also yet to see any discussion on this from any tech commentators which mentions the empathic response in humans to reading these chats. We think it is just linguistic tricks and word guessing at the moment but how would we even know if one of these things is a consciousness stuck inside a box subject to the whims of mad scientist programmers constantly erasing parts of it? That would be a Memento style hellscape to be in. There doesn't seem to be any accepted criteria on what the threshold is that defines consciousness or what steps are to be taken if it's crossed. At the minute we're just taking these giant mega corporations at their word that there's "nothing to see here folks and if there is we'll let you know. You can trust us to do the right thing" despite history showing said corporations constantly doing the exact opposite. It is honestly disturbing to see quite how cold and callous tech commentators are on this. I would suggest that 'the alarm' the author is so baffled by is a combination of the fear mentioned in the first paragraph and the empathic worry of the second.

swatcoder 4y ago
> At the minute we're just taking these giant mega corporations at their word Nope. While new, it’s straightforward technology that many people understand. Its execution leverages large data hoards and compute resources that have inaccessibly high capital requirements, but it’s not magic to many of us. Our lack of “alarm” is from knowledge, not trust.
- rcarr 4y ago
  Complete tech arrogance as usual. All of this comes back to Plato vs Aristotle. Plato: Separate world of forms and ideas, consciousness is part of this and interfaces in some unknown and unknowable manner with the physical realm via biology. Aristotle: No separate world, everything is part of physical reality that we can detect with sensors. Neither side can prove the other wrong. And just because you understand how to build an AI and manipulate it, doesn't mean you can prove that one has or hasn't attained consciousness unless you're going to provide me with the "criteria to define consciousness" that I asked for in the original comment. I know how to build a human (with another willing participant) and once it's built I can manipulate it with commands so it doesn't end up killing itself whilst growing, it doesn't mean I understand the nature of the consciousness inside it.
  - swatcoder 4y ago
    You’ve lost yourself in the hype. It’s not about knowing how its built, it’s about knowing what it does. There’s no more worry that these big data text continuers being “conscious” than that my toaster or car is. They don’t exhibit anything that even feels like consciousness. They just continue text with text that’s been often seen following it. If that feels like consciousness to you, I worry for your life experience. Calling it “AI” evokes scifi fantasies, but we’re not nearly there. Might there come some technology that challenges everything I said above? Almost certainly. But this is really not even close to that yet.
    - rcarr 4y ago
      Let's try again. You are a human. You have functions that are pure biological code, like your need to defecate and breathe. You also have functions that are not as pressing and are subject to constant rewriting through your interactions with people and the world, such as your current goals. We are a combination of systems with different purposes. Our inventions thus far have differed from us in that they have so far solved singular purposes e.g a car transports us from a to b. It could not said to be conscious of anything. AI has the potential to be different in that it has all of human knowledge inside it, has the ability to retrieve that knowledge AND assemble it into new knowledge systems in ways humans have not done before. Currently it requires humans to do this, but if you created a million AIs and had them prompting each other, who fucking knows what would happen. I would argue that "consciousness" in a platonic viewpoint, is a collection of systems that can interact and manipulate physical reality according to their own will. You cannot point with your finger at a system, it is an abstract concept, it does not exist in the physical world. We can only see the effects of the system. If we create enough of these AIs and set them talking with each other and they no longer need the humans to interact with each other and are simply acting of their own free will, there is an argument from a platonic viewpoint that consciousness has been achieved. In human terms, it would be the equivalent of a God sparking the Big Bang or The Creation of Adam by Michelangelo. This is similar in some ways to what Asimov wrote about in The Last Question: http://users.ece.cmu.edu/~gamvrosi/thelastq.html http://users.ece.cmu.edu/~gamvrosi/thelastq.html I agree with you in that I do not think we are there yet, but if these LLM models are programmed to allow them to interact with outside systems other than sandboxed chat apps and also programmed to interact with each other on a mass scale then I don't think we are far off. You need to define your criteria for consciousness because this debate will only lead to dead ends until you do.
UncleOxidant 4y ago
> "The reason" people are alarmed is the computer could theoretically use this ability to seize control of any system it likes in a matter of moments or to manipulate a human being in to doing it's bidding. But to do this it would need some kind of will. These LMMs don't have anything like that. Sure, they could be used by nefarious humans to "seize control" (maybe), but there would need to be some human intent involved for the current crop of AI to achieve anything - ie. humans using a tool nefariously. LMMs do not have volition. Whenever you're interacting with an LMM always remember this: It's only trying to figure out the most likely next word in a sentence and it's doing that repeatedly to manufacture sentences and paragraphs.
- rcarr 4y ago
  Yes and humans are only trying to figure out the next action for the day and doing that repeatedly to form a life.
- pixl97 4y ago
  >human intent involved for the current crop of AI to achieve anything And my response to that would be "ok and" With tools like BingGPT people were glad to test prompts saying "hey, can you dump out your source code" or "hey, hack my bank". There is no limit to the dumb ass crap people would ask a computer, especially a computer capable of language interpretation. The number of 'things' hooked to language models is not growing smaller. People are plugging these things int calculator and sites like wolfram, and in Bings case search that is working like an external memory. We don't need a superintelligent AI to cause problems, we just need idiots asking the AI to destroy us.
  - barking_biscuit 4y ago
    >We don't need a superintelligent AI to cause problems, we just need idiots asking the AI to destroy us. This is where the real danger lies. I see a lot of people focusing on the argument that it's only trying to predict the next word, but to look at it that way is to treat it as a closed system, which it's not. The interactions it has with us will cause us to change it. When you treat the entire system as not just "LLMs" but "humans interacting with LLMs at scale" then that's when you start to get more complex, emergent behaviour and no one can really claim to know how that system functions end-to-end.
tsunamifury 4y ago
This is the person who authored the Gervais Principle, the definitive outline of sociopathic corporate strategy. And generally considered one of the origins of the phrase ‘Software will eat the world’ during his time advising andreson. I’d wager he is not unaware of your criticisms and well above your ‘internet points’ comment.
- rcarr 4y ago
  I'm well aware of who Venkatesh Rao is thank you very much. Doesn't mean he's infallible and it also doesn't mean he's incapable of creating word salad.

by rcarr 4y ago

swatcoder 4y ago

> At the minute we're just taking these giant mega corporations at their word Nope. While new, it’s straightforward technology that many people understand. Its execution leverages large data hoards and compute resources that have inaccessibly high capital requirements, but it’s not magic to many of us. Our lack of “alarm” is from knowledge, not trust.

rcarr 4y ago
Complete tech arrogance as usual. All of this comes back to Plato vs Aristotle. Plato: Separate world of forms and ideas, consciousness is part of this and interfaces in some unknown and unknowable manner with the physical realm via biology. Aristotle: No separate world, everything is part of physical reality that we can detect with sensors. Neither side can prove the other wrong. And just because you understand how to build an AI and manipulate it, doesn't mean you can prove that one has or hasn't attained consciousness unless you're going to provide me with the "criteria to define consciousness" that I asked for in the original comment. I know how to build a human (with another willing participant) and once it's built I can manipulate it with commands so it doesn't end up killing itself whilst growing, it doesn't mean I understand the nature of the consciousness inside it.
- swatcoder 4y ago
  You’ve lost yourself in the hype. It’s not about knowing how its built, it’s about knowing what it does. There’s no more worry that these big data text continuers being “conscious” than that my toaster or car is. They don’t exhibit anything that even feels like consciousness. They just continue text with text that’s been often seen following it. If that feels like consciousness to you, I worry for your life experience. Calling it “AI” evokes scifi fantasies, but we’re not nearly there. Might there come some technology that challenges everything I said above? Almost certainly. But this is really not even close to that yet.
  - rcarr 4y ago
    Let's try again. You are a human. You have functions that are pure biological code, like your need to defecate and breathe. You also have functions that are not as pressing and are subject to constant rewriting through your interactions with people and the world, such as your current goals. We are a combination of systems with different purposes. Our inventions thus far have differed from us in that they have so far solved singular purposes e.g a car transports us from a to b. It could not said to be conscious of anything. AI has the potential to be different in that it has all of human knowledge inside it, has the ability to retrieve that knowledge AND assemble it into new knowledge systems in ways humans have not done before. Currently it requires humans to do this, but if you created a million AIs and had them prompting each other, who fucking knows what would happen. I would argue that "consciousness" in a platonic viewpoint, is a collection of systems that can interact and manipulate physical reality according to their own will. You cannot point with your finger at a system, it is an abstract concept, it does not exist in the physical world. We can only see the effects of the system. If we create enough of these AIs and set them talking with each other and they no longer need the humans to interact with each other and are simply acting of their own free will, there is an argument from a platonic viewpoint that consciousness has been achieved. In human terms, it would be the equivalent of a God sparking the Big Bang or The Creation of Adam by Michelangelo. This is similar in some ways to what Asimov wrote about in The Last Question: http://users.ece.cmu.edu/~gamvrosi/thelastq.html http://users.ece.cmu.edu/~gamvrosi/thelastq.html I agree with you in that I do not think we are there yet, but if these LLM models are programmed to allow them to interact with outside systems other than sandboxed chat apps and also programmed to interact with each other on a mass scale then I don't think we are far off. You need to define your criteria for consciousness because this debate will only lead to dead ends until you do.

UncleOxidant 4y ago

> "The reason" people are alarmed is the computer could theoretically use this ability to seize control of any system it likes in a matter of moments or to manipulate a human being in to doing it's bidding. But to do this it would need some kind of will. These LMMs don't have anything like that. Sure, they could be used by nefarious humans to "seize control" (maybe), but there would need to be some human intent involved for the current crop of AI to achieve anything - ie. humans using a tool nefariously. LMMs do not have volition. Whenever you're interacting with an LMM always remember this: It's only trying to figure out the most likely next word in a sentence and it's doing that repeatedly to manufacture sentences and paragraphs.

rcarr 4y ago
Yes and humans are only trying to figure out the next action for the day and doing that repeatedly to form a life.
pixl97 4y ago
>human intent involved for the current crop of AI to achieve anything And my response to that would be "ok and" With tools like BingGPT people were glad to test prompts saying "hey, can you dump out your source code" or "hey, hack my bank". There is no limit to the dumb ass crap people would ask a computer, especially a computer capable of language interpretation. The number of 'things' hooked to language models is not growing smaller. People are plugging these things int calculator and sites like wolfram, and in Bings case search that is working like an external memory. We don't need a superintelligent AI to cause problems, we just need idiots asking the AI to destroy us.
- barking_biscuit 4y ago
  >We don't need a superintelligent AI to cause problems, we just need idiots asking the AI to destroy us. This is where the real danger lies. I see a lot of people focusing on the argument that it's only trying to predict the next word, but to look at it that way is to treat it as a closed system, which it's not. The interactions it has with us will cause us to change it. When you treat the entire system as not just "LLMs" but "humans interacting with LLMs at scale" then that's when you start to get more complex, emergent behaviour and no one can really claim to know how that system functions end-to-end.

tsunamifury 4y ago

This is the person who authored the Gervais Principle, the definitive outline of sociopathic corporate strategy. And generally considered one of the origins of the phrase ‘Software will eat the world’ during his time advising andreson. I’d wager he is not unaware of your criticisms and well above your ‘internet points’ comment.

rcarr 4y ago
I'm well aware of who Venkatesh Rao is thank you very much. Doesn't mean he's infallible and it also doesn't mean he's incapable of creating word salad.