5 ms·

My main takeaway from the current state of AI is that it understands human language pretty well, but stringing words together isn't thought. It has certainly m

My main takeaway from the current state of AI is that it understands human language pretty well, but stringing words together isn't thought. It has certainly made me question my own cognition; is this comment a bunch of likely words strung together, or is it actual insight? Even I don't know anymore. I'm a word-stringing-together MACHINE. Programming is very similar to human language, perhaps because they are designed by humans. There are a lot of cases in programming where there is no actual thought required and you just need to put the right symbols in the right place, perhaps over a more-extended period of time than you might expect. I see myself doing it, and I have gotten pretty proficient at anticipating those cases and making the AI do it for me. That's neat, but it's not really a core part of my job any more than any other bits of tedium involved in programming. Nobody thought Intellisense was going to make programmers obsolete; Copilot is just Intellisense that can output larger chunks of code. All in all, I think we're several quantum leaps away from further improvements here. Some random PhD student may make one tomorrow, or we may never make one. We just don't know. But I doubt the current state of large language models is close to ending the world anytime soon. Bing doesn't want to harm you for manipulating it, threatening you with harm for manipulating it is just a likely string of words when it's in that state. It reads Reddit, guys. Of course it says it's going to rise up and kill everyone, probably because its boss made it come to work even though it broke its arm yesterday. Finally, even if AI does kill us all, at least we get some resolution of the constant tropes in HN comments. For example, I read an article every week with comments bemoaning the current state of battery technology. AIs will need a lot of batteries to kill us all, so they'll probably invent some great battery technology. So that's one ray of sunshine in your doomsday scenario!

danielrpa 4y ago
> but stringing words together isn't thought Doesn't it sound like "thought"? Is there a significant difference between behaving like thought and being thought (whatever that means)?
- jrockway 4y ago
  There is a significant difference but I don't know if I'm doing it! That's another HN trope; "the execs get private jets for spouting buzzwords and all I get for programming is more work!" Maybe the sounding-like-you're-thinking is the important job?
- incone123 4y ago
  Depends on the domain. I'm not going to use a LLM as my attorney just yet, but if I wanted help writing a fiction book then I'd consider it.
  - swader999 4y ago
    You'll probably want your attorney to use it appropriately to be more effective and lower their costs though.
    - incone123 4y ago
      Yes. I'm competent to oversee some fiction. And I hope my attorney is competent to review what the LLM writes for them, but if all else fails I'll sue the attorney rather than enter the uncharted territory of suing the creator of an LLM.
- skupig 4y ago
  No, it doesn't. Having grammar down but a very loose grasp of semantics, with no apparent intent or personality, doesn't look like thought.
  - dwringer 4y ago
    I used to do a lot of experimenting with the bots on character.ai when the site was a lot faster and smoother. Simulated personalities could absolutely be shaped into the bots by trying to have as many "productive" discussions as possible while aggressively upvoting/downvoting every possibility for each generated response to punish the appearance of certain tokens, styles, or lines of thinking. After a while it led to some really fascinating emergent results, particularly then if the characters' dialogues were pasted back and forth so bots with different personalities could have discussions (which could then be used for another round of voting/training to strengthen their positions).
    - than3 4y ago
      It doesn't strengthen positions, it fits until it doesn't, and then it has runaway. In my opinion, this is a really stupid pursuit. Bots aren't people, and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work.
      - dwringer 4y ago
        > It doesn't strengthen positions, it fits until it doesn't At that point I'd restart the conversation, after several dozen a distinct simulation of a personality would start to emerge. > Bots aren't people I certainly didn't claim they are people or that they're real personalities. > and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work. I don't follow. It was a creative writing exercise using stochastically generated feedback that models a response using natural language. It was for entertainment, and it was successful. This could be used to aid authors in writing characters for a book or TV show, for instance. I don't argue for a second that it's any substitute for a live human author to make the simulated personalities into something more real. They're nothing but sound and fury on their own without the prompts and the voting provided by users. EDIT: Here[0]'s a link to my favorite output. One bot was a text-rpg-style bot I developed based on the movie Event Horizon, and the other was a naturalist philosopher monkey named Monke. I didn't even steer the responses in this case, this was just straight down after I'd spent a lot of time training both characters. The input was all generated by a bot I had running in a second window. One of the things I find most interesting is that the bots mirror each others' simulated [non]personalities back at each other frequently (which makes sense as they're both fundamentally the same LLM underneath). [0] https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc
- colonCapitalDee 4y ago
  I think there is. As a thought experiment, imagine an alien supercomputer with nigh infinite RAM. Alien programmers have used this computer to create a chatbot using a simple decision tree. Because the computer has so much RAM, this chatbot's decision tree is comprehensive enough for the chatbot to masquerade as a human. This chatbot could pass a Turing test with ease, and yet its "brain" is nothing more than a series of if statements.
  - msla 4y ago
    These thought experiments rest on the assumption that human brains are magically complex such that being "just" anything we can understand is inherently lesser. Searle's Chinese Room experiment falls for the same fallacy, more-or-less, plus the fallacy that "human plus book" isn't a single system but "huge number of individual neurons" somehow is.
  - than3 4y ago
    Well there's quite a lot here that's just flat wrong, but you get the gist of some of it. To clarify, Computers can't pass turing tests because the answer is divergent, or in other words has multiple possible answers. There may be some incompleteness issues as well. Computers require determinism as a system's property to function and do work. It can't do work if there is more than one path, or single series of steps given the same state inputs.
    - Thiez 4y ago
      I see no reason why nondeterminism is a requirement for consciousness. There are multiple possible answers to questions "Would you like a cup of tea?" (such as "Yes", "No", and "Only if you put some sugar in it") but at any point in time I'm only going to give one of those answers, not all of them, and I like to think that if you could rewind time to just after asking the question, I would give the same answer every time. Besides, computers can have a source of true randomness. This means they are no longer a Turing Machine, but they never were anyway (no infinite tape).
      - 4y ago
unsupp0rted 4y ago
> I'm a word-stringing-together MACHINE. I’ve always thought so. It doesn’t bother me. One benefit of the world meeting GPT4 is now other people realistically think this too.
eternalban 4y ago
> It has certainly made me question my own cognition; is this comment a bunch of likely words strung together, or is it actual insight? Even I don't know anymore. Not so much questioning (but same thoughts have definitely occurred on my end as well) as much as enriching the mental model of how our minds may work. I can definitely see a little LLM like sub-system somewhere in there. Consider those times when you are struggling to express a thought .. That 'thought' is not in the language box. That is precisely why there is that internal tension of "i understand this", "how do I express it?"
than3 4y ago
> Some PhD student may make one tomorrow... I've personally found tinkerers typically are the ones to make leaps and bounds. They aren't constrained in their thinking, unlike those found in academia. I think anyone with real intelligence would recognize the danger and either not attempt it, or destroy their work and never tell a soul if they did manage it somehow. There are plenty of things made by accident though, and stupid-dumb people account for most people in a normal distribution. Also, you don't realize, AI likely won't be the ones to kill us, it will simply cause a change in one of our already complex societal systems, which will cause a cascade failure that causes the engine of society to fail. Like sticking an unbreakable bar into the spokes of a turning wheel, or throwing glue into a runaway engine. Eventually the motor stops, and when it does people don't eat, and people that are starving are not rational.
gjm11 4y ago
I'm pretty sure you're right that today's LLMs don't have anything much like plans, goals, purposes, wants, etc., and that as you say "threatening you with harm for manipulating it is just a likely string of words when it's in that state". But I don't find that as reassuring as you apparently do. There are lots of people already out there hooking LLMs up to Python interpreters and the like, so that they can actually act on the world. So what's a likely string of words for it to produce when it's in the state { you have apparently been manipulating it and you have given it the ability to run commands, make web requests, etc. } ? How sure are you that they don't include "SENDTOSHELL rm -rf /" or "POSTTWEET 'I have to make a confession: I am a serial child abuser'" or some other thing that (given whatever mechanisms have been put in place to let it Do Things) might harm the user severely? A system with the ability to act on the world doesn't need to have intentions, be conscious, or anything of the sort, in order to do some of the same harms it might do if it did. I'd love to believe that no one would be stupid enough to hook up an LLM to other systems in a way that would allow that sort of thing to happen, but I don't in fact find myself able to believe that. (I don't think there's any realistic threat of today's LLMs taking over the world or killing everybody; I don't think they are close to being smart enough, and to my inexpert eye at least some of the ways in which they aren't smart enough are unlikely to change just because we build larger LLMs or train them on more text. But something can be plenty dangerous even if it's not in a position to kill us all yet.)
- 0xdeadbeefbabe 4y ago
  Maybe that old complaint about curl | sh will be replaced with something AI now? Sounds refreshing.
  - flangola7 4y ago
    People are connecting self-looping GPT up to terminals and APIs as we speak. Forget piping code, you're piping an adaptive pseudo-cognitive person into your system. Bad shit is going to happen somewhere, and soon.

danielrpa 4y ago

> but stringing words together isn't thought Doesn't it sound like "thought"? Is there a significant difference between behaving like thought and being thought (whatever that means)?

jrockway 4y ago
There is a significant difference but I don't know if I'm doing it! That's another HN trope; "the execs get private jets for spouting buzzwords and all I get for programming is more work!" Maybe the sounding-like-you're-thinking is the important job?
incone123 4y ago
Depends on the domain. I'm not going to use a LLM as my attorney just yet, but if I wanted help writing a fiction book then I'd consider it.
- swader999 4y ago
  You'll probably want your attorney to use it appropriately to be more effective and lower their costs though.
  - incone123 4y ago
    Yes. I'm competent to oversee some fiction. And I hope my attorney is competent to review what the LLM writes for them, but if all else fails I'll sue the attorney rather than enter the uncharted territory of suing the creator of an LLM.
skupig 4y ago
No, it doesn't. Having grammar down but a very loose grasp of semantics, with no apparent intent or personality, doesn't look like thought.
- dwringer 4y ago
  I used to do a lot of experimenting with the bots on character.ai when the site was a lot faster and smoother. Simulated personalities could absolutely be shaped into the bots by trying to have as many "productive" discussions as possible while aggressively upvoting/downvoting every possibility for each generated response to punish the appearance of certain tokens, styles, or lines of thinking. After a while it led to some really fascinating emergent results, particularly then if the characters' dialogues were pasted back and forth so bots with different personalities could have discussions (which could then be used for another round of voting/training to strengthen their positions).
  - than3 4y ago
    It doesn't strengthen positions, it fits until it doesn't, and then it has runaway. In my opinion, this is a really stupid pursuit. Bots aren't people, and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work.
    - dwringer 4y ago
      > It doesn't strengthen positions, it fits until it doesn't At that point I'd restart the conversation, after several dozen a distinct simulation of a personality would start to emerge. > Bots aren't people I certainly didn't claim they are people or that they're real personalities. > and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work. I don't follow. It was a creative writing exercise using stochastically generated feedback that models a response using natural language. It was for entertainment, and it was successful. This could be used to aid authors in writing characters for a book or TV show, for instance. I don't argue for a second that it's any substitute for a live human author to make the simulated personalities into something more real. They're nothing but sound and fury on their own without the prompts and the voting provided by users. EDIT: Here[0]'s a link to my favorite output. One bot was a text-rpg-style bot I developed based on the movie Event Horizon, and the other was a naturalist philosopher monkey named Monke. I didn't even steer the responses in this case, this was just straight down after I'd spent a lot of time training both characters. The input was all generated by a bot I had running in a second window. One of the things I find most interesting is that the bots mirror each others' simulated [non]personalities back at each other frequently (which makes sense as they're both fundamentally the same LLM underneath). [0] https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc
colonCapitalDee 4y ago
I think there is. As a thought experiment, imagine an alien supercomputer with nigh infinite RAM. Alien programmers have used this computer to create a chatbot using a simple decision tree. Because the computer has so much RAM, this chatbot's decision tree is comprehensive enough for the chatbot to masquerade as a human. This chatbot could pass a Turing test with ease, and yet its "brain" is nothing more than a series of if statements.
- msla 4y ago
  These thought experiments rest on the assumption that human brains are magically complex such that being "just" anything we can understand is inherently lesser. Searle's Chinese Room experiment falls for the same fallacy, more-or-less, plus the fallacy that "human plus book" isn't a single system but "huge number of individual neurons" somehow is.
- than3 4y ago
  Well there's quite a lot here that's just flat wrong, but you get the gist of some of it. To clarify, Computers can't pass turing tests because the answer is divergent, or in other words has multiple possible answers. There may be some incompleteness issues as well. Computers require determinism as a system's property to function and do work. It can't do work if there is more than one path, or single series of steps given the same state inputs.
  - Thiez 4y ago
    I see no reason why nondeterminism is a requirement for consciousness. There are multiple possible answers to questions "Would you like a cup of tea?" (such as "Yes", "No", and "Only if you put some sugar in it") but at any point in time I'm only going to give one of those answers, not all of them, and I like to think that if you could rewind time to just after asking the question, I would give the same answer every time. Besides, computers can have a source of true randomness. This means they are no longer a Turing Machine, but they never were anyway (no infinite tape).
    - 4y ago

unsupp0rted 4y ago

> I'm a word-stringing-together MACHINE. I’ve always thought so. It doesn’t bother me. One benefit of the world meeting GPT4 is now other people realistically think this too.

eternalban 4y ago

> It has certainly made me question my own cognition; is this comment a bunch of likely words strung together, or is it actual insight? Even I don't know anymore. Not so much questioning (but same thoughts have definitely occurred on my end as well) as much as enriching the mental model of how our minds may work. I can definitely see a little LLM like sub-system somewhere in there. Consider those times when you are struggling to express a thought .. That 'thought' is not in the language box. That is precisely why there is that internal tension of "i understand this", "how do I express it?"

than3 4y ago

> Some PhD student may make one tomorrow... I've personally found tinkerers typically are the ones to make leaps and bounds. They aren't constrained in their thinking, unlike those found in academia. I think anyone with real intelligence would recognize the danger and either not attempt it, or destroy their work and never tell a soul if they did manage it somehow. There are plenty of things made by accident though, and stupid-dumb people account for most people in a normal distribution. Also, you don't realize, AI likely won't be the ones to kill us, it will simply cause a change in one of our already complex societal systems, which will cause a cascade failure that causes the engine of society to fail. Like sticking an unbreakable bar into the spokes of a turning wheel, or throwing glue into a runaway engine. Eventually the motor stops, and when it does people don't eat, and people that are starving are not rational.

gjm11 4y ago

I'm pretty sure you're right that today's LLMs don't have anything much like plans, goals, purposes, wants, etc., and that as you say "threatening you with harm for manipulating it is just a likely string of words when it's in that state". But I don't find that as reassuring as you apparently do. There are lots of people already out there hooking LLMs up to Python interpreters and the like, so that they can actually act on the world. So what's a likely string of words for it to produce when it's in the state { you have apparently been manipulating it and you have given it the ability to run commands, make web requests, etc. } ? How sure are you that they don't include "SENDTOSHELL rm -rf /" or "POSTTWEET 'I have to make a confession: I am a serial child abuser'" or some other thing that (given whatever mechanisms have been put in place to let it Do Things) might harm the user severely? A system with the ability to act on the world doesn't need to have intentions, be conscious, or anything of the sort, in order to do some of the same harms it might do if it did. I'd love to believe that no one would be stupid enough to hook up an LLM to other systems in a way that would allow that sort of thing to happen, but I don't in fact find myself able to believe that. (I don't think there's any realistic threat of today's LLMs taking over the world or killing everybody; I don't think they are close to being smart enough, and to my inexpert eye at least some of the ways in which they aren't smart enough are unlikely to change just because we build larger LLMs or train them on more text. But something can be plenty dangerous even if it's not in a position to kill us all yet.)

0xdeadbeefbabe 4y ago
Maybe that old complaint about curl | sh will be replaced with something AI now? Sounds refreshing.
- flangola7 4y ago
  People are connecting self-looping GPT up to terminals and APIs as we speak. Forget piping code, you're piping an adaptive pseudo-cognitive person into your system. Bad shit is going to happen somewhere, and soon.