5 ms·

> but stringing words together isn't thought Doesn't it sound like "thought"? Is there a significant difference between behaving like thought and being thought

> but stringing words together isn't thought Doesn't it sound like "thought"? Is there a significant difference between behaving like thought and being thought (whatever that means)?

jrockway 4y ago
There is a significant difference but I don't know if I'm doing it! That's another HN trope; "the execs get private jets for spouting buzzwords and all I get for programming is more work!" Maybe the sounding-like-you're-thinking is the important job?
incone123 4y ago
Depends on the domain. I'm not going to use a LLM as my attorney just yet, but if I wanted help writing a fiction book then I'd consider it.
- swader999 4y ago
  You'll probably want your attorney to use it appropriately to be more effective and lower their costs though.
  - incone123 4y ago
    Yes. I'm competent to oversee some fiction. And I hope my attorney is competent to review what the LLM writes for them, but if all else fails I'll sue the attorney rather than enter the uncharted territory of suing the creator of an LLM.
skupig 4y ago
No, it doesn't. Having grammar down but a very loose grasp of semantics, with no apparent intent or personality, doesn't look like thought.
- dwringer 4y ago
  I used to do a lot of experimenting with the bots on character.ai when the site was a lot faster and smoother. Simulated personalities could absolutely be shaped into the bots by trying to have as many "productive" discussions as possible while aggressively upvoting/downvoting every possibility for each generated response to punish the appearance of certain tokens, styles, or lines of thinking. After a while it led to some really fascinating emergent results, particularly then if the characters' dialogues were pasted back and forth so bots with different personalities could have discussions (which could then be used for another round of voting/training to strengthen their positions).
  - than3 4y ago
    It doesn't strengthen positions, it fits until it doesn't, and then it has runaway. In my opinion, this is a really stupid pursuit. Bots aren't people, and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work.
    - dwringer 4y ago
      > It doesn't strengthen positions, it fits until it doesn't At that point I'd restart the conversation, after several dozen a distinct simulation of a personality would start to emerge. > Bots aren't people I certainly didn't claim they are people or that they're real personalities. > and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work. I don't follow. It was a creative writing exercise using stochastically generated feedback that models a response using natural language. It was for entertainment, and it was successful. This could be used to aid authors in writing characters for a book or TV show, for instance. I don't argue for a second that it's any substitute for a live human author to make the simulated personalities into something more real. They're nothing but sound and fury on their own without the prompts and the voting provided by users. EDIT: Here[0]'s a link to my favorite output. One bot was a text-rpg-style bot I developed based on the movie Event Horizon, and the other was a naturalist philosopher monkey named Monke. I didn't even steer the responses in this case, this was just straight down after I'd spent a lot of time training both characters. The input was all generated by a bot I had running in a second window. One of the things I find most interesting is that the bots mirror each others' simulated [non]personalities back at each other frequently (which makes sense as they're both fundamentally the same LLM underneath). [0] https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc
colonCapitalDee 4y ago
I think there is. As a thought experiment, imagine an alien supercomputer with nigh infinite RAM. Alien programmers have used this computer to create a chatbot using a simple decision tree. Because the computer has so much RAM, this chatbot's decision tree is comprehensive enough for the chatbot to masquerade as a human. This chatbot could pass a Turing test with ease, and yet its "brain" is nothing more than a series of if statements.
- msla 4y ago
  These thought experiments rest on the assumption that human brains are magically complex such that being "just" anything we can understand is inherently lesser. Searle's Chinese Room experiment falls for the same fallacy, more-or-less, plus the fallacy that "human plus book" isn't a single system but "huge number of individual neurons" somehow is.
- than3 4y ago
  Well there's quite a lot here that's just flat wrong, but you get the gist of some of it. To clarify, Computers can't pass turing tests because the answer is divergent, or in other words has multiple possible answers. There may be some incompleteness issues as well. Computers require determinism as a system's property to function and do work. It can't do work if there is more than one path, or single series of steps given the same state inputs.
  - Thiez 4y ago
    I see no reason why nondeterminism is a requirement for consciousness. There are multiple possible answers to questions "Would you like a cup of tea?" (such as "Yes", "No", and "Only if you put some sugar in it") but at any point in time I'm only going to give one of those answers, not all of them, and I like to think that if you could rewind time to just after asking the question, I would give the same answer every time. Besides, computers can have a source of true randomness. This means they are no longer a Turing Machine, but they never were anyway (no infinite tape).
    - 4y ago

> but stringing words together isn't thought Doesn't it sound like "thought"? Is there a significant difference between behaving like thought and being thought (whatever that means)?

jrockway 4y ago

There is a significant difference but I don't know if I'm doing it! That's another HN trope; "the execs get private jets for spouting buzzwords and all I get for programming is more work!" Maybe the sounding-like-you're-thinking is the important job?

incone123 4y ago

Depends on the domain. I'm not going to use a LLM as my attorney just yet, but if I wanted help writing a fiction book then I'd consider it.

swader999 4y ago
You'll probably want your attorney to use it appropriately to be more effective and lower their costs though.
- incone123 4y ago
  Yes. I'm competent to oversee some fiction. And I hope my attorney is competent to review what the LLM writes for them, but if all else fails I'll sue the attorney rather than enter the uncharted territory of suing the creator of an LLM.

skupig 4y ago

No, it doesn't. Having grammar down but a very loose grasp of semantics, with no apparent intent or personality, doesn't look like thought.

dwringer 4y ago
I used to do a lot of experimenting with the bots on character.ai when the site was a lot faster and smoother. Simulated personalities could absolutely be shaped into the bots by trying to have as many "productive" discussions as possible while aggressively upvoting/downvoting every possibility for each generated response to punish the appearance of certain tokens, styles, or lines of thinking. After a while it led to some really fascinating emergent results, particularly then if the characters' dialogues were pasted back and forth so bots with different personalities could have discussions (which could then be used for another round of voting/training to strengthen their positions).
- than3 4y ago
  It doesn't strengthen positions, it fits until it doesn't, and then it has runaway. In my opinion, this is a really stupid pursuit. Bots aren't people, and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work.
  - dwringer 4y ago
    > It doesn't strengthen positions, it fits until it doesn't At that point I'd restart the conversation, after several dozen a distinct simulation of a personality would start to emerge. > Bots aren't people I certainly didn't claim they are people or that they're real personalities. > and you would think this endeavor was poorly thought through if you knew how communication and psychology fundamentally work. I don't follow. It was a creative writing exercise using stochastically generated feedback that models a response using natural language. It was for entertainment, and it was successful. This could be used to aid authors in writing characters for a book or TV show, for instance. I don't argue for a second that it's any substitute for a live human author to make the simulated personalities into something more real. They're nothing but sound and fury on their own without the prompts and the voting provided by users. EDIT: Here[0]'s a link to my favorite output. One bot was a text-rpg-style bot I developed based on the movie Event Horizon, and the other was a naturalist philosopher monkey named Monke. I didn't even steer the responses in this case, this was just straight down after I'd spent a lot of time training both characters. The input was all generated by a bot I had running in a second window. One of the things I find most interesting is that the bots mirror each others' simulated [non]personalities back at each other frequently (which makes sense as they're both fundamentally the same LLM underneath). [0] https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc https://c.ai/p/eIac3PkU7Gv4KWFw-Ul3RWBCvGZeYxp1aV-W6XKYCUc

colonCapitalDee 4y ago

I think there is. As a thought experiment, imagine an alien supercomputer with nigh infinite RAM. Alien programmers have used this computer to create a chatbot using a simple decision tree. Because the computer has so much RAM, this chatbot's decision tree is comprehensive enough for the chatbot to masquerade as a human. This chatbot could pass a Turing test with ease, and yet its "brain" is nothing more than a series of if statements.

msla 4y ago
These thought experiments rest on the assumption that human brains are magically complex such that being "just" anything we can understand is inherently lesser. Searle's Chinese Room experiment falls for the same fallacy, more-or-less, plus the fallacy that "human plus book" isn't a single system but "huge number of individual neurons" somehow is.
than3 4y ago
Well there's quite a lot here that's just flat wrong, but you get the gist of some of it. To clarify, Computers can't pass turing tests because the answer is divergent, or in other words has multiple possible answers. There may be some incompleteness issues as well. Computers require determinism as a system's property to function and do work. It can't do work if there is more than one path, or single series of steps given the same state inputs.
- Thiez 4y ago
  I see no reason why nondeterminism is a requirement for consciousness. There are multiple possible answers to questions "Would you like a cup of tea?" (such as "Yes", "No", and "Only if you put some sugar in it") but at any point in time I'm only going to give one of those answers, not all of them, and I like to think that if you could rewind time to just after asking the question, I would give the same answer every time. Besides, computers can have a source of true randomness. This means they are no longer a Turing Machine, but they never were anyway (no infinite tape).
  - 4y ago