4 ms·

Its wild how this alwayyysss is the argument. Its just "oh so you think humans are special!" >:| and a gigantic "what-if" its a purely emotional provocation an

Its wild how this alwayyysss is the argument. Its just "oh so you think humans are special!" >:| and a gigantic "what-if" its a purely emotional provocation and a universe sized leap, not an argument for LLM's having intelligence or sentience. Anything could be anything, wowww! This goes back to what the other person was saying, "I cannot reason about what is going on behind the curtain, therefore..."

mewpmewp2 3y ago
Not arguing LLMs have sentience, but more so whether something that could be considered as "simplistic" as "statistics" could yield in a more complex result. Whether LLMs have intelligence depends on your definition of intelligence. Could connections of artificial neurons arranged in certain way as a result of training on data yield in human level intelligence?
- mjburgess 3y ago
  Always remember that ML is starting from training data, ie., from a very very very large number of (Prompt,Answer) pairs. Remember also, that companies like OpenAI are tracking what prompts fail and adding it to their datasets. So their initial data is ever-more just a record of questions-and-answers. Given this, we should expect that the vast majority of questions we have, of ChatGPT will be answered very very well. What has this to do with intelligence? Nothing at all. Intelligence is not answering questions correctly. It's what you use when you dont have the answers and arent even clear on the question.
  - MacsHeadroom 3y ago
    This is not how ML works generally nor LLMs like ChatGPT specifically. What you're describing sounds like RLHF, which changes the style of responses and impacts things like refusals but does not add to a model's intelligence (in fact it reduces model intelligence). An LLM's intelligence comes from pretraining in which there are no prompts, or answers, only corpus and perplexity.
    - mjburgess 3y ago
      You can map the weights to an uncompressed (Q,A) space, and then map them back again to weight-space -- all without loss of information. The actual domain space they were compressed from is irrelevant; their values are equivalent.
      - MacsHeadroom 3y ago
        Q & A are not part of pretraining.
whycombinater 3y ago
All knowledge/predictions are encoded as a chain of probabilities that something is true, otherwise, what else is it? My brain calculates 0.8 * 0.3 * 0.5 * 0.6 in order to make a 3-pointer, but Michael Jordan's brain ... well his mitochondria does a triple back flip and inverts the voltage into a tachyon particle.

mewpmewp2 3y ago

Not arguing LLMs have sentience, but more so whether something that could be considered as "simplistic" as "statistics" could yield in a more complex result. Whether LLMs have intelligence depends on your definition of intelligence. Could connections of artificial neurons arranged in certain way as a result of training on data yield in human level intelligence?

mjburgess 3y ago
Always remember that ML is starting from training data, ie., from a very very very large number of (Prompt,Answer) pairs. Remember also, that companies like OpenAI are tracking what prompts fail and adding it to their datasets. So their initial data is ever-more just a record of questions-and-answers. Given this, we should expect that the vast majority of questions we have, of ChatGPT will be answered very very well. What has this to do with intelligence? Nothing at all. Intelligence is not answering questions correctly. It's what you use when you dont have the answers and arent even clear on the question.
- MacsHeadroom 3y ago
  This is not how ML works generally nor LLMs like ChatGPT specifically. What you're describing sounds like RLHF, which changes the style of responses and impacts things like refusals but does not add to a model's intelligence (in fact it reduces model intelligence). An LLM's intelligence comes from pretraining in which there are no prompts, or answers, only corpus and perplexity.
  - mjburgess 3y ago
    You can map the weights to an uncompressed (Q,A) space, and then map them back again to weight-space -- all without loss of information. The actual domain space they were compressed from is irrelevant; their values are equivalent.
    - MacsHeadroom 3y ago
      Q & A are not part of pretraining.

whycombinater 3y ago

All knowledge/predictions are encoded as a chain of probabilities that something is true, otherwise, what else is it? My brain calculates 0.8 * 0.3 * 0.5 * 0.6 in order to make a 3-pointer, but Michael Jordan's brain ... well his mitochondria does a triple back flip and inverts the voltage into a tachyon particle.