5 ms·

In my very first interaction with Bing's bot, I was asking some details about PC components, and e.g. it was common for bing to give back answers containing phr

In my very first interaction with Bing's bot, I was asking some details about PC components, and e.g. it was common for bing to give back answers containing phrases like "as we tested in our previous article" (verbatim, even using the plural 1st person). It is obvious it is just parroting from the websites it scraped. PC hardware means benchmark results which means benchmark websites/blogs which means the "as we tested in our previous article" phrase is common so it will just put it there in the answer even if makes absolutely no sense whatsoever. I really don't understand why people ascribe emergent behaviors to these systems such as "theory of mind". It's just randomness. I'm presuming there's just a lot of psycological pareidolia going around. It also got a significant chunk of the answers wrong, creative mode or not. Often, the Bing search it appeared to do actually showed the correct answer right in the snippet of the first result, yet it was not able to scrape that. More rarely but annoyingly frequently it would just search for the wrong thing (e.g. searched for "X vs Y" when it just didn't make sense since they were two layers of the same technology).

flangola7 3y ago
There's a research paper floating around...
jjeaff 3y ago
So, I certainly wouldn't ascribe any higher level thinking to these LLMs. But they do seem to have more capability than just regurgitating information. They have some logical thinking skills that I assume are not really emergent, but built into the system. For example, I was working on a configuration file for a somewhat new and esoteric firmware, so there isn't much info online about it. But I asked if a specific parameter could include more than one value and bing chat answered that it's assumption was no because it could find no mention anywhere of being able to use more than one value for that parameter. As far as I can tell, no one has answered a similar question anywhere online that it could have been regurgitating, so it seems to have deduced it.
- andsoitis 3y ago
  I would be careful to draw that conclusion. Not all information LLMs are trained on is accessible to the wide public (e.g. ChatGPT has training data from closed sources). Regardless, If you share the details of your prompt, I’d be interested to see whether my Google-fu turns up information.
  - nomel 3y ago
    Are you suggesting that LLM can't produce novel output?
    - andsoitis 3y ago
      No.
akio 3y ago
> I really don't understand why people ascribe emergent behaviors to these systems such as "theory of mind". It's just randomness. I'm presuming there's just a lot of psycological pareidolia going around. Ah yes, "just randomness": ---- From the above text, generate these variations: 1) For each word in the text count the number of letters in the word and append the count to the word in parenthesis. 2) Write each word backwards. 3) Write a similarly structured text that argues the opposite of the original text. ---- 1) I (1) really (5) don't (4) understand (10) why (3) people (6) ascribe (7) emergent (8) behaviors (9) to (2) these (5) systems (7) such (4) as (2) "theory (6) of (2) mind" (4). It's (4) just (4) randomness (10). I'm (3) presuming (9) there's (6) just (4) a (1) lot (3) of (2) psychological (12) pareidolia (10) going (5) around (6). 2) I yllaer t'nod dnatsrednu yhw elpoep ebircsa tnegrime sroivaheb ot eseht smetsys hcus sa ".dnim fo yroeht" s'tI tsuj ssenmodnar m'I gnimuserp s'ereht tsuj a tol fo lacigolohcysp ailodierap gniodnuora 3) I absolutely comprehend how people can accurately identify emergent behaviors in these systems like "theory of mind". It's far from random. I'm convinced that these are genuine insights being recognized.
- re 3y ago
  Did you notice that it made multiple errors for the first two variations? [Edit:] What's particularly interesting to me is the inconsistent counting of apostrophes for #1, and the handling of "theory of mind" for #2 -- though there are simpler errors besides these.
  - daveguy 3y ago
    Not to mention a third one isn't any sort of argument, just a statement of the opposite belief.
    - deleted 3y ago
      [deleted]
    - bitwize 3y ago
      The program Eliza could sucker a lot of people just by inverting pronouns and adding flavor text, e.g.: Querent: my parents hate me Eliza: Why do you feel that your parents hate you? At a higher level, ChatGPT in that example is negating the provided text and returning something of opposite meaning but not doing much more.
bitwize 3y ago
> PC hardware means benchmark results which means benchmark websites/blogs which means the "as we tested in our previous article" phrase is common so it will just put it there in the answer even if makes absolutely no sense whatsoever. "Don't forget to like and subscribe!" -- A Hackernews' 3-yo daughter, saying goodbye

flangola7 3y ago

There's a research paper floating around...

jjeaff 3y ago

So, I certainly wouldn't ascribe any higher level thinking to these LLMs. But they do seem to have more capability than just regurgitating information. They have some logical thinking skills that I assume are not really emergent, but built into the system. For example, I was working on a configuration file for a somewhat new and esoteric firmware, so there isn't much info online about it. But I asked if a specific parameter could include more than one value and bing chat answered that it's assumption was no because it could find no mention anywhere of being able to use more than one value for that parameter. As far as I can tell, no one has answered a similar question anywhere online that it could have been regurgitating, so it seems to have deduced it.

andsoitis 3y ago
I would be careful to draw that conclusion. Not all information LLMs are trained on is accessible to the wide public (e.g. ChatGPT has training data from closed sources). Regardless, If you share the details of your prompt, I’d be interested to see whether my Google-fu turns up information.
- nomel 3y ago
  Are you suggesting that LLM can't produce novel output?
  - andsoitis 3y ago
    No.

akio 3y ago

> I really don't understand why people ascribe emergent behaviors to these systems such as "theory of mind". It's just randomness. I'm presuming there's just a lot of psycological pareidolia going around. Ah yes, "just randomness": ---- From the above text, generate these variations: 1) For each word in the text count the number of letters in the word and append the count to the word in parenthesis. 2) Write each word backwards. 3) Write a similarly structured text that argues the opposite of the original text. ---- 1) I (1) really (5) don't (4) understand (10) why (3) people (6) ascribe (7) emergent (8) behaviors (9) to (2) these (5) systems (7) such (4) as (2) "theory (6) of (2) mind" (4). It's (4) just (4) randomness (10). I'm (3) presuming (9) there's (6) just (4) a (1) lot (3) of (2) psychological (12) pareidolia (10) going (5) around (6). 2) I yllaer t'nod dnatsrednu yhw elpoep ebircsa tnegrime sroivaheb ot eseht smetsys hcus sa ".dnim fo yroeht" s'tI tsuj ssenmodnar m'I gnimuserp s'ereht tsuj a tol fo lacigolohcysp ailodierap gniodnuora 3) I absolutely comprehend how people can accurately identify emergent behaviors in these systems like "theory of mind". It's far from random. I'm convinced that these are genuine insights being recognized.

re 3y ago
Did you notice that it made multiple errors for the first two variations? [Edit:] What's particularly interesting to me is the inconsistent counting of apostrophes for #1, and the handling of "theory of mind" for #2 -- though there are simpler errors besides these.
- daveguy 3y ago
  Not to mention a third one isn't any sort of argument, just a statement of the opposite belief.
  - deleted 3y ago
    [deleted]
  - bitwize 3y ago
    The program Eliza could sucker a lot of people just by inverting pronouns and adding flavor text, e.g.: Querent: my parents hate me Eliza: Why do you feel that your parents hate you? At a higher level, ChatGPT in that example is negating the provided text and returning something of opposite meaning but not doing much more.

bitwize 3y ago

> PC hardware means benchmark results which means benchmark websites/blogs which means the "as we tested in our previous article" phrase is common so it will just put it there in the answer even if makes absolutely no sense whatsoever. "Don't forget to like and subscribe!" -- A Hackernews' 3-yo daughter, saying goodbye