4 ms·

That was accurate only before the instruction tuning.

by lostmsu 10mo ago

Terr_ 10mo ago
I don't see how that gives the algorithm an ego when it didn't have one before.
- lostmsu 10mo ago
  I don't see how that does not considering that base models never refer to themselves as "I" and most of the modern instruct ones do. I can foresee the objection, that there's still a distinction between an author and a script, but for instruct models at this point it seems no different than author intentionally lying when talking about themselves.
  - Terr_ 10mo ago
    > it seems no different than author intentionally lying That begs a really big question, assuming that humanoid type of "intent" already existed and was somehow mis-aimed the whole time. Not to be confused with the algorithmic intent of `f(tokens,opts) -> next_token` .
    - roywiggins 10mo ago
      The blame attaches to the corporation that trained the "I" outputs into the model. Chat-tuned LLMs are mostly finetuned to roleplay as a friendly assistant at all times. This isn't exactly a lie- the models are really being designed to be helpful assistants- but the side-effect is that they also present as coherent personalities when they aren't. When a model outputs stuff like "I am FooGPT, a friendly chatbot" it is roleplaying just as much as when it's outputting stuff like "Hello, my name is Abraham Lincoln, I was the 16th President of the United States."
      - Terr_ 10mo ago
        Right, there's a pervasive issue here that involves illusions and assumptions coming from the minds of the humans perceiving everything. It's like that meme where people are asked how a mirror "knows" what object is being held when a piece of opaque paper is placed in between the object and the nearest mirror surface. Both are genuinely useful, but with mirrors we've built an accepted body of knowledge and authority, telling people to distrust their intuition and analyze it as light-paths. LLMs are another kind of reflection—of languages—but the same guardrails aren't established, and some people have a rather strong profit motive to encourage consumers and investors to fall for the illusions.
    - lostmsu 10mo ago
      What do you mean intent already existed? The whole point is it didn't until instruction tuning.

That was accurate only before the instruction tuning.

by lostmsu 10mo ago

That was accurate only before the instruction tuning.

Terr_ 10mo ago

I don't see how that gives the algorithm an ego when it didn't have one before.

lostmsu 10mo ago
I don't see how that does not considering that base models never refer to themselves as "I" and most of the modern instruct ones do. I can foresee the objection, that there's still a distinction between an author and a script, but for instruct models at this point it seems no different than author intentionally lying when talking about themselves.
- Terr_ 10mo ago
  > it seems no different than author intentionally lying That begs a really big question, assuming that humanoid type of "intent" already existed and was somehow mis-aimed the whole time. Not to be confused with the algorithmic intent of `f(tokens,opts) -> next_token` .
  - roywiggins 10mo ago
    The blame attaches to the corporation that trained the "I" outputs into the model. Chat-tuned LLMs are mostly finetuned to roleplay as a friendly assistant at all times. This isn't exactly a lie- the models are really being designed to be helpful assistants- but the side-effect is that they also present as coherent personalities when they aren't. When a model outputs stuff like "I am FooGPT, a friendly chatbot" it is roleplaying just as much as when it's outputting stuff like "Hello, my name is Abraham Lincoln, I was the 16th President of the United States."
    - Terr_ 10mo ago
      Right, there's a pervasive issue here that involves illusions and assumptions coming from the minds of the humans perceiving everything. It's like that meme where people are asked how a mirror "knows" what object is being held when a piece of opaque paper is placed in between the object and the nearest mirror surface. Both are genuinely useful, but with mirrors we've built an accepted body of knowledge and authority, telling people to distrust their intuition and analyze it as light-paths. LLMs are another kind of reflection—of languages—but the same guardrails aren't established, and some people have a rather strong profit motive to encourage consumers and investors to fall for the illusions.
  - lostmsu 10mo ago
    What do you mean intent already existed? The whole point is it didn't until instruction tuning.