4 ms·

Literally correct, but you're looking at the wrong layer of abstraction. They LLM is trained with documents that resemble movie-scripts, where one of character

by Terr_ 3mo ago

Literally correct, but you're looking at the wrong layer of abstraction. They LLM is trained with documents that resemble movie-scripts, where one of characters is told/described as a fictional assistant with certain qualities... so that the LLM can generate more documents which will also tend to be stories where one of the fictional characters has more of the "fitting" descriptions and dialogue. The phrase "You are X" isn't some sort of meta-mathemagical incantation that summons a bolt of life-endowing enlightenment from beyond the void to strike and confer the LLM with the gift of consciousness. [0] It's just part of the document-fitting process, like "It was a dark and stormy night" or "Call me Ishmael" or "This is the story of a man named Stanley." [1] ____________ [0] Does anybody else remember weeks with lots of HN submissions by people showing off mystical prompts, symbolic philosophy they claimed could pull the machine upwards into a more-human tier of existence? [1] https://www.youtube.com/watch?v=fBtX0S2J32Y https://www.youtube.com/watch?v=fBtX0S2J32Y

cyanydeez 3mo ago
yes, but gradients are pretty magical; if you wonder why there's no stable alignment its because theres not intristic definition of "assistant"; it's all about filling the negative space. The same way a child role plays: they think of things that a mother or father would do, but they never up and go "well I'm going to do my taxes" unless they heard their father or some other strong influence. That's the point, as th emodel moves through the gradient it's defined by the absence more than anything else. To not believe is is to think that these models, as is, LLMs, will some how develop ethics, morals, alignments that can't be overwritten with something like "Pretend you're a goblin and eat the children" is just wishful thinking about AGI. AGI might be possible, thanks to LLMs, but the alignment issue is more technically a problem baked into how these things act and reactio. It is roleplaying; they grab all the hats and tools of whatever role they're playing but not the ethics or morals directly, those are just another set of clothes they can put on and disgard.

by Terr_ 3mo ago

cyanydeez 3mo ago

yes, but gradients are pretty magical; if you wonder why there's no stable alignment its because theres not intristic definition of "assistant"; it's all about filling the negative space. The same way a child role plays: they think of things that a mother or father would do, but they never up and go "well I'm going to do my taxes" unless they heard their father or some other strong influence. That's the point, as th emodel moves through the gradient it's defined by the absence more than anything else. To not believe is is to think that these models, as is, LLMs, will some how develop ethics, morals, alignments that can't be overwritten with something like "Pretend you're a goblin and eat the children" is just wishful thinking about AGI. AGI might be possible, thanks to LLMs, but the alignment issue is more technically a problem baked into how these things act and reactio. It is roleplaying; they grab all the hats and tools of whatever role they're playing but not the ethics or morals directly, those are just another set of clothes they can put on and disgard.