4 ms·

Right, I think the Skyrim example another commenter linked is a good showcase of how that model had no real conception of what it was "doing" and no real cognit

Right, I think the Skyrim example another commenter linked is a good showcase of how that model had no real conception of what it was "doing" and no real cognitive ability to seek an understanding. I do think that some kind of "embodiment" or at least the addition of some kind of structured semantic model on top of/in addition to the raw token/vector embeddings is probably going to be necessary. I am interested though, in how much of that internal/semantic understanding can be "emulated" by a sufficiently rich text model over the course of a conversation. There was an interesting article a little while back about improving GPT-3's (IIRC) performance on math problems by structuring the prompts so that it would describe working out all the intermediate steps in solving the problem. I do suspect that whatever additional pieces we need to reach real general AI, the first really convincing examples of it will come not from looking at the architecture of the raw model, but at how it can be prompted to generate a believable accounting of it's own internal states over the course of an interaction.

GalahiSimtam 4y ago
But that example was made with Replika, nonetheless I chose to keep it for its unintended humor. GPT-3 is known to be good at creative fiction https://www.gwern.net/GPT-3#folktales https://www.gwern.net/GPT-3#folktales I suppose a bigger issue may be with keeping its confabulation to a single character, so that the AI won't effectively play the part of NPCs as well. There shouldn't be a problem with eliciting motivation because GM can ask the bot "What could you do in this situation? Please list your top three favorite actions", and those LLMs like their lists. The ineptness of the LLMs may get them killed eventually, but it would be something more like: a character A asks PC to deliver a sword to A's father; PC goes to where A's father lives; the first person they met there is B, unrelated to A; PC says to B "I am here to deliver this sword to your father"; GM facepalms as character B's job is to protect the household from assassins.

GalahiSimtam 4y ago

But that example was made with Replika, nonetheless I chose to keep it for its unintended humor. GPT-3 is known to be good at creative fiction https://www.gwern.net/GPT-3#folktales https://www.gwern.net/GPT-3#folktales I suppose a bigger issue may be with keeping its confabulation to a single character, so that the AI won't effectively play the part of NPCs as well. There shouldn't be a problem with eliciting motivation because GM can ask the bot "What could you do in this situation? Please list your top three favorite actions", and those LLMs like their lists. The ineptness of the LLMs may get them killed eventually, but it would be something more like: a character A asks PC to deliver a sword to A's father; PC goes to where A's father lives; the first person they met there is B, unrelated to A; PC says to B "I am here to deliver this sword to your father"; GM facepalms as character B's job is to protect the household from assassins.