4 ms·

How does the computer answer "Do you like what i'm wearing today?" ? Well, if we say the computer is, in fact, not participating in the world with us -- it is

How does the computer answer "Do you like what i'm wearing today?" ? Well, if we say the computer is, in fact, not participating in the world with us -- it is merely predicting "the next word", then it cannot. I am not asking for any answer to this question. I want to know what it (like a friend) actually thinks about what i'm wearing. To do this, it would need to be a competent language user; not a word annoucer. It would, in otherwords, need to know what the language was about -- and need to be able to make a judgement of taste based on its prior experiences, etc. I dont think our ability to misattribute a capacity of languge to things (eg., to bugs bunny) is salient -- we are fools, easily fooled. Bugs bunny doesnt exist. In this case, the star trek computer, insofar as it actually answers the questions its asked -- is routinely depicted as being actually present in the world with us. That the show might claim "no it isnt!", or we otherwise hold this premise whilst observing that it is, is just foolishness. Bugs bunny likewise, is depicted with the premise that bugs is within his own world; this likewise, is irrelevant.

FeepingCreature 5y ago
Well, GPT is not the sort of thing that can have a "you." But it has seen dialogues that have a "you" in it, and it knows how a "you" tends to answer. For instance, depending on context, it may be operating under a different model for the "you" agent - the sort of person who likes a red dress, or the sort of person who likes suspenders. If we assume a multimodal GPT, it's going to draw on its pattern recognition from movies and its context window for what it's previously said as "you" or what you've prompted it as in order to guess what the agent it's pattern completing for "you" would think of your dress. In effect, I'm saying that just because GPT is not a word-user, that doesn't mean that its model of "you" - the layered system of patterns that generates its prediction for words that come after "I think your dress looks" - isn't a word-user. The "you" model, effectively, takes in sensory input, processes it, and produces output. Because the language model has learnt to complete sentences using agents as predictive patterns - because agents compress language - the you pattern acts agentic, despite the fact that the language model itself is not "committed" to this agent and will, if you reset its context window, readily switch to pattern predicting another agent. GPT is not an agent, but GPT can predict an agent, and this is equivalent to containing it.
- mjburgess 5y ago
  I dont think it is equivalent. If you assume it has the same modal properties, sure -- let's say that's plausible. Ie., if GPT said on the occasion it was asked Q, an answer A, in a possible world W, such that this answer A was the "relevant and reasonable" answer in W -- then GPT is "doing something interesting". Eg., if I am wearing red shoes (World W1) and it says "i like your red shoes" in W1, then that's for-sure really interesting. My issue is that it isnt doing this; GPT is completely insensitive to what world its in and just generates an average A in reply to a world-insensitive Q. If you take a langauge-user, eg. me, and enumerate my behaviour in all possible worlds you will get somehting like what GPT is aiming to capture. Ie., what i would say, if asked Q, in world-1, wolrd-2, world-infinity. My capacity to answer the question in "relevant and reasonable" ways across a gegnuine infinity of possible worlds comes from actual capacities i have to obvserve, imagination, explore, question, intereact, etc. It doesnt come from being an implementation of the (Q, A, W) pattern -- which is an infintity on top of an infinity. No model which seeks to directly implement (Q, A, W) can ever have the same properties of an actual agent. That model would be physically impossible to store. So GPT does not "contain" an agent in the sense that QAW patterns actually occur as they should. And no route through modelling those patterns will ever produce the "agency pattern". You actually need to start with the capacities of agents themselves to generate these in the relevant situations, which is not a matter of a compressed representation of QAW possibilities -- its the very ability to imagine them peicemeal (investigate, explore, etc .)
  - FeepingCreature 5y ago
    I mean, how would you discover that you're in world W? If you ask "what do you think about my red shoes?" and I say "I think your red shoes are pretty", then you will say this is just me completing the pattern. But if I have no idea what shoes you're wearing, then even I, surely agreed to be an agent, could not compliment your clothing. So I'm not sure how this distinction works. > It doesnt come from being an implementation of the (Q, A, W) pattern Well, isn't this just a (Q, A, W, H) pattern though? You have a hidden state that you draw upon in order to map Qs onto As, in addition to the worldstate that exists outside you. But inasmuch as this hidden state shows itself in your answers, then GPT has to model it in order to efficiently compress your pattern of behavior. And inasmuch as it doesn't ever show itself in your answers, or only very rarely, it's hard to see how it can be vital to implementing agency. And, of course, teaching GPT this multi-step approach to problem solving is just prompting it to use a "hidden" state, by creating a situation in which the normally hidden state is directly visualized. So the next step would be to allow GPT to actually generate a separate window of reasoning steps that are not directly compared against the context window being learnt, so it can think even when not prompted to. I'm not sure how to train that though.

FeepingCreature 5y ago

Well, GPT is not the sort of thing that can have a "you." But it has seen dialogues that have a "you" in it, and it knows how a "you" tends to answer. For instance, depending on context, it may be operating under a different model for the "you" agent - the sort of person who likes a red dress, or the sort of person who likes suspenders. If we assume a multimodal GPT, it's going to draw on its pattern recognition from movies and its context window for what it's previously said as "you" or what you've prompted it as in order to guess what the agent it's pattern completing for "you" would think of your dress. In effect, I'm saying that just because GPT is not a word-user, that doesn't mean that its model of "you" - the layered system of patterns that generates its prediction for words that come after "I think your dress looks" - isn't a word-user. The "you" model, effectively, takes in sensory input, processes it, and produces output. Because the language model has learnt to complete sentences using agents as predictive patterns - because agents compress language - the you pattern acts agentic, despite the fact that the language model itself is not "committed" to this agent and will, if you reset its context window, readily switch to pattern predicting another agent. GPT is not an agent, but GPT can predict an agent, and this is equivalent to containing it.

mjburgess 5y ago
I dont think it is equivalent. If you assume it has the same modal properties, sure -- let's say that's plausible. Ie., if GPT said on the occasion it was asked Q, an answer A, in a possible world W, such that this answer A was the "relevant and reasonable" answer in W -- then GPT is "doing something interesting". Eg., if I am wearing red shoes (World W1) and it says "i like your red shoes" in W1, then that's for-sure really interesting. My issue is that it isnt doing this; GPT is completely insensitive to what world its in and just generates an average A in reply to a world-insensitive Q. If you take a langauge-user, eg. me, and enumerate my behaviour in all possible worlds you will get somehting like what GPT is aiming to capture. Ie., what i would say, if asked Q, in world-1, wolrd-2, world-infinity. My capacity to answer the question in "relevant and reasonable" ways across a gegnuine infinity of possible worlds comes from actual capacities i have to obvserve, imagination, explore, question, intereact, etc. It doesnt come from being an implementation of the (Q, A, W) pattern -- which is an infintity on top of an infinity. No model which seeks to directly implement (Q, A, W) can ever have the same properties of an actual agent. That model would be physically impossible to store. So GPT does not "contain" an agent in the sense that QAW patterns actually occur as they should. And no route through modelling those patterns will ever produce the "agency pattern". You actually need to start with the capacities of agents themselves to generate these in the relevant situations, which is not a matter of a compressed representation of QAW possibilities -- its the very ability to imagine them peicemeal (investigate, explore, etc .)
- FeepingCreature 5y ago
  I mean, how would you discover that you're in world W? If you ask "what do you think about my red shoes?" and I say "I think your red shoes are pretty", then you will say this is just me completing the pattern. But if I have no idea what shoes you're wearing, then even I, surely agreed to be an agent, could not compliment your clothing. So I'm not sure how this distinction works. > It doesnt come from being an implementation of the (Q, A, W) pattern Well, isn't this just a (Q, A, W, H) pattern though? You have a hidden state that you draw upon in order to map Qs onto As, in addition to the worldstate that exists outside you. But inasmuch as this hidden state shows itself in your answers, then GPT has to model it in order to efficiently compress your pattern of behavior. And inasmuch as it doesn't ever show itself in your answers, or only very rarely, it's hard to see how it can be vital to implementing agency. And, of course, teaching GPT this multi-step approach to problem solving is just prompting it to use a "hidden" state, by creating a situation in which the normally hidden state is directly visualized. So the next step would be to allow GPT to actually generate a separate window of reasoning steps that are not directly compared against the context window being learnt, so it can think even when not prompted to. I'm not sure how to train that though.