4 ms·

>some overly-simplistic geopolitical objective function, like GDP or territory controlled at time t+1. The hard part here is writing down that objective functi

>some overly-simplistic geopolitical objective function, like GDP or territory controlled at time t+1. The hard part here is writing down that objective function. Remember, an AI/ML/cogsci algorithm is locked inside a black box, that being the hardware it runs on. Any objective function for RL must be expressed as a function (preferably a smoothly differentiable one, for gradient descent) of the sense-data available to the agent and the agent's hypothesis class about the world. Naive RL tends to optimize the function by, where at all possible, systematically decorrelating the agent's sense-data and reinforcement signal from the distal causes we intend them to represent.

SiempreViernes 8y ago
You mean write down a good objective function, making terrible ones are trivial. And the hard part is figuring out if the objective function you've spent six months refining actually gives sensible scores or if you just encoded all the mean of your data in that one hand tuned constant with six digits.
AndrewKemendo 8y ago
In my opinion, it's an impossible and undesirable task. So for example, write down the objective function for current General Intelligence: Humans. It's impossible, and has been the work of the field of Philosophy/Economics since we started seriously thinking about it.
- eli_gottlieb 8y ago
  Ehhhh we're pretty close on that one, actually. https://www.nature.com/articles/s41562-017-0069 https://www.nature.com/articles/s41562-017-0069
  - AndrewKemendo 8y ago
    Did you link the right article? Nothing in that article discusses interpersonal neurological response systems or anything relating to how mores and boundaries are created. Seems like you're linking to something which may be on the track to narrow down consciousnesses which is a separate question - and one I also question the benefit of caring about.
    - eli_gottlieb 8y ago
      >Nothing in that article discusses interpersonal neurological response systems or anything relating to how mores and boundaries are created. Well, we weren't discussing social reasoning and behavior, so I linked an article talking about the systems governing the brain's "objective function".
      - AndrewKemendo 8y ago
        In fact I was, but I wasn't explicit enough, so that's my fault. My original statement should have been: "We can't model Humanity's Collective Objective Function" - which is what would be behind what we are interested in: Stable functioning muti-agent systems. I think EY took a crack at this a long time ago and rightly abandoned the concept (see: CEV). Even with that clarification I disagree with the premise that we can model an "objective function" for an individual strictly in-vivo. Modelling an individual agent's reasoning/function system doesn't account for the environmental context it exists inside of, gives input into and responds to. So even if it was possible to understand the mechanism for intra-personal decision criteria, and I don't think it probably is, I don't think it's generalizable without having the context of inputs. Assuming that we could do this, I don't think you can extrapolate intentionality directly from individual to collective groups - which for an AGI is what is existentially important as it needs to be collectively general to solve the existential problem. I also don't think this is desirable as a framework for AGI - as humans, despite our intelligent status, are quite unstable and sub-optimal in groups.
        eli_gottlieb 8y ago
        >My original statement should have been: "We can't model Humanity's Collective Objective Function" - which is what would be behind what we are interested in: Stable functioning muti-agent systems. I think EY took a crack at this a long time ago and rightly abandoned the concept (see: CEV). If no such thing exists, then it was the wrong thing to investigate, so stop being interested in it. >Even with that clarification I disagree with the premise that we can model an "objective function" for an individual strictly in-vivo. Modelling an individual agent's reasoning/function system doesn't account for the environmental context it exists inside of, gives input into and responds to. So even if it was possible to understand the mechanism for intra-personal decision criteria, and I don't think it probably is, I don't think it's generalizable without having the context of inputs. That's just an inverse reasoning/theory-of-mind problem, one that normal theory-of-mind models and actual human brains solve every day. >Assuming that we could do this, I don't think you can extrapolate intentionality directly from individual to collective groups - which for an AGI is what is existentially important as it needs to be collectively general to solve the existential problem. What's this about "collectively general" and "the existential problem"? You seem to have gone off the deep end into philosophy salad. >I also don't think this is desirable as a framework for AGI - as humans, despite our intelligent status, are quite unstable and sub-optimal in groups. Considering you don't seem to know much about how humans work and what causes us to work well or badly in various situations, this statement comes off as almost racist.

SiempreViernes 8y ago

You mean write down a good objective function, making terrible ones are trivial. And the hard part is figuring out if the objective function you've spent six months refining actually gives sensible scores or if you just encoded all the mean of your data in that one hand tuned constant with six digits.

AndrewKemendo 8y ago

In my opinion, it's an impossible and undesirable task. So for example, write down the objective function for current General Intelligence: Humans. It's impossible, and has been the work of the field of Philosophy/Economics since we started seriously thinking about it.

eli_gottlieb 8y ago
Ehhhh we're pretty close on that one, actually. https://www.nature.com/articles/s41562-017-0069 https://www.nature.com/articles/s41562-017-0069
- AndrewKemendo 8y ago
  Did you link the right article? Nothing in that article discusses interpersonal neurological response systems or anything relating to how mores and boundaries are created. Seems like you're linking to something which may be on the track to narrow down consciousnesses which is a separate question - and one I also question the benefit of caring about.
  - eli_gottlieb 8y ago
    >Nothing in that article discusses interpersonal neurological response systems or anything relating to how mores and boundaries are created. Well, we weren't discussing social reasoning and behavior, so I linked an article talking about the systems governing the brain's "objective function".
    - AndrewKemendo 8y ago
      In fact I was, but I wasn't explicit enough, so that's my fault. My original statement should have been: "We can't model Humanity's Collective Objective Function" - which is what would be behind what we are interested in: Stable functioning muti-agent systems. I think EY took a crack at this a long time ago and rightly abandoned the concept (see: CEV). Even with that clarification I disagree with the premise that we can model an "objective function" for an individual strictly in-vivo. Modelling an individual agent's reasoning/function system doesn't account for the environmental context it exists inside of, gives input into and responds to. So even if it was possible to understand the mechanism for intra-personal decision criteria, and I don't think it probably is, I don't think it's generalizable without having the context of inputs. Assuming that we could do this, I don't think you can extrapolate intentionality directly from individual to collective groups - which for an AGI is what is existentially important as it needs to be collectively general to solve the existential problem. I also don't think this is desirable as a framework for AGI - as humans, despite our intelligent status, are quite unstable and sub-optimal in groups.
      - eli_gottlieb 8y ago
        >My original statement should have been: "We can't model Humanity's Collective Objective Function" - which is what would be behind what we are interested in: Stable functioning muti-agent systems. I think EY took a crack at this a long time ago and rightly abandoned the concept (see: CEV). If no such thing exists, then it was the wrong thing to investigate, so stop being interested in it. >Even with that clarification I disagree with the premise that we can model an "objective function" for an individual strictly in-vivo. Modelling an individual agent's reasoning/function system doesn't account for the environmental context it exists inside of, gives input into and responds to. So even if it was possible to understand the mechanism for intra-personal decision criteria, and I don't think it probably is, I don't think it's generalizable without having the context of inputs. That's just an inverse reasoning/theory-of-mind problem, one that normal theory-of-mind models and actual human brains solve every day. >Assuming that we could do this, I don't think you can extrapolate intentionality directly from individual to collective groups - which for an AGI is what is existentially important as it needs to be collectively general to solve the existential problem. What's this about "collectively general" and "the existential problem"? You seem to have gone off the deep end into philosophy salad. >I also don't think this is desirable as a framework for AGI - as humans, despite our intelligent status, are quite unstable and sub-optimal in groups. Considering you don't seem to know much about how humans work and what causes us to work well or badly in various situations, this statement comes off as almost racist.