3 ms·

You shouldn't optimize at all but instead pick multiple, 'distantly related' measures and satisfice over them. That's sort of what humans do to have a robust o

You shouldn't optimize at all but instead pick multiple, 'distantly related' measures and satisfice over them. That's sort of what humans do to have a robust objective function. Sufficient food, water, physical safety, love, social interaction, meaningful work, spiritual fulfillment—fulfill each of them and humans generally move on to the next 'unfulfilled' goal on the list rather than endlessly maximizing something that ceases to improve their life quality.

maxxxxx 8y ago
Exactly. Don't reduce everything down to one factor. In business you need profit AND customer retention AND ongoing development AND some level of luck AND a lot of other factors. Same for health. You need enough sleep AND good food AND a good mental state AND good genes AND a lot of other things. Focusing on only one thing while neglecting others doesn't work.
- mlthoughts2018 8y ago
  But then your "one factor" just becomes a combination of various other factors. I don't think the point of Goodhart's Law is related to "over-optimizing for just one thing," rather it's a statement about whether any metric (even a broad, diverse metric that takes into account multiple different things) can retain informativeness once it is set up as an optimization target.
  - maxxxxx 8y ago
    I think you could set up an optimization target for a company that says "be profitable and grow for the next 30 years". You would see quickly that this target is a complex beast with a lot of different factors at play that need to be balanced against each other. I don't think Goodhart's law would apply here.
davidmanheim 8y ago
That's often helpful, but not sufficient in general. See the dynamics discussed in my article here; https://arxiv.org/abs/1803.04585 https://arxiv.org/abs/1803.04585
mlthoughts2018 8y ago
That just sounds like coordinate gradient descent on a multi-term objective function. Just same old optimization but with some metaparameters or component-wise conditioning. In fact, I think to argue against optimization, you’d basically have to boil it down to existing philosophical arguments against utilitarianism. Either way, I’m skeptical if this conclusion is very connected to Goodhart’s Law, because it assumes some type of normative axiom that would have to precede your interpretation of Goodhart’s Law. My main interest was to pointout that it creates a sort of impossibility theorem situation if you accept Goodhart’s Law (I don’t personally accept it as a general rule, only something that applies to some types of measures and not others). The impossibility criteria would be something like: - If one finds a good measure, one should optimize it. - If a measure is an optimization target, it is not a good measure. - For a given entity, there exists a set of variables governing how well that entity can meet a goal (e.g. measures can possibly contain relevant information). Basically you could only choose 2. - If you pick the first two, it means there are no general variables governing the outcone you want; they are constantly changing and fundamentally unpredictable, so you might get away with optimizing something briefly, but its informativeness immediately disappears. This type of thinking is common in efficient market theory, for example, and could be connected to a fundamental belief about non-determinism or non-stationarity. - If you choose the first and third, it seems like standard utilitarianism. - If you choose the latter two, it seems like fatalism or religion — there are quantities that govern achieving desired outcomes, but they cannot be treated as objective optimization criteria and can only be analyzed subjectively or fatalistically.

maxxxxx 8y ago

Exactly. Don't reduce everything down to one factor. In business you need profit AND customer retention AND ongoing development AND some level of luck AND a lot of other factors. Same for health. You need enough sleep AND good food AND a good mental state AND good genes AND a lot of other things. Focusing on only one thing while neglecting others doesn't work.

mlthoughts2018 8y ago
But then your "one factor" just becomes a combination of various other factors. I don't think the point of Goodhart's Law is related to "over-optimizing for just one thing," rather it's a statement about whether any metric (even a broad, diverse metric that takes into account multiple different things) can retain informativeness once it is set up as an optimization target.
- maxxxxx 8y ago
  I think you could set up an optimization target for a company that says "be profitable and grow for the next 30 years". You would see quickly that this target is a complex beast with a lot of different factors at play that need to be balanced against each other. I don't think Goodhart's law would apply here.

davidmanheim 8y ago

That's often helpful, but not sufficient in general. See the dynamics discussed in my article here; https://arxiv.org/abs/1803.04585 https://arxiv.org/abs/1803.04585

mlthoughts2018 8y ago

That just sounds like coordinate gradient descent on a multi-term objective function. Just same old optimization but with some metaparameters or component-wise conditioning. In fact, I think to argue against optimization, you’d basically have to boil it down to existing philosophical arguments against utilitarianism. Either way, I’m skeptical if this conclusion is very connected to Goodhart’s Law, because it assumes some type of normative axiom that would have to precede your interpretation of Goodhart’s Law. My main interest was to pointout that it creates a sort of impossibility theorem situation if you accept Goodhart’s Law (I don’t personally accept it as a general rule, only something that applies to some types of measures and not others). The impossibility criteria would be something like: - If one finds a good measure, one should optimize it. - If a measure is an optimization target, it is not a good measure. - For a given entity, there exists a set of variables governing how well that entity can meet a goal (e.g. measures can possibly contain relevant information). Basically you could only choose 2. - If you pick the first two, it means there are no general variables governing the outcone you want; they are constantly changing and fundamentally unpredictable, so you might get away with optimizing something briefly, but its informativeness immediately disappears. This type of thinking is common in efficient market theory, for example, and could be connected to a fundamental belief about non-determinism or non-stationarity. - If you choose the first and third, it seems like standard utilitarianism. - If you choose the latter two, it seems like fatalism or religion — there are quantities that govern achieving desired outcomes, but they cannot be treated as objective optimization criteria and can only be analyzed subjectively or fatalistically.