5 ms·

When using evolutionary pressures to guide software, you have to remember that your stated goal (via fitness functions) often does not truly reflect your intend

When using evolutionary pressures to guide software, you have to remember that your stated goal (via fitness functions) often does not truly reflect your intended goal. For example, when I was using genetic algorithms to pick stock trades, I tried to maximize total_net_gain_loss / number_of_trades. The GA quickly figured out that 0 trades was the best answer. In hindsight, Duh! But I wanted to make trades! "The only winning move is not to play"

sago 9y ago
That is one kind of problem. The 'Genie' problem: be very specific what you wish for because the genie will misinterpret you. But it's not the only kind of problem. There are inherent biases in evolutionary systems. One example: in systems with varying length genotypes there is a massive pressure to bloat. Even if you encode 'small' as a strong requirement in your fitness function, it may not be enough (or it may be enough to completely defeat whatever your real goal was). There are inherent biases. In the (simulated) genetics, in the genotype to phenotype mapping, in the evolutionary operators, even if you get the fitness function right. Evolutionary computing as an engineering tool is hard.
- fallous 9y ago
  Just picking a nit, but the problem with a genie isn't that it misinterprets you but that it literally does exactly what you ask for. The mismatch between "what you wanted" and "what you asked for" isn't the genie's fault but yours.
  - todd8 9y ago
    Yes, the genie gives you exactly what you ask for but misinterpretes what you wish for.
  - deleted 9y ago
    [deleted]
- tabtab 9y ago
  Re: "The 'Genie' problem: be very specific what you wish for because the genie will misinterpret you." My first wish is to beta-test my next 2 wishes.
allthenews 9y ago
This parallels much of mathematics. Many formulations, even in physics, admit so called trivial solutions which amount conceptually to zero.
posterboy 9y ago
My first thought: Why not just use max gain? of course that would leas to catastrophic failure over short or long. Second thought: Why not both? Just like speed is a derivative, so perhaps you want a set of differential equations, also gain per square time, too. On another note, "not to play" should be a zero-division error.

sago 9y ago

That is one kind of problem. The 'Genie' problem: be very specific what you wish for because the genie will misinterpret you. But it's not the only kind of problem. There are inherent biases in evolutionary systems. One example: in systems with varying length genotypes there is a massive pressure to bloat. Even if you encode 'small' as a strong requirement in your fitness function, it may not be enough (or it may be enough to completely defeat whatever your real goal was). There are inherent biases. In the (simulated) genetics, in the genotype to phenotype mapping, in the evolutionary operators, even if you get the fitness function right. Evolutionary computing as an engineering tool is hard.

fallous 9y ago
Just picking a nit, but the problem with a genie isn't that it misinterprets you but that it literally does exactly what you ask for. The mismatch between "what you wanted" and "what you asked for" isn't the genie's fault but yours.
- todd8 9y ago
  Yes, the genie gives you exactly what you ask for but misinterpretes what you wish for.
- deleted 9y ago
  [deleted]
tabtab 9y ago
Re: "The 'Genie' problem: be very specific what you wish for because the genie will misinterpret you." My first wish is to beta-test my next 2 wishes.

allthenews 9y ago

This parallels much of mathematics. Many formulations, even in physics, admit so called trivial solutions which amount conceptually to zero.

posterboy 9y ago

My first thought: Why not just use max gain? of course that would leas to catastrophic failure over short or long. Second thought: Why not both? Just like speed is a derivative, so perhaps you want a set of differential equations, also gain per square time, too. On another note, "not to play" should be a zero-division error.