3 ms·

I was thinking about what type of intellectual endeavor might be "safe" from this Invention Machine, and obviously anything that has an intractable evaluation f

by rajat 17y ago

I was thinking about what type of intellectual endeavor might be "safe" from this Invention Machine, and obviously anything that has an intractable evaluation function is "safe". Take, for example, user interface design. Can we build an evaluation function that will give an answer as to whether one design will be more preferred than another? Is something that we considered ground-breaking when it first came out (ie. the iPhone), obvious from "first principles" of design? The evaluation function is intractable because you couldn't possibly put the generated designs in front of a number of people to evaluate it because of the time it will take to iterate. But then, could you evolve the evaluation function? Take a random sample of things that are considered to have fantastic user interfaces, break them down and evolve an evaluation function that is capable of designing at that level. Then test it against another set of randomly selected samples. And so on. Finally, test the designs on humans.

Tichy 17y ago
"you couldn't possibly put the generated designs in front of a number of people to evaluate it because of the time it will take to iterate" It's common practice these days, google A/B Testing. Or Mixpanel, I guess.
- rajat 17y ago
  A/B testing wouldn't be nearly fast enough iteration for something like this, I think. I don't know how many iterations it takes for the algorithm to come up with something useful, but even if it is anywhere in the 100s, A/B testing won't work or it'll take years... How quickly, in terms of iterations, does this machine usually take to come up with something? I mean, it could get lucky pretty quickly, but what is a reasonable number to expect? I don't know.
ahi 17y ago
I think the intractable evaluation problem is salient for many other fields as well. A machinist might know a particular design can't be built, but may not be able to quantify why. I think it's certainly possible that the universe ends before sufficiently complex problems can be processed by a GA or even adequately defined. On the other hand, displayed UI could probably be tackled by GAs even if the evaluation function is never codified. Stick a couple hundred no skill workers in front of a screen and you could get a couple hundred thousand evaluations an hour; in effect industrializing design.

by rajat 17y ago

Tichy 17y ago

"you couldn't possibly put the generated designs in front of a number of people to evaluate it because of the time it will take to iterate" It's common practice these days, google A/B Testing. Or Mixpanel, I guess.

rajat 17y ago
A/B testing wouldn't be nearly fast enough iteration for something like this, I think. I don't know how many iterations it takes for the algorithm to come up with something useful, but even if it is anywhere in the 100s, A/B testing won't work or it'll take years... How quickly, in terms of iterations, does this machine usually take to come up with something? I mean, it could get lucky pretty quickly, but what is a reasonable number to expect? I don't know.

ahi 17y ago

I think the intractable evaluation problem is salient for many other fields as well. A machinist might know a particular design can't be built, but may not be able to quantify why. I think it's certainly possible that the universe ends before sufficiently complex problems can be processed by a GA or even adequately defined. On the other hand, displayed UI could probably be tackled by GAs even if the evaluation function is never codified. Stick a couple hundred no skill workers in front of a screen and you could get a couple hundred thousand evaluations an hour; in effect industrializing design.