4 ms·
Working on DNNs is more similar to gambling on the slot machines then it is to traditional programming. I gather training data, write a python script, and then
by opwieurposiu 8y ago
Working on DNNs is more similar to gambling on the slot machines then it is to traditional programming. I gather training data, write a python script, and then pay around $1 for GPU time to see if it's a winner. Instead of watching spinning reels I watch loss function graphs. 95% of the time it is a bust, but every once in a while you get a big payoff. I feel very smart when I win, but deeper investigation inevitably reveals I have no idea why A works and B does not.
- taneq 8y agoThat's the difference between you and evolution, though, right? You care about the outcome and the reasons behind it. Evolution doesn't care about either.
- okket 8y agoEvolution does have a strong bias towards offspring survival, though it is in some cases OK with the law of big numbers and statistical probability.
- taneq 8y agoI'd say that's less something evolution has and more something it is. Although "overall surviving fecundity" or something probably describes what it's optimising for pretty well.
- deleted 8y ago[deleted]
- chx 8y agoEvolution doesn't optimize for anything but the result are such that, to quote: > The prevalent genes in a sexual population must be those that, as a mean condition, through a large number of genotypes in a large number of situations, have had the most favourable phenotypic effects for their own replication.
- taneq 8y agoIt optimizes for something, not in the sense of intent but in the sense that there's something that increases as evolution takes place. It's just pretty hard to describe exactly what. Darwin called it by the delightfully circular name "fitness", where evolution is the survival of the fittest and the fittest are those that survive. (Of course, fitness is relative to an environment and the environment changes as things evolve, so it's not like there's some global 'fitness' property that increases. Just that, all else held constant, the thing which evolves tends to become better suited to reproducing in its environment.)
- nerdponx 8y agoWhat deeper investigation have you conducted that was unhelpful that way? Curious.
- k__ 8y agoDoesn't this make the profession rather stressful? I mean how can you sell your work to anyone if you fail most of the time?