4 ms·

I think that's exactly the point. Evolution produced our brains, and no one understands them yet. But evolution was able to produce them because it cares about

I think that's exactly the point. Evolution produced our brains, and no one understands them yet. But evolution was able to produce them because it cares about success-testing, not understanding. I think we limit ourselves quite a bit in the mainstream software industry by insisting that everything be hand-crafted by humans and understandable. (Not to mention how short we fall of making software "understandable").

TeMPOraL 10y ago
By "limiting" ourselves this way we can go forward much faster than evolution can. We create designs in hours what would take evolution millions of years to achieve. Also, it's a mistake to treat products of evolution as some kind of gold standard. It's literally worst design that could still work. We can do better, but we do it via different means. Another thing is that understanding is a key component of trust. Animals come to mind here. We can't understand their firmware per se; by observing them over thousands of years, both passively and actively (through domestication, training regimens, etc.), we've learned to expect some things from them but not others. So I can be sure that e.g. my cat won't kill me in my sleep. But I can absolutely not trust it to go somewhere when I want it to go there, or to stay in place when I want it to stay, or even to eat food when I give it. We want to write software that will bear much greater, more important responsibilities than any animal in history. So we need to make them more understandable than animals if we're to ever trust them with those responsibilities.
- charlieflowers 10y ago
  Sure, I agree. Evolution did not take us from horse and buggy to planes and autonomous cars in 100-ish years. But when we push up against the limits of the problems we can solve with software, it's usually because it has outgrown our ability to understand it. That's exactly what is happening with this new burst of progress in machine learning. We're "teaching" the computer to do things, and we've let go of the ability to directly understand how the computer "represents its learnings."

TeMPOraL 10y ago

By "limiting" ourselves this way we can go forward much faster than evolution can. We create designs in hours what would take evolution millions of years to achieve. Also, it's a mistake to treat products of evolution as some kind of gold standard. It's literally worst design that could still work. We can do better, but we do it via different means. Another thing is that understanding is a key component of trust. Animals come to mind here. We can't understand their firmware per se; by observing them over thousands of years, both passively and actively (through domestication, training regimens, etc.), we've learned to expect some things from them but not others. So I can be sure that e.g. my cat won't kill me in my sleep. But I can absolutely not trust it to go somewhere when I want it to go there, or to stay in place when I want it to stay, or even to eat food when I give it. We want to write software that will bear much greater, more important responsibilities than any animal in history. So we need to make them more understandable than animals if we're to ever trust them with those responsibilities.

charlieflowers 10y ago
Sure, I agree. Evolution did not take us from horse and buggy to planes and autonomous cars in 100-ish years. But when we push up against the limits of the problems we can solve with software, it's usually because it has outgrown our ability to understand it. That's exactly what is happening with this new burst of progress in machine learning. We're "teaching" the computer to do things, and we've let go of the ability to directly understand how the computer "represents its learnings."