3 ms·

To play devil's advocate though, sometimes the ML training process is picking up a signal that humans simply haven't noticed. The moon waxing gibbous is a fun

To play devil's advocate though, sometimes the ML training process is picking up a signal that humans simply haven't noticed. The moon waxing gibbous is a fun example - what if moon phase were to affect our biology in subtle ways? There are plenty of things that people feel or know to be true but have trouble putting into words. That guy you know who just gives you a creepy vibe, but you can't put your finger on exactly why?

jerf 7y ago
To some extent, I tend to agree. Sometimes we humans are a bit too attached to our stories, and when a computer shows us they may not be true, our instinct is to simply disregard the computer, rightly or wrongly. After all, it isn't as if we haven't all had computers be wrong several thousand times for ourselves, right? But there is also an extent to which if my example did occur, it would still be flawed, at least with modern AI. Even if all these signals are in some abstract sense truly signals that you are a terrorist, in the real world it is still not correct to simply add them together, or whatever other simple operation the AI is doing. For all that deep learning may be cool, it's still got some giant, gaping holes in it compared to however human cognition works. Humans can look at say "Yes, even if those are all 1% signals, it really isn't sensible to add them all together." The current state of AI is not able to come to that conclusion, or at least, not well enough. If that changes in the future, well, I'll update my beliefs as appropriate then. It's hard to guess the "cognitive shape" the biases of the next breakthrough in AI may have.
meheleventyone 7y ago
The problem is that correlation does not necessarily imply causation. So if a ML process finds a counter-intuitive correlation we should seek to understand why rather than assume the results are actually meaningful. Sadly I think this is the step that is being gratuitously skipped to the extent that “working” models are hardly interrogated.

jerf 7y ago

To some extent, I tend to agree. Sometimes we humans are a bit too attached to our stories, and when a computer shows us they may not be true, our instinct is to simply disregard the computer, rightly or wrongly. After all, it isn't as if we haven't all had computers be wrong several thousand times for ourselves, right? But there is also an extent to which if my example did occur, it would still be flawed, at least with modern AI. Even if all these signals are in some abstract sense truly signals that you are a terrorist, in the real world it is still not correct to simply add them together, or whatever other simple operation the AI is doing. For all that deep learning may be cool, it's still got some giant, gaping holes in it compared to however human cognition works. Humans can look at say "Yes, even if those are all 1% signals, it really isn't sensible to add them all together." The current state of AI is not able to come to that conclusion, or at least, not well enough. If that changes in the future, well, I'll update my beliefs as appropriate then. It's hard to guess the "cognitive shape" the biases of the next breakthrough in AI may have.

meheleventyone 7y ago

The problem is that correlation does not necessarily imply causation. So if a ML process finds a counter-intuitive correlation we should seek to understand why rather than assume the results are actually meaningful. Sadly I think this is the step that is being gratuitously skipped to the extent that “working” models are hardly interrogated.