5 ms·

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in t

by hyperion2010 10y ago

This is not just an issue for neural nets, but also for brains. Our interpretations of our own actions should always be considered posthoc rationalizations in the absence of some falsifiable experiment being conducted to demonstrate the validity of the interpretation. Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, thus we suffer the same kinds of issues, mitigated only by the fact that we have inherited developmental programs that have been subjected to a huge variety of adverse situations that have rigorously tested their performance (by killing anything that failed).

chiliap2 10y ago
I strongly agree but I'm left wondering, what would a falsifiable experiment look like in this case?
- unabst 10y ago
  If it's an assertion, we can test it. "The sky is blue" is an interpretation that can be re-tested. "My car has a flat" can be tested. If it's something that happened, we can't repeat history, so we're left digging up falsifiable evidence. This is the work of detectives when they solve cases, or historians re-evaluating historic facts. You don't re-test blood you no longer have, you're only able to scrutinize the analysis. You don't test again for genocide, you seek documents, records of communication, money trails, etc. A super casual modern day example would be an assertion like "Joe must be gay". You never know until you ask him, and that is if he's honest with you. But everyday-life is full of gross approximations based on bias and partial information. But the key, even in Joe's case, is you have to go back to the real source. If all we can do is talk about Joe, we can never know for sure. Much like celebrities we are so certain are gay, but haven't come out openly yet. So as long as computers can't leave their data centers, there really isn't any amount of data that can prove anything to them. They can't experience the truth, because ultimately the truth rests on experience and experience of evidence.
colllectorof 10y ago
You're hand-waving a real problem by asserting rather misanthropic ideas with scant justification. Show some child a cat and ask why it's not a crocodile. They will be able to explain it through the differences in shape, color, behavior and other features of those animals. Whether you consider it post-hoc is unimportant. The explanation is still real and relevant. So people can explain their deliberate reasoning. Also, reason about reasoning. If they truly can't, it's called gut feeling and no one rational trusts those for solving complex problems with important outcomes. Especially at scale. Neural networks, at least right now, have no capabilities of this sort. There are some attempts at visualization, but they are inherently limited, because they must make assumptions about the domain of the problem.
- empath75 10y ago
  Most actions people take are not the result of deliberate reasoning. You may be able to explain why a cat is different than a dog, but your brain doesn't go through that categorization process before recognizing a cat.
  - EGreg 10y ago
    How do you know?
    - rbanffy 10y ago
      The intermediate steps don't usually bubble up to conscious levels. Sometimes I "know" a design is bad. Explaining why to others requires stepping through a process that May or may not be the one my brain used to make that approach "feel" wrong.
  - csydas 10y ago
    Eh, I disagree, since a lot of times we rely heavily on visual cues to make assessments and determinations, and are often wrong just because of poor vision. Take for example persons with androgynous features and the difficulty some people have with identifying them. Children are particularly useful for this insight since they are unabashedly curious and inquisitive, and it's not uncommon to hear a child blurt out "is that a boy or a girl?"; it would seem to me that their categorization process, even if not refined, gets goofed by androgyny; something as simple as hair length or facial hair greatly influences a child's ability to easily discern if persno is male or female. At some level there is a categorization process that seems to happen with humans when it comes to recognition, we just assign high confidence to certain factors. In the case of children, it seems to be they look at person and use common factors like hair length, facial hair, and body shape to determine gender. The sound of one's voice also helps, but this usually isn't something we can pick up as easily or from afar in public places. As an adult, we probably have more refined points, but they are essentially the same as a child's point of view. Instead of "body shape" adults look for specific facial features, size of breasts, walking gait, style of clothing, hair style, etc. These are all just granular differences of how a child understands it. This gets into personal experience, but when I was in college, I had long hair and really fine skin/facial features. All the time I would hear little kids ask "why does he look like a girl?" or "why does that girl have a beard?". I had no issue with their confusion, they were just curious kids. Something confused their understanding, and once it was explained "sometimes men have long hair", they just sort of accepted it.
kilotaras 10y ago
> Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, One example of this is Anton-Babinski syndrome [1]. The patient can't see but is completely sure he can. [1] https://en.wikipedia.org/wiki/Anton%E2%80%93Babinski_syndrome https://en.wikipedia.org/wiki/Anton%E2%80%93Babinski_syndrom...

chiliap2 10y ago

I strongly agree but I'm left wondering, what would a falsifiable experiment look like in this case?

unabst 10y ago
If it's an assertion, we can test it. "The sky is blue" is an interpretation that can be re-tested. "My car has a flat" can be tested. If it's something that happened, we can't repeat history, so we're left digging up falsifiable evidence. This is the work of detectives when they solve cases, or historians re-evaluating historic facts. You don't re-test blood you no longer have, you're only able to scrutinize the analysis. You don't test again for genocide, you seek documents, records of communication, money trails, etc. A super casual modern day example would be an assertion like "Joe must be gay". You never know until you ask him, and that is if he's honest with you. But everyday-life is full of gross approximations based on bias and partial information. But the key, even in Joe's case, is you have to go back to the real source. If all we can do is talk about Joe, we can never know for sure. Much like celebrities we are so certain are gay, but haven't come out openly yet. So as long as computers can't leave their data centers, there really isn't any amount of data that can prove anything to them. They can't experience the truth, because ultimately the truth rests on experience and experience of evidence.

colllectorof 10y ago

You're hand-waving a real problem by asserting rather misanthropic ideas with scant justification. Show some child a cat and ask why it's not a crocodile. They will be able to explain it through the differences in shape, color, behavior and other features of those animals. Whether you consider it post-hoc is unimportant. The explanation is still real and relevant. So people can explain their deliberate reasoning. Also, reason about reasoning. If they truly can't, it's called gut feeling and no one rational trusts those for solving complex problems with important outcomes. Especially at scale. Neural networks, at least right now, have no capabilities of this sort. There are some attempts at visualization, but they are inherently limited, because they must make assumptions about the domain of the problem.

empath75 10y ago
Most actions people take are not the result of deliberate reasoning. You may be able to explain why a cat is different than a dog, but your brain doesn't go through that categorization process before recognizing a cat.
- EGreg 10y ago
  How do you know?
  - rbanffy 10y ago
    The intermediate steps don't usually bubble up to conscious levels. Sometimes I "know" a design is bad. Explaining why to others requires stepping through a process that May or may not be the one my brain used to make that approach "feel" wrong.
- csydas 10y ago
  Eh, I disagree, since a lot of times we rely heavily on visual cues to make assessments and determinations, and are often wrong just because of poor vision. Take for example persons with androgynous features and the difficulty some people have with identifying them. Children are particularly useful for this insight since they are unabashedly curious and inquisitive, and it's not uncommon to hear a child blurt out "is that a boy or a girl?"; it would seem to me that their categorization process, even if not refined, gets goofed by androgyny; something as simple as hair length or facial hair greatly influences a child's ability to easily discern if persno is male or female. At some level there is a categorization process that seems to happen with humans when it comes to recognition, we just assign high confidence to certain factors. In the case of children, it seems to be they look at person and use common factors like hair length, facial hair, and body shape to determine gender. The sound of one's voice also helps, but this usually isn't something we can pick up as easily or from afar in public places. As an adult, we probably have more refined points, but they are essentially the same as a child's point of view. Instead of "body shape" adults look for specific facial features, size of breasts, walking gait, style of clothing, hair style, etc. These are all just granular differences of how a child understands it. This gets into personal experience, but when I was in college, I had long hair and really fine skin/facial features. All the time I would hear little kids ask "why does he look like a girl?" or "why does that girl have a beard?". I had no issue with their confusion, they were just curious kids. Something confused their understanding, and once it was explained "sometimes men have long hair", they just sort of accepted it.

kilotaras 10y ago

> Human brains are excellent at creating a coherent story about the world they experience based on the data at hand, One example of this is Anton-Babinski syndrome [1]. The patient can't see but is completely sure he can. [1] https://en.wikipedia.org/wiki/Anton%E2%80%93Babinski_syndrome https://en.wikipedia.org/wiki/Anton%E2%80%93Babinski_syndrom...