3 ms·
Sure — abstractions elide details. But “this portion of the network appears to recognize faces” is still a useful abstraction. In the same way it is for neurol
by zmgsabst 4y ago
Sure — abstractions elide details.
But “this portion of the network appears to recognize faces” is still a useful abstraction. In the same way it is for neurology. Even if both NNs and brains sometimes see faces that aren’t there because the network isn’t “recognizing faces” but a system which signals on a complex set of heuristics that mostly detects faces.
How does the answer change if I only want 99% confidence and accept the existence of illusions? — can we describe the bulk of the behavior in a summary?
- janalsncm 4y agoIt would be very satisfying if we could have an explanation like that. But that’s far from guaranteed, unless that was part of the explicit design of an MMOE for example. Very likely if you ask “what part of the network recognizes faces” the best answer is “the whole thing”.
- int_19h 4y agoThere's some evidence for emergent "modules" inside NNs. https://www.alignmentforum.org/posts/j84JhErNezMxyK4dH/llm-modularity-the-separability-of-capabilities-in-large https://www.alignmentforum.org/posts/j84JhErNezMxyK4dH/llm-m...