By training a binary linear classifier over the vector space of the other NNs responses to a set of inputs with that specific feature. Language shouldn't be ne

By training a binary linear classifier over the vector space of the other NNs responses to a set of inputs with that specific feature. Language shouldn't be necessary if your feature can be conveyed through examples. But yes, it's a big assumption to say that all features can be isolated as linearly decidable subsets of the activation space. I would guess one could get better results with stronger, non linear classifiers combined with more abstract generalizations to directional derivatives.