So I tried thinking about what this really means, and I guess to some extent it measures a weighted "consensus" of inputs? First consider the simplified case of

So I tried thinking about what this really means, and I guess to some extent it measures a weighted "consensus" of inputs? First consider the simplified case of real weights. Then it will simply detect if they inputs are similar numbers or not. Similar numbers -> similar angles -> addition of vectors all pointing in same direction -> high magnitude. With complex weights there is a also a rotation of the vectors which complicates the picture (before 2.1, 1.9, 1.8 would be strong agreement, with rotation it might instead be something like 2.1, -0.1, 0.5) but I think you can still somehow thing of it as measuring if there is a consensus or not. Now since the inputs are features from a previous layer I guess what it is looking for is stuff like "Feature A should be 1.6 more strongly activated than Feature B".