4 ms·
Do you mean a graph that contains all neurons from the network in one structure? Similar to these: https://gfycat.com/BonyTotalArthropods https://gfycat.com/Bon
by chaxor 3y ago
Do you mean a graph that contains all neurons from the network in one structure?
Similar to these:
https://gfycat.com/BonyTotalArthropods https://gfycat.com/BonyTotalArthropods
https://gfycat.com/BitesizedWeeBlacklemur https://gfycat.com/BitesizedWeeBlacklemur
That would be wonderful and I have been trying to do this. However, unfortunately some 'assumptions'/shortcuts have to be made. For example, the attention matrix is not known without input, so if just the structure of the network (weighted by the weights) is wanted, you have to put in some value 'p' ('1', '-1', w/e) to these edges. Also skip connections have to be dealt with explicitly instead of just adding them to a block diagonal matrix as one would with an MLP.
I am very interested if someone has a good solution that already has done these things though.
- amelius 3y agoThose are nice diagrams. Yes, well actually I'm interested in anything that's between the abstracted form in the paper and the fully expanded form where you see all neurons.
- chaxor 3y agohttps://miro.medium.com/v2/resize:fit:1100/format:webp/0*Y4G0mSaguuNEWFI2.png https://miro.medium.com/v2/resize:fit:1100/format:webp/0*Y4G... Does that work?
- amelius 3y agoAnother cool diagram. But one thing it misses is that you can't follow the arrows from input to output. For example, you might be tempted to think that the Keys or Queries are inputs to the neural net since there is no arrow going into them.