3 ms·
If you have the option to disable the mask, isn't it then a generic nn.Block?
by _ea1k 3y ago
If you have the option to disable the mask, isn't it then a generic nn.Block?
- eduardoleao052 3y agoIt would be with some simple tweaks. For instance, the current block does not support Cross-Attention, just Self-Attention.
- GaggiX 3y agoThe ability to disable the mask in nn.MultiHeadSelfAttention, then having nn.DecoderBlock and a nn.EncoderBlock.
- eduardoleao052 3y agoI think that’s it. I’ll probably add that soon