3 ms·
> The type of computation that happens for that attention step that you refer to is parallel Flash attention, which is widely used, is no longer parallel. The
by visarga 3y ago
> The type of computation that happens for that attention step that you refer to is parallel
Flash attention, which is widely used, is no longer parallel. The attention matrix is solved batch by batch.