2 ms·
> Specifically, the sLSTM has still recurrence (memory mixing) in it, i.e. you cannot fully parallelize the computation. If you mean that you cannot fully para
by zozbot234 3y ago
> Specifically, the sLSTM has still recurrence (memory mixing) in it, i.e. you cannot fully parallelize the computation.
If you mean that you cannot fully parallelize inference, this might be true but also not quite relevant since the computational demands of inference are low. And you can always "parallelize" training to some extent, just by training larger batches.
- korbip 3y agoThis was formulated a bit unclear. It is not possible to parallelize in the sequence dimension for training as it is possible for Transformers. In the batch dimension you can always do it.