2 ms·
So in an effective Mamba query the question goes at the end, after input data? I thought that the question should go at the beginning, so it can decide which in
by sp332 3y ago
So in an effective Mamba query the question goes at the end, after input data? I thought that the question should go at the beginning, so it can decide which information in the data is relevant.
- eropple 3y agoI could be wrong, as I haven't used Mamba, but it seems to remain similar to transformers in that it doesn't "decide" anything and streams tokens to follow the existing ones; attention isn't a thing in the same way, but recency does still have impact. To that end, putting context after the question makes it more likely to follow the context, not the question.