3 ms·

Been diving deep into the Mamba paper and put together my notes here: https://blog.oxen.ai/mamba-linear-time-sequence-modeling-with-selective-state-spaces-arxi

Been diving deep into the Mamba paper and put together my notes here: https://blog.oxen.ai/mamba-linear-time-sequence-modeling-with-selective-state-spaces-arxiv-dives/ https://blog.oxen.ai/mamba-linear-time-sequence-modeling-wit... Took me awhile to wrap my head around some of the terminology, so hopefully this helps anyone else trying to grok it.

andy99 3y ago
You may also be interested in https://github.com/rbitr/llm.f90/tree/master/ssm https://github.com/rbitr/llm.f90/tree/master/ssm it's my inference only implementation of mamba which ends up being much simpler than the training code in the original repo