For all we know, Mamba could be overhyped and could only work for small models. Remember retnet[1] which said the same thing, equal to transformer in linear tim

For all we know, Mamba could be overhyped and could only work for small models. Remember retnet[1] which said the same thing, equal to transformer in linear time. [1]: Retentive Network: A Successor to Transformer for Large Language Models: https://arxiv.org/abs/2307.08621 https://arxiv.org/abs/2307.08621