I don't get state space models/mamba, Essentially, as information comes in, they have a filter to decide which information is worth remembering regardless what

I don't get state space models/mamba, Essentially, as information comes in, they have a filter to decide which information is worth remembering regardless what future tasks they will be facing. This seems to work well only when the task is fixed, such as translation. Transformers on the other hand remember everything within a window, but when a task comes in, they can look back and fetch relevant information. I think transformers are inherently stronger, it resembles how human think. The (fetched) memory has to be task dependent.