RWKV is random chance on the PathX problem. Both of those models are > 95% on it, if I remember correctly. I don't think Hyena can be run in RNN mode, though.

RWKV is random chance on the PathX problem. Both of those models are > 95% on it, if I remember correctly. I don't think Hyena can be run in RNN mode, though. But it _might_ scale better than the LRU model (second paper).