Honest question, just curious what's your definition of "different arch", is a model with GQA=8 different than one with GQA=16 in your mind? I guess no? How ab

Honest question, just curious what's your definition of "different arch", is a model with GQA=8 different than one with GQA=16 in your mind? I guess no? How about GQA vs MHA, or GQA vs MLA? If anything attention-like is same in your mind, is S5 and RWKV different arch given that both are some kind of linear RNN?