I think the group working on rwkv has yes, even if they don't show the details in this article. (I followed them on their discord channels for quite some time)

I think the group working on rwkv has yes, even if they don't show the details in this article. (I followed them on their discord channels for quite some time). The interesting take away for me was that training rwkv from zero to intelligible sentences for minority language was more faster (in units of tokens trained!) than other architectures, making it more accessible for cases where large corpus like the Pile don't or can't exist.