3 ms·
Where is RWKV? With 10K+ github stars, it's become a foundation model for many.
by cs702 3y ago
Where is RWKV? With 10K+ github stars, it's become a foundation model for many.
- ShamelessC 3y agoRWKV is definitely cool. Using GitHub stars to extract meaningful insights is impossible on the other hand. I don’t think coolness is a qualifier for “foundational” either. RWKV needs to be widely deployed to the point that even its embeddings are more useful than an otherwise superior model (because embeddings are somewhat costly to calculate if you were to recalculate every time state of the art changes e.g. CLIP)
- swyx 3y agoRWKV probably has more usage than, say, AI21 Jurassic-2 on this chart
- ShamelessC 3y agoAh I see. So the answer really is just VC/FAANG funding then. Indeed, GitHub stars is effective in that case. The whole idea of foundation models is such an obvious ploy to either get citations by reputable labs or as the foundation (no pun) for a legal framework that lets big companies and no one else license their models.
- swyx 3y agowouldnt be that cynical. everyone just touching different parts of the elephant.
- dragonwriter 3y agoWhile RWKV is a foundation model as that term is usually used [0], its probably not a significant enough one by the criteria used here to rate coverage: there are lots of foundation models. [0] which is not about of popularity or importance, but breadth of training data and flexibility of output.
- cs702 3y agoActually, according to the OP, the 10 listed models were chosen based on their developers' "influence, heterogeneity, and status as established companies."
- dragonwriter 3y agoYes, that's why those 10 were selected of the many dozens of foundation models, not what makes them foundation models in the first place.
- cs702 3y agoAh, I see what you meant. Sorry if I misunderstood. We're in agreement :-)