3 ms·
People built an entire ecosystem around llama. This could benefit FB, however their models are too limited to compete with chatGpt, so the open source tools are
by seydor 3y ago
People built an entire ecosystem around llama. This could benefit FB, however their models are too limited to compete with chatGpt, so the open source tools arent particularly useful to them. FB is probably using better models by now. OTOH the open source community has an entire set of tools for the moment when Llama will be replaced with something better and completly open sourced model. at the moment , OSS seems to have benefited from their release.
- MichaelRazum 3y agoVicuna looks pretty good. But as said, commercial use not possible. Why do you think that Llama can be replaced? I mean it is extremely costly to train that thing. And it is there even a clean open source data set for the task? PS: wouldn't be surprised if Meta, OpenAi, or google will train something for a Billion $ in costs of compute or more.
- seydor 3y agothe 2048 sequence length makes it uncompetitive, especially now that we are entering ~infinite-length bots. $1M for training is peanuts for FB. There are open datasets like redPajama and alternative models are (hopefully) coming up