3 ms·
The fact that Metas models have managed to be less accessible than fully closed, API-only offerings makes me really struggle to see the value here. If you’re wi
by cowthulhu 3y ago
The fact that Metas models have managed to be less accessible than fully closed, API-only offerings makes me really struggle to see the value here. If you’re willing to jump through corporate hoops, OpenAI can have you up and running in a few minutes, with few relevant restrictions for most businesses. If you’re willing to trade performance and ease of use for control, then there are plenty of open source models with decent performance you can fine-tune. Metas models are like the worst of both worlds - subject to restrictive licensing and approval with limited control (at least legally), but requiring a fair bit of elbow grease to get working compared to just hitting an API.
- lamroger 3y agoI thought llama is open weights and open to commercial with reasonable restrictions. I haven't heard practitioners who fine tune llama having trouble with it
- refulgentis 3y ago? They aren't subject to restrictive licensing. It's about a 10 liner to run behind an API
- spott 3y agoThere are two clauses in the llama license that I’m aware of that are restrictive: 1) if you have more than 700M daily active users you can’t use it. 2) you can’t use the output of this model to train other models. Are you aware of others, cause I’m not, and those aren’t unreasonable.
- dharmab 3y agoThe TOS also outlines a number of restricted use cases, which are broader than other models. For example, Llama prohibits any ITAR related use, while Anthropic limits specific usage related to defense and weapons. (There are many peaceful technologies that are covered by ITAR).
- cowthulhu 3y agoActually - fair enough, I think I was overestimating how much people consider the ability to use the output to train your own models important. I guess for me these felt really restrictive since that always felt like the end game - collect a bunch of data, then train a smaller, cheaper to run model for a specific use case - but I’m clearly an outlier!
- bick_nyers 3y agoI think you are misunderstanding just a bit. You can certainly fine-tune/train LLAMA for a specific use case. Just grab LLAMA 7B and start training, that will fit on plenty of devices and you can even quantize the weights and make it run faster (and on even smaller devices). What you cannot do is use LLAMA outputs to train your ChatGPT weights.
- djohnston 3y agoWhat on earth are you talking about. Metas models are the foundations for almost all the popular open source LLMs.
- satvikpendem 3y agoIn what sense are they foundations? Are they using the literal code and models? Or just using the architecture and reimplementing and retraining the new models?