2 ms·
Free, and open source models. Now and forever.
by 01284a7e 11mo ago
Free, and open source models. Now and forever.
- justonceokay 11mo agoWhat is a free model worth if it’s running on another company’s server farm, trained with data you do not have access to?
- Gracana 11mo agoThat is literally the thing the parent poster wants to avoid by running open models. [edit] I was a little unfair -- lack of access to training data is a bit of an issue (perhaps moreso for analysis than for for actual use, considering what it takes to train these models). I'm thankful that some of them are also distributed as base models, which should be relatively unbiased compared to what happens later during finetuning.
- GCUMstlyHarmls 11mo agoRun them on what though?
- Gracana 11mo agoThree power supplies, an old server, a grocery cart and a box fan, and every 3090 you and your friends can get your hands on.
- boppo1 11mo agoI want models I can run on my machine.
- sipjca 11mo agoI agree, but what about the training data that goes into it (intentional poisoning of the training data, for a variety of reasons, $, power, etc.)
- the_real_cher 11mo agoTo run your own chatgpt level model would require half a million bucks in infrastructure.
- jsheard 11mo agoThe problem is that training a free and open source model costs just as much as training a closed one, but has even fewer potential avenues for recouping that investment. The money still has to come from somewhere. I'm not sure if open weights are immune to being compromised by ads anyway, they can't serve pay-per-impression ads on the output side, but there's nothing stopping the creator from accepting funding in exchange for biasing the training one way or another. Coming soon: Foobar-600B, a new SOTA open weight model kindly sponsored by Coca Cola, Exxon Mobil and the Heritage Foundation. Please pay no attention to the men behind the curtain.
- ACCount37 11mo agoI would laugh my ass off if Coca Cola Company ends up being the company that solves alignment - so that it can align an "open weight" AI with its corporate interests. Without that though? Our ability to manipulate LLMs is so shaky I would be really surprised if anyone managed to pull off this kind of model manipulation and have it remain undetected.
- pxoe 11mo agoI almost believed that they just did, they aren't without their share of quirky and unusual projects and sponsorships.
- gldrk 11mo agoJust wait until someone leaks an internal SOTA model. Would be deeply ironic given how much AI robber barons ‘respect’ others’ copyright and trade secrets.
- Adrig 11mo agoI'm not sure about that. Reports have shown that models from China or Mistral can achieve 80% or more of OpenAI's performance for a fraction of the cost. If you're tucked in right behind the absolute frontier models, the economics change completely
- andy99 11mo agoI’m wondering how long it will be until they are also “sponsored” to have ad content trained in. I personally despise advertising but nobody is building these things out of the goodness of their heart. There needs to be some ongoing incentive to train and release open models. Similarly, I’m wondering when huggingface is going to need to start showing returns and starts putting ads into transformers etc.