6 ms·
Mistral Remove "Committing to open models" from their website
- cdme 3y agoTime to build a moat.
- samketchup 3y agoI hope they keep tweeting magnet links for model releases :(
- firebaze 3y agogood thing that https://huggingface.co/TheBloke/dolphin-2_2-yi-34b-GGUF https://huggingface.co/TheBloke/dolphin-2_2-yi-34b-GGUF surpasses even Mixtral MoE. Let's hope this continues.
- seattleeng 3y agoSeems like "open source" as a marketing tactic (or perhaps strategy, if they do continue to release open models) has peaked. I'm not really complaining, we get a lot of stuff for free as engineers (especially software), but it does seem different for a company to release an open model without any future commitments (e.g. Google) vs making open weights your raison d'etre and then pivoting quite quickly. The first feels transactional but honest, and the other a bit too... machiavellian? I do think it's too soon to pass judgement; this could just be a normal "freemium" strategy from days old, where you just pay up if you like the smaller/cheaper/free versions of their models.
- kromem 3y agoI think we'll continue to see n-1 open models used to build momentum to closed n models for the foreseeable future. Which is fine, as it will accelerate to diminishing returns on the +1 difference.
- a_wild_dandan 3y agoMistral's following OpenAI's playbook: embracing the clopen source movement. Their community betrayal (sorry, "pivot") hurts, but it shouldn't. Burning goodwill has always been far more lucrative than burning an enormous check. Hopefully Meta's strategy of committing to open source as incentive for attracting top researchers (who can go anywhere and deeply want their work shared) keeps working. But I'm not hopeful. Eventually Llama-n will remain closed source, some researchers will leave, followed by a blog post word salad about safety -- or whatever in vogue pretext works best tomorrow. Ugh.
- dontupvoteme 3y agoIt's not nearly a comparable situation.
- throwawaymanbot 3y ago[dead]
- hyperliner 3y ago[dead]
- chasd00 3y agoMaybe it was required per the Microsoft investment. I wouldn't be surprised.
- jampekka 3y agoThat company does try its very best to keep computing crappy.
- aeurielesn 3y agoI couldn't agree more.
- aunty_helen 3y agoIf you put 13 billion into a moat to be threatened by an open source competitor what would you do? An extra billion or two to protect the interest of your trillion dollar company seems well worth it.
- ricardobeat 3y agoGood thing we have antitrust laws. Can’t wait to see what happens next…
- fzzzy 3y agoAntitrust laws take decades to have an effect.
- SirensOfTitan 3y agoThat’s a shame—we cannot allow a handful of companies and VCs to capture most of the value of AI, especially if it actually starts replacing human jobs, it’ll just accelerate wealth inequality and social unrest.
- jauntywundrkind 3y agoThe economic prospects are grim. Couple that with creating a world where humanity is both at the whim of these tuning systems, and fundamentally unable to observe and learn about this golem, where IP keeps it as magic in our world: it feels like the most infernal of machines.
- ysofunny 3y agoI cannot let me get tired of repeating this: the underlying fundamental problem is that capitalism does not play nice with digital assets. an AI model is a very valuable digital asset right now, so there's a covert war being fought over public access to this.
- thomastjeffery 3y agoThe real fundamental problem is that "digital assets" are bullshit. An AI model is literally constructed by explicitly disrespecting copyright. The idea that a company gets to turn around and demand respect for their AI model's copyright is patently absurd.
- mlazos 3y agoI feel like this can be trivially worked around by fine tuning from the initially copyrighted weights. They’re not demanding copyright, they’re just keeping them secret. If Meta keeps releasing high quality open source models, I don’t expect the closed source models to have an advantage for long.
- jampekka 3y agoIntellectual property itself is a bad concept. Capitalism works for squeezing margins for bulk commodity production (largely on the expence of the worker), but for zero-marginal cost stuff it's a huge hinderance for progress.
- candiodari 3y agoBut if you have cloud based models ... same with mainframes 30-40 years ago ... it doesn't matter. Because the power cloud gives to the model owners is 10x more than the strictest intellectual property laws do. I don't believe, for example, that there's any intellectual property law that would let me yoink my intellectual property from you for any reason after you've bought and paid for it. In other words: I think the capitalism discussion is kind of pointless here. Capitalism isn't what gives these companies power. It's the cloud. Mainframe computing 2.0.
- MattDaEskimo 3y agoIt seems to me that large software companies often adopt an open-source approach initially to attract enthusiasts and stand out from leading competitors, but tend to adopt similar philosophies as their rivals once they achieve significant recognition or investment. It's all a marketing tactic, and I ain't for it.
- gwern 3y agoYou can see this as an endgame of 'commoditize your complement' (https://gwern.net/complement https://gwern.net/complement): you're happy to contribute to commodification while you are the small scrappy player, but at some point, if you are really successful, you will want to pull up the ladder after yourself, as it were.
- flir 3y agoIdle question: what's the complement of the model? The GPUs? By that logic, the hardware companies will be your best bet for open model development. (Everything old is new again).
- gwern 3y agoThe complement play for Mistral was that they'd be the low-cost SaaS serving their own models or any better ones that the FLOSS releases created and the upsell: you prototype and develop on Mistral models like Mixtral and then when you become a big boy, you outsource hosting or customization to them (with their razor-thin margins enforced by competition). The model releases generate demand for their low-cost hosting and consulting. (Think Red Hat, not OpenAI.) As opposed to an all-inclusive API where the pricing can be as high as the consumer-surplus with very fat margins, because it's all-or-nothing.
- miki123211 3y agoMistral just doesn't seem like an interesting company to me any more. They started off as the kind of people who released their models as magnet links and made them actually user-aligned instead of California-aligned. This is what I like to see from an AI company. Now, their models are no different from Open AI, Anthropic, Google, Meta and everybody else.
- vitorgrs 3y agoMeta actually releases the weights though (for now)
- andy99 3y agoIt will be interesting to see if/how they walk back from releasing the weights. They have put a lot of effort into the "open" (not open source but very intentionally trying to conflate) approach to AI. But probably almost nobody is paying attention. On the other hand, Meta has very little to gain by closing off their models. What would they do with them and who would use them? Llama was a coup because even though the license sucked, a passable model with available weights and llama.cpp allowed it to soar over the others of the time. Hopefully the benefit they got from that trumps any calls from the safety crowd to not share weights.
- vitorgrs 3y agoReally depends how good llama 3 will be. I see people thinking it would be better than GPT 4. But they still need to build a model as good as GPT 3.5 even, as llama 2 is... not.
- gumby 3y agoCould you expand more on what you mean by "California-aligned" (especially if you are thinking beyond AI models, though that alone could be interesting).
- tazu 3y agoI assume they mean "ethnically diverse Nazis"[1]. [1]: https://www.nytimes.com/2024/02/22/technology/google-gemini-german-uniforms.html https://www.nytimes.com/2024/02/22/technology/google-gemini-...
- summerlight 3y agoModel developments have become seriously capital intensive endeavors. Probably Mistral found themselves cornered by this commitment and they won't be able to secure any serious investments without changing this stance, MSFT in this case.
- anonym29 3y agoRelated: https://news.ycombinator.com/item?id=39511530 https://news.ycombinator.com/item?id=39511530
- fuddle 3y agoDamn, I was hoping Mistral would carry the open source torch for AI. I might be forced to create an actual open source AI company.
- andy99 3y agoFrom reddit: Chinese models seem to be the last hope now, LOL. It's going to be really interesting as two poles of power develop geopolitically, if the west or whatever you call it has to look to China or what we (the west) would consider the pole we look down on, for actually "free" ML models. (Edit, reminds me a bit of the silicon valley joke where Erlich tells Jin Yang he can't smoke in California as we don't enjoy the same freedoms you do in China)
- digging 3y agoI'm not really familiar with Chinese AI research, but my passing familiarity with the CCP makes me seriously question how free those ML models actually are, or will remain (if they currently are). I can't imagine the CCP wants AI enabling dissent or critical discussion of the CCP.
- rvnx 3y agoThey are actually very good, cf: https://qwenlm.github.io/blog/qwen-vl/ https://qwenlm.github.io/blog/qwen-vl/ and it's possible to unalign them (like Dolphin-Yi).
- dist-epoch 3y agoThe chinese gov is already pushing for regulations, for making sure AI doesn't "disrupt" society, etc...
- kinpy 3y agoMaybe we need a meta website similar to Dogpile from the 90s which would search all the search engines. When Gemini refuses to draw you a white family having a picnic China-GPT can help out. Ask a question about Taiwan and maybe something else can answer and so on. Also a wokeness benchmark would be great.
- z_ 3y agoDeepseek (https://github.com/deepseek-ai/DeepSeek-Coder?tab=readme-ov-file https://github.com/deepseek-ai/DeepSeek-Coder?tab=readme-ov-...) code is MIT and the model license is available too. edit I should mention that using the Web service Deepseek provides will unceremoniously shutdown terms deemed to be too sensitive. Self hosted models do not appear to be as aggressive.
- dist-epoch 3y agoThey imagined that it would be really difficult to run the models locally, so people wouldn't bother, but with stuff like llama.c & friends it's trivial.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- dkjaudyeqooe 3y agoThis is big tech or whoever using big money to lock up what they think is the dawn of 'true AI'. Buying people off is pretty easy! I'm sure I'd feel the same way and take the money. But just like you can't write an iPhone app without Apple's consent, soon we won't be able to do any serious AI work without the consent of, and payment to, MS or Google or whoever. I wonder what the talented people at OpenAI or Mistral tell themselves. That they're doing something good for society or technology? Probably, but already AI is being concentrated into a few hands. Nvidia has a virtual monopoly on hardware and training huge models is out of reach, open research got us here but that's looking increasingly shaky. Personally I think LLMs are a red herring, so there is still a chance to change the outcome, but we should take a lesson from what's transpired with OpenAI and Mistral and only support actual open development.
- bayindirh 3y ago> I wonder what the talented people at OpenAI or Mistral tell themselves. “We are doing something amazing, we need no permission”. Also, “we are good people and we have to guard this from bad actors”. IOW, they are fooling themselves (and following the money and military contracts).
- dontupvoteme 3y ago>I wonder what the talented people at OpenAI or Mistral tell themselves. At Mistral it's probably no small part "It shouldn't just be a few big american companies who control all of this" (and Mensch has spoken along these lines) I think in the worst case at least you'll have an EU big company in the cartel, but given what the EU has done for regular people against big tech I think that's a good thing
- Havoc 3y agoStraight after a MS investment? Pretty sure somewhere a MS competition lawyer just choked on his bagel. Also what’s the deal with company commitments being more like one night stands these days
- DANmode 3y agoCorporations are people, my friend! And people, especially "businesspeople" have been getting pretty cutthroat, on the whole.
- jarbus 3y agoI suppose the only major player in open source models will be META then