7 ms·
> Access to the model will be granted on a case-by-case basis to academic researchers. Non commercial license
by laluser 4y ago
> Access to the model will be granted on a case-by-case basis to academic researchers. Non commercial license
- svengrunner2049 4y agoMicrosoft and OpenAI, Google losing $100bn from a bad demo, layoffs, cost cutting pressure to innovate again... I feel the days of full openness in AI research from corporations are over.
- mochomocha 4y agoI think we indeed hit "peak open-source" for AI and there unfortunately won't be as much sharing in the coming years. When the economy is down, people and companies think more in "zero-sum-game". I hope to be proven wrong.
- yacine_ 4y agohttps://twitter.com/EMostaque/status/1629160706368053249 https://twitter.com/EMostaque/status/1629160706368053249 :)
- rvz 4y agoThis is what everyone needs to watch. It was Stability.ai that spooked OpenAI for disrupting DALLE-2 with a open-source AI model; Stable Diffusion. I am betting they are going to do it again for ChatGPT. The endgame for this AI race is obvious and when it comes to AI models, open-source ones always disrupt fully closed source AI companies. But first, we'll see which 'AI companies' will survive the lawsuits, regulations and fierce competition for funding.
- somebodythere 4y agoThe golden age of open-source AI is ahead of us. Open-source AI companies are being launched and funded. High quality, large, labeled data sets have never been more accessible, and scaling law plateaus means there is going to be a lot more momentum on data- and compute-optimization, meaning current SOTA models will start fitting on smaller and smaller hardware, down to commodity hardware.
- flangola7 4y agoUntil legislation clamps it down.
- Karunamon 4y agoAt least in the United States, it's well established that code is protected by freedom of speech. https://www.eff.org/press/archives/2008/04/21-40 https://www.eff.org/press/archives/2008/04/21-40
- flangola7 4y agoYeah I don't expect that to last very long. Once it is discovered you can have an AI army that will tear apart any human force like tissue paper, AI will be classified and regulated as a WMD. They'll amend the Constitution if they have to, or get SCOTUS to do it for them.
- rnd0 4y agoUntil recently, reproductive rights were well established too. Shit changes, yo.
- Karunamon 4y agoThis is why activist judiciaries inventing rights that plainly are not there (and in fact, supporters at the time understood that this was on shaky ground legally) are dangerous to rely on. The situation here is not comparable.
- A4ET8a8uTh0 4y agoThis. We already had discussions of 'attacks' on models based on public data sets so 'good sets' may soon become the thing to go after ( and suddenly data brokers may really want to up the prices of their sets ). We might actually see more privacy as a result as the data brokers will start charging a premium for clear sets. Naturally, as predictions go, don't quote me on that. I was wrong before.
- laluser 4y agoThere is hope: https://www.bloomberg.com/news/articles/2023-02-21/amazon-s-aws-joins-with-ai-startup-hugging-face-as-chatgpt-competition-heats-up https://www.bloomberg.com/news/articles/2023-02-21/amazon-s-....
- alfalfasprout 4y agoI wouldn't be remotely so quick to throw in the towel. ML research tends to operate in "jumps" and plateaus. At this point, the concepts behind the big LLMs are relatively well known and the bottleneck is cost of compute + cost of training data. Thing is, cost of compute keeps coming down. OpenAI's "win" wasn't even so much in the research but in the design of ChatGPT as an interface. Its own model makes the same kinds of egregious mistakes as google and FB's own LLMs. Also, OpenAI was willing to just deal with the ethical fallout of releasing it into the wild with the ability to generate authoritative sounding falsehoods. I suspect we're going to go back to a period soon where a lot of the innovation we're seeing is around interfaces and infra to make interacting with LLMs natural and applying them to product use cases where they make sense.
- simonh 4y agoI really don't blame OpenAI for ethical issues over opening up access to ChatGPT. They're not claiming it's responses are factually correct, and arguably by making it openly available they have done more than anyone else to raise awareness of the risks and limitations of LLMs. We need access to these things to make informed decisions of what are or are not appropriate uses. Microsoft and Google are a different story, they're specifically pushing these as authoritative sources of information. If we hadn't had access to ChatGPT and the ability to learn it's ins and outs, it might have taken longer to expose so may of the flaws in the Microsoft and Google services.
- hackernewds 4y agowhy does Yann LeCun advertise that it is open source then?
- mertd 4y agoI assume you could train one like this provided that you bring your own training data and computing resources.
- p1esk 4y agoBecause it is open source: https://github.com/facebookresearch/llama https://github.com/facebookresearch/llama
- kaoD 4y agoIt feels like open source as a term became obsolete(-ish). Is it open source (in spirit) if only the inference code is open source? I mean, technically it might be, but not being able to train it myself is basically against FSF's freedom 1 if you consider the model the software. That repo is basically releasing a binary (the weights) and an open source runtime to run them.
- nazka 4y agoI agree… This is not open source. If you publish a wrapper that represent just 0.01% of your code and your product and hide all the rest as closed source that is considered open source now? Then any closed source providing an API become open source with this bogus definition. I guess now Windows is open source too, just miss the 99.99% of the rest of the code. It’s absurd and it damages the open source world and its real definition.
- p1esk 4y agoNo, this is open source. The model code and everything needed to verify their results is provided. No one promised the training code, and the training code is not needed to reproduce the results.
- generalizations 4y agoHere's hoping someone with access puts it on bittorrent.
- 1letterunixname 4y agoDLP, overall architecture, and many more aspects put the kibosh on that. The career risk isn't worth it especially when tech is deployed client- and network-side to detect just such exfil attempts. The average of network, security, and client management staff tend to be PEs (SREs) who can code, some have PhDs, and are the cream of what was previously organized as "corporate IT" world. So I fail to see any incentive to throw away their career and reputation by giving away IP for $0. There's a metric s*ton of optimized hardware to generate models. And I have my doubts if Sama at OpenAI, even with 10 gigabucks from Microsoft, can sustain growth, organizational culture, and long-term investment at the scale others are bringing online with less trouble and more experience. The future interaction will AI models will most likely be through an API because the models themselves are becoming too large to fit even on the most extreme DIY NAS solutions. TL;DR: it's not happening.
- generalizations 4y ago> The future interaction will AI models will most likely be through an API because the models themselves are becoming too large to fit even on the most extreme DIY NAS solutions. And yet I spent last night running GPT-J-6B on my desktop CPU at 2 tokens / sec. People are finally starting to optimize these models, and there's a ton of optimization to go. We'll definitely be running these locally in the next few years. This model especially looks like an ideal candidate for CPU optimization, given the pairity with GPT3, and that it's within spitting distance of the size of models like GPT-J-6B.
- Hyption 4y agoThere are plenty low paid students or working students or admins in-between.
- imranq 4y agoYou can make small LLMs more performant than large ones like GPT-3 by fine-tuning on specific tasks or providing them tools to offload precise calculations: e.g. Toolformer: https://arxiv.org/abs/2302.04761 https://arxiv.org/abs/2302.04761 which uses APIs and functions to improve GPT-J beyond GPT-3 for various tasks