8 ms·
> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-source
by sharkjacobs 3y ago
> He reiterated his belief in the importance of open source and said that OpenAI was considering open-sourcing GPT-3. Part of the reason they hadn’t open-sourced yet was that he was skeptical of how many individuals and companies would have the capability to host and serve large LLMs.
Am I reading this right? "We're not open sourcing GPT-3 because we don't think it would be useful to anyone else"
- greenie_beans 3y agolmao i had the same reaction. sounds like some bullshit.
- bibanez 3y agoI agree, this is so bizarre
- ftxbro 3y agoyes i also can't wrap my head around how a ceo of a billion dollar company isn't sincere in his public statements
- wintogreen74 3y agoReally? Even after saying this? "While Sam is calling for regulation of future models, he didn’t think existing models were dangerous and thought it would be a big mistake to regulate or ban them."
- cinntaile 3y agoIt was a tongue in cheek reaction.
- ethanbond 3y agoWhy couldn’t that be true? E.g. even scientists who worked on the Manhattan Project (justifiably) had antipathy toward the much more powerful hydrogen bomb. It’s possible to think squirt guns shouldn’t be regulated but AR-15s should, or AR-15s shouldn’t but cruise missiles should. Or driving at 25mph should be allowed but driving 125mph shouldn’t.
- MarcoZavala 3y ago[dead]
- RosanaAnaDana 3y agoIts just a way to lie that doesn't sound as much like a lie.
- sebzim4500 3y agoIt is weird, but GPT-3 is worse than much smaller LLaMA models so I doubt it would see much use anyway.
- flangola7 3y agoAre you referring to DaVinci or ChatGPT-3.5
- sebzim4500 3y agoDaVinci
- killjoywashere 3y agoHow do you measure this? Pointers to papers would be very helpful
- sebzim4500 3y agoThe LLaMA paper had a bunch of comparisons
- colordrops 3y agoAren't the LLaMA weights leaked though? Did Facebook ever open up its license?
- renonce 3y agoDoesn’t matter if you only use it yourself. No one will know.
- xxprogamerxy 3y agoHe wants the release of the model to primarily benefit individuals and smaller teams as opposed to large deep-pocketed firms.
- rurp 3y agoAnd he'll do that by... keeping ChatGPT models away from individuals and small teams and in the hands of a few large deep-pocketed firms? The great thing about open source is that people can try different approaches and gravitate towards what works best for them. Sam knows that of course, he's just being disingenuous because the truth makes him look bad.
- stavros 3y agoReads to me like "we don't know how many people will have hardware powerful enough to run this".
- ryanmercer 3y agoExactly. If you make it open source, great, cool, but only well-funded entities - like massive corporations - can even afford the hardware costs.
- stavros 3y agoEh, better than nobody.
- TigeriusKirk 3y agoHow can you sign a statement that AI presents an extinction risk on par with nuclear weapons and then even consider open sourcing your research? We don't provide nuclear weapons for everyone to keep in their basement, why would someone who believes AI is an existential risk provide their code?
- paxys 3y agoMore like – it won't be useful to small-time developers (since they won't have the capability to host and run it themselves) and so all the benefits will be reaped by AWS and other large players.
- sheepscreek 3y agoThis is what I understood as well. They want to either democratize adoption or not release it. The last thing they/anyone wants is for another BigCo or Govt to h take undue advantage of the model (through fine-tuning?) when others cannot. That said, I can imagine a GPTQ/4-bit quantized model to be smaller and easier to run on somewhat commodity clusters? Or it could run with GGML/llama.cpp on a cloud instance with a TB of RAM? After seeing what people were able to do with LLaMA, I am positive that the community will find a way to run it - albeit with some loss in performance. It would be truly amazing if they used their computing to develop quantized models as well.
- deleted 3y ago[deleted]
- lostmsu 3y agoIf you really need, 170B parameter model can infer a few tokens per minute on commodity hardware.
- renonce 3y agoA big chunk of developments based on Facebook’s LLaMA model are by small-time developers and individuals, not large players. Facebook has already shown a viable way to release models in the way you described.
- razcle 3y agoI think I worded this poorly. What he said was that a lot of people say they want open-source models but they underestimate how hard it is to serve them well. So he wondered how much real benefit would come from open-sourcing them. I think this is reasonable. Giving researchers access is great but for most small companies they're likely better off having a service provider manage inference for them rather than navigate the infra challenge.
- roganartu 3y agoThe beauty of open source is that the community will either figure out how to make it easier, or collectively decide it’s not worth the effort. We saw this with stable diffusion, and we are seeing it with all the existing OSS LLMs. “It’s too hard, trust us” doesn’t really make sense in that context. If it is indeed too hard for small orgs to self host then they won’t. Hiding behind the guise of protecting these people by not open sourcing it seems a bit disingenuous.
- rushingcreek 3y agoIf he says he's inclined to open-source GPT-3, I don't see any good arguments not in favor of giving startups the choice of how they can run inference.
- solarkraft 3y agoYou're saying the same thing. "I'm not sharing my chocolate with you because you probably wouldn't like it"
- choppaface 3y agoHere is how hard it is to serve and use LLMs: https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp
- doctor_eval 3y ago“The original implementation of llama.cpp was hacked in an evening.”
- 3y ago
- candiddevmike 3y agoOpenAI: Regulations must be passed to protect our moat Also OpenAI: Meta is pissing in our moat, let's drop a hint about open sourcing our shit too!
- TapWaterBandit 3y agoWhen you stop listening to what Sam Altman says and just focus on what he does, you can see the guy is a bit of a snake. Greedy power-hungry man imho.
- rafark 3y agoI’ve tried very hard to like him because like it or not, ChatGPT has revolutionized the AI industry but he’s so hypocritical I just can’t stand him.
- colordrops 3y agoPretty common for dislikeable people to be the most successful at an endeavor. It's not a coincidence.
- mellosouls 3y agoSam Altman is responsible for leading the team that have revolutionised AI in its position within society. There is plenty to criticise OpenAI for but what he and they have achieved is extraordinary, and there is no need for that sort of toxic personal attack.
- throwaway1777 3y agoIt’s fine to feel the ends justify the means but not everyone believes that.
- jasmer 3y agoSam is responsible for marketing the team which popularized a certain kind of AI product. No need for personal attack.
- f6v 3y agoI can’t comment on his personality because I don’t know him. But it’s delusional to think the leadership’s personal qualities are irrelevant to developing AI. There’s going to be a lot of subjectivity in fine-tuning the answers.
- barbariangrunge 3y agoIf small organizations and teams can’t use it, then open sourcing it mostly just benefits big tech That’s not ideal How does open source licensing work with respect to trained ai models anyway? Is something like the mit license even that valuable here? Or is it?
- sokoloff 3y agoIf the only barrier to a small team/org using it is cost/effort of hosting (as opposed to some licensing shenanigans), I fail to see how not releasing it is better for the world than releasing it would be. Even if it benefits big tech more than a small team. Am I somehow being protected by a benevolent sama not open-sourcing the model?
- bane 3y agoMost of the companies I work with are actively putting in place policies to prevent employees from using OpenAI's service because nobody wants to send their proprietary IP to them. Almost all of these companies have the technical ability, desire, and means to self-host for their employee community. Imagine the internal coup for CTO/CIOs everywhere to buy whatever is the latest Nvidia GPU cluster box, stick it in the on-prem datacenter, load a licensed GPT model and provide "AI as a service to our employees". Except what's happening is everybody is looking at buying the box from Nvidia, and sticking a large actually open model on it and simply ignoring OpenAI.
- f6v 3y ago> Almost all of these companies have the technical ability, desire, and means to self-host for their employee community. Well, one of companies I worked for could have hosted a canary service for cron jobs. But we bought it instead of building because we were focused on building features. And here you’re talking about hosting an entire LLM.
- tirpen 3y agoThe cron job canary was probably not a service that employees were uploading tonnes of company confidential material into, was it? So I fail to see how the comparison makes sense. The reason companies shun OpenAI and want a self hosted alternative isn't related to costs, it's becasue they don't want their code, internal emails, documentation etc to be uploaded to Microsoft and thus also directly to the NSA.
- f6v 3y ago> The cron job canary was probably not a service that employees were uploading tonnes of company confidential material into, was it? But they do to Slack and MS Teams. Also go mail services and other places.
- braindead_in 3y agoIt is a shame that Sama does not believe in Open Source. The community can solve their GPU bottleneck issue by making it run on CPUs and edge devices in a matter of days.