9 ms·
This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others
by la64710 4y ago
This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.
- scratchyone 4y agoWhy did you comment this 3 separate times?
- deleted 4y ago[deleted]
- TOMDM 4y agoDon't get me wrong, I'd love to be able to run GPT3 and the subsequent finetuned versions thereof myself, but OpenAI has essentially no financial incentive to do so. A few years ago, we could maybe lean on their open aspirations to get that done, but with the "limited profitibility model" they've since instead adopted, I think that dream is mostly gone. At least we still get the occasional treat like Whisper out of them.
- ronsor 4y agoIt's not as if anyone could afford to self-host the giant GPT-3 model anyway.
- TOMDM 4y agoI'm rather sure there are a mid sized number of "anyones" who would buy the hardware required to run it given the opportunity. But I certainly agree that not everyone can, and very few individuals. The thing is a monster of a model.
- deleted 4y ago[deleted]
- drdaeman 4y agoWhy not? Please correct me if I'm wrong but from what I've heard, It's not millions or hundreds of thousands, it's "only" tens of thousands dollars' of hardware we're talking about. There are enthusiasts that spend comparably on their hobbies. And a few dozen like minded people banding up together, shelling out a couple grands each? I'd say that's a totally realistic scenario.
- deleted 4y ago[deleted]
- t3rmi 4y agoOpen AI's CEO Sam Altman's take is that they will only ever allow API access to their models to avoid misuse. I don't get HN's take with wanting everything open sourced. Some things are expensive to create and dangerous in the wrong hands. Not everything can and should be open sourced.
- smeagull 4y agoYeah, just like the existence of Windows prevented Linux from ever existing. The wrong hands have the money to seek alternatives. All this policy does is keep it out of the hands of the public, and ensure that whatever open alternatives start up won't be OpenAI's.
- scifibestfi 4y agoIsn't an open source ChatGPT inevitable? There's already open source AI art.
- TOMDM 4y agoCheck out the BLOOM models if you want to see a first stab at that. If you can find an economical way of running it though, let me know.
- generalizations 4y agoA cluster of 6 year old 24GB NVIDIA Teslas should do the trick...they run for about $100 apiece. Put 12 or so of them together and you have the VRAM for a GPT3 clone.
- TOMDM 4y agoHuh. Tempting. Amazon has them listed at $200, but still, that's only $2,400 for 12 of them. Still, adds up once you get the hardware you'd need to NVlink 12 of them, and then on top of that, the price of power/perf you get probably isn't great compared to modern compute. Wonder what your volume would have to be before getting a box with 8 A100's from Lambdalabs would be the better tradeoff.
- kube-system 4y agoTheir end goal is to be stinkin’ rich, and it looks like that might happen. They’re not going to let the fear of an hour of downtime undermine that.
- motoboi 4y agoSomething like 1,5Tb memory to run this model in inference mode.
- antonvs 4y agoIs that really all? We regularly run multi TB memory clusters for big data processing and ML. I imagined it would be much bigger than that. To put that in perspective, 24x 64 GB nodes is 1.5 TB.
- albntomat0 4y agoMy understanding is that all the memory has to be GPU memory, with proper interconnects. Still not that crazy, all things considered
- lossolo 4y ago> 24x 64 GB nodes is 1.5 TB Looking at your calculations indicates that you mean RAM but it's 1.5 TB GPU VRAM (but this is assuming they use 64 bit precision, which is likely wrong so it's ~750 GB), not RAM.
- lossolo 4y agoYou meant 700 GB? 32 bit precision in 175B model is around ~700GB + overhead or around 350 GB if they use half-precision.
- motoboi 4y agoI was wrong. It seems they use fall precision. So 350gb. What is amazing, human language (languages?) and knowledge encoded in so little space.
- alfor 4y agoDon’t worry, we will see plenty of other GPT very soon. The genie is out of the bottle.
- whazor 4y agoI see OpenAI as a 'company' that (eventually) wants to earn money with their models. The 'open' part is trying to commercialize part it in such a way that everyone can use it. Also be open with the research and risks. But not open as in open-source.