10 ms·
Ask HN: Can you crowdfund the compute for GPT?
I'm just curious to know whether it's possible to crowdfund the compute costs for GPT models? It seems like this stuff is going to start to get beyond what any one individual can run, meaning it's in the hands of corporations or people with deep pockets. Can groups of people pool together money to run shared models? Because the alternative is that the companies just run away with the technology and leave the rest of us to wait for APIs or use whatever they give us.
- jejeyyy77 4y agoI’d donate if the model and code became open source afterwards like stable diffusion.
- adg001 4y agoSeconded.
- justplay 4y agocount me in too.
- badsectoracula 4y agoAFAIK while the code is open source, the Stable Diffusion model license isn't open source though as it has usage restrictions that you wouldn't find in real open source license.
- speedgoose 4y agoYes: https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals
- asim 4y agoAh interesting. So I guess if they validate the thesis we could see a number of these right? Maybe co-op style, because you'd want some level of governance, not just, oh here's the cash and let's go run it, or oh hey let's bittorrent style do it but without vetting what stuff it's being used for e.g I don't want models being built for nefarious stuff on my computer.
- danaugrs 4y agoCheck out https://petals.ml/ https://petals.ml/.
- DavidPP 4y agoFunny, I was asking myself the same question this morning, I was wondering if there was the equivalent of SETI@Home but to pool resources for model training.
- cosmojg 4y agohttps://petals.ml/ https://petals.ml/
- deleted 4y ago[deleted]
- cdnsteve 4y agoWhat type of hardware is needed as the bare bones for this?
- TheRoque 4y agoDepends on which version you're running. According to https://github.com/amirgholami/ai_and_memory_wall https://github.com/amirgholami/ai_and_memory_wall And a guy with a youtube channel called "Asianometry" who did a video on this https://www.youtube.com/watch?v=5tmGKTNW8DQ https://www.youtube.com/watch?v=5tmGKTNW8DQ the full GPT-3 model needs thousands of high-end GPUs (meaning: not your home 1080 GPU)
- inglor 4y agoIt's no different from any other type of server I think? Crowdfunding compute comes with a lot of security and privacy concerns and often if you're building a product putting that part of it in the hands of someone reliable is preferable. It depends on what part of what you're building is your core intellectual property and such. I'll ask back: what products would you build on top of a crowdfunded-compute GPT like model?
- asim 4y agoPersonally? One I'm interested in is a personal assistant that has access to 100+ APIs that I can command solely with text e.g ability to schedule future tasks, ability to get answers to questions around real world events and data available through APIs, ability to combine a lot of these things. What Alexa could/should have been. Separately, examination of religious and historic tests in alternate languages providing superior transliteration and translation into english than what humans could achieve.
- mdorazio 4y agoIt’s not clear to me why your first example needs AI. If you have a defined text query setup (maybe with prefixes depending on what you want) the rest is basic case-based API selection, no?
- asim 4y agoInitially yes, so simple commands to APIs works, I've done this. What's harder and actually requires programming is when you want to combine the use of multiple APIs into some sort of workflow. You could start to write some sort of DSL for this but actually seeing how ChatGPT works, its going to be far more effective to have something interpret human language and write something that will trigger the APIs as and when it needs to.
- wongarsu 4y agoFrom my understanding that is exactly what Microsoft's XiaoIce is doing in the Chinese market. Just with a text interface instead of a voice interface. I guess there's not much interest to develop something equivalent for the English-speaking market because it's difficult to monetize. https://en.wikipedia.org/wiki/Xiaoice https://en.wikipedia.org/wiki/Xiaoice https://arxiv.org/pdf/1812.08989.pdf https://arxiv.org/pdf/1812.08989.pdf (actually a better introduction than the wikipedia)
- smnrg 4y agoSide hot take: 1. “AI” requires compute time for training (GPT, etc.) 2. If you use any “AI” service in the future, you could have to share idle computing power to improve that “AI.” 3. This may be tokenized via crypto. More contributions = more “AI” usage available for you. Could the use of tokens or cryptocurrency incentivize participants to contribute their idle computing power to facilitate the process beyond those with deep pockets?
- psychphysic 4y agoWould this work? I can't see meaningful work being down on the dataset piecewise. Don't you need the load the entire parameter set to backpropagate and potentially at quite high precision. Then probably load less than 8gb as typical max vram. And then finally send that all back. I think you really need dedicated data centres if only to move the parameters around and load the entire thing at once.
- bastawhiz 4y agoOr we could just...pay for stuff. My dear mother isn't going to spin up a GPU so she can talk to her ai assistant. I don't want to be told that I can't do stuff on my phone because I need to purchase a second stack of GPUs for my home because my kids talked to Elmo too much or whatever. We can make data centers that will gladly take money in exchange for their services, and I'm happy to provide money for them to supply those services.
- NonTrivialZer0 4y agoACT includes a proof-of-compute mechanism to ensure the reliability of the resources on the decentralized network. Users can register their instances with the system and be rewarded with ACT tokens for doing so. The system will also continuously monitor the instances, and remove instances that fail to meet quality standards or provide inaccurate results. ACT also includes a reputation system that allows holders to track the performance of the resources on the decentralized network. Users that provide accurate and stable resources will have a higher reputation score and be more likely to be selected for providing compute resources. The decentralized network is managed by a process for adding and removing resources. This ensures that the network is always providing high-quality resources and maintain the stability of the ACU value. https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogDEAlDPV9icT_v2pfwYS8/edit#heading=h.4s7avqao9hjl https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogD...
- xwdv 4y agoI’d be more interested in closed co-ops where individuals provide computer power in exchange for access to the model. I think freeloaders will eat up compute time.
- beernet 4y agoIt's not that easy. Access to enough compute is one thing. However, you also need a proper dataset (beyond Common Crawl and Wikipedia), excellent research expertise and engineering capabilities. So even if you throw money or free credits for cloud compute out there it will not be enough. We've seen this happen with EleutherAI who were not capable of reaching their initial target of "replicating" GPT-3 and could only deliver the GPT-NeoX 20B model despite all the free compute etc.
- rjh29 4y agoThere is Open Street Map (or Wikipedia for that matter). A large enough army of volunteers could produce or tag a dataset that rivals Google's data, but it would be a lot of work.
- orbz 4y agoAgreed, I was thinking the same thing. Raw compute is probably the cheapest/easiest part of this problem.
- oliwary 4y agoThis is my impression as well. Here is an example of the engineering capabilities required to train the modern models: https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf https://github.com/facebookresearch/metaseq/blob/main/projec... [PDF] It's a 114 page document detailing the months of full-time work for multiple engineers that went into training a 175B language model at meta AI.
- nathias 4y agothere were crypto projects like that, but I don't think any survived until today
- justplay 4y agoi will fund. worst case if it close-crowdfunded i will invest there too.
- deleted 4y ago[deleted]
- alexb_ 4y agoI've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 from their pool of connected graphics cards - Individuals can connect their computers to the Company Beta network and receive compensation for doing so. In total $20,000,000 is distributed. Company Alpha gets their cloud computing done for cheap, Company Beta pockets the $10,000,000 difference for running a network, the individuals make money with their graphics cards, except this time it's actual United States Dollars. What am I missing here that would make this type of business unfeasible?
- londons_explore 4y agoML typically needs decently high bandwidth and low latency links between computers too. Sometimes the algorithm or the data is commercially sensitive. Those are the two main reasons a 'rent out my GPU while I sleep' scheme wouldn't work. Neither are insurmountable though.
- deleted 4y ago[deleted]
- smoldesu 4y agoCompany Alpha will probably just call up AWS or Azure and sleep easy. Company Beta can't really compete because they have to pay both themselves and their contributors; AWS could sell ML compute at-cost and still turn a profit. What made this type of business feasible/attractive for cryptocurrency is that the miners were Company Beta. There wasn't two mouths to feed or two pockets to line, just a direct and transparent reward scheme for people donating compute. Projects like Folding@Home have leveraged world-scale networks before, but I'm not aware of anyone who's managed to monetize it.
- throwaway14356 4y ago
- londons_explore 4y agoThe easy way to find out is to try... You start a kickstarter, and when we get to our target of $10M, we can make a start on the computation!
- mensetmanusman 4y agoIf only crypto had started after gpt, then we would have had a clear goal of what to compute and how to incentivize the economics.
- asim 4y agoSo is blockchain just an economic and architectural model to run crowdsourced things of value like GPT models? I mean it seems like a good first use case.
- mensetmanusman 4y agoThey usually aren’t calculating anything interesting, but in principal there could be something like that made.
- stealthcat 4y agoLast time I saw a project aiming to be like a fast decentralized REPL but with blockchain. https://github.com/Kindelia/Kindelia-Chain https://github.com/Kindelia/Kindelia-Chain
- jesuscript 4y agoStar Citizen raised half a billion dollars for a game that is never going to be released and all they did was promise a game in space with spaceships, sooo ... Yeah, I think we could crowdfund a billion dollars for this, but we'd need some really competent people making sure it gets used optimally.
- asim 4y agoI think the second part of what you said is key. Seeing this thread hit top of HN I can see there's a desire or need for something like this, but how you'd even go implementing it, I have no idea. I have mildly speculative ideas but nothing that would result in a successful outcome. So it's like OK, lets say you crowdsource even $20m, now what. What's the structure of this thing that has $20m, how do you use it, what do you build, how do you give access to other people? My gut says, co-op style entity, no shares, its membership based, people put in the money for that and infra costs. You don't try do anything fancy, buy compute for cheap, Hetzner, bare metal elsewhere, whatever, find the best open source project currently for GPT and start to run it, provide an API to it and then start to gauge feedback, do everything in the open, hold weekly meetings, be totally transparent about the costs. Ultimately sustainability is still hard because people are crowdfunding you, so is that forever? Is there a self sustainable model there? So many questions.
- jesuscript 4y agoYou'd need a good volunteer team of experts willing to shepherd it first and foremost. For it to be a volunteer team you'd need everyone to ideologically believe in the idea. They should be able to build an outline, line item wise, how each dollar is spent. I think that will help get it started. Then swap to a non-profit monthly sub model where people are simply paying for use (pay out the cost for labor and hardware at that point only). A utility bill essentially.
- asim 4y agoI like the analogy of a utility bill, that makes sense.
- DrNosferatu 4y agoBut what about something for individuals? You contribute n GPU minutes to the community adhoc pool, you get to use n minutes in parallel from said pool. Some proof of work underlying blockchain. Just for individuals, no promises of deep data / model privacy. Has anyone implemented something like this?
- solumos 4y agoCheck out Akash Network https://akash.network/ https://akash.network/
- latchkey 4y agoSupport for GPUs are still in the works. The larger issue here though is the amount of bandwidth and memory required. Downloading billions of images to train a model just isn't going to work. People are working on splitting the training into smaller chunks (together.xyz is one example), but they aren't quite there yet.
- est 4y agoBecause GPT models will be made smaller eventually. Like how people mirror the initial Stable Diffusion model, only to find better, faster and smaller new versions later.
- cellis 4y agoDistributed training could be done but how would data validation work? There would need to be some kind of voting per item as you need to assume adversarial input ( honestly a crypto token could work here for governance ). EleutherAI could be used to bootstrap but the hardware reqs for Neo-X are quite steep ( around 20 GB cuda memory per card , so you could only allow those who had a 3090+ and are skilled enough to install cuda 11 and do devops ). Im happy to contribute though!
- saneehaf 4y ago[flagged]
- iNic 4y agoI believe distributed compute *could* be one of the few applications where crypto might make sense? You could earn tokens by supplying compute and pay to get it, and the token price is a natural market price for compute.
- askiiart 4y agoEver heard of banano? It's pretty much a joke, but it is basically this.
- tonmoy 4y ago> Can groups of people pool together money to run shared models? Then share the profit among the group of people? It’s called a company.
- hcks 4y agoYou can, Microsoft is a publicly traded company
- sharemywin 4y agoI created a subReddit for this in case enough people want to actually try to make this happen. https://www.reddit.com/r/AiCrowdFund/ https://www.reddit.com/r/AiCrowdFund/
- edwnj 4y ago[dead]
- JL-Akrasia 4y agoHere's a project called AI Compute Token thats gaining some traction https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogDEAlDPV9icT_v2pfwYS8/edit?usp=sharing https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogD...
- fxtentacle 4y agoYou might be interested in the LEAM.AI initiative, which is basically the EU planning to fund the creation of an Open Source GPT-3 competitor. Their planning documents contain pages upon pages on all the related challenges, such as generating and storing the dataset, keeping your cluster running, tolerating node failures. In short: The compute time alone won't help you succeed.
- waffletower 4y agoUnfortunately, it can be difficult to crowd-source funding when funding services, like Kickstarter cave to opposition to controversial compute projects. Unstable Diffusion, despite quickly reaching their funding goals, was suspended on Kickstarter. https://www.kickstarter.com/projects/unstablediffusion/unstable-diffusion-unrestricted-ai-art-powered-by-the-crowd https://www.kickstarter.com/projects/unstablediffusion/unsta...
- deleted 4y ago[deleted]
- inc417 4y agoSalad.com is years ahead and ready for just this. :)
- ctoth 4y agohttps://github.com/THUDM/GLM-130B https://github.com/THUDM/GLM-130B might be a useful place to look
- bobjmiles 4y agoIt's no good for training, only inference, but there's salad.com that has 10k's of DAU with GPUs and they've built a managed container service, with a inference API coming soon https://salad.com/salad-inference-endpoints https://salad.com/salad-inference-endpoints
- Rebelgecko 4y agoElutherai?
- mshake2 4y agoMaybe someone can make a cryptocurrency where mining and "proof of work" is accomplished by submitting an adjustment to weights and biases such that it improves the test score on a public dataset.
- sinenomine 4y agoYes, you can. It's not rocket science - lambdalabs and preemptible instances on various public clouds are possible choices. And you probably should.
- ofou 4y agoCerebras will be the cheapest way, I think. Let's do the math.
- asciimike 4y agoA little old, but: https://khairy2011.medium.com/tpu-vs-gpu-vs-cerebras-vs-graphcore-a-fair-comparison-between-ml-hardware-3f5a19d89e38 https://khairy2011.medium.com/tpu-vs-gpu-vs-cerebras-vs-grap...
- asim 4y agoI didn't even think about this. They do have a cloud product in the works https://www.cerebras.net/product-cloud/ https://www.cerebras.net/product-cloud/ Wonder if crowdfunding could be used to gain shared access to this
- charles_f 4y agoMaybe a model like seti@home?
- haldujai 4y agoWhat is your desired objective and what are you going to train on? There are plenty of publicly available model checkpoints so you don’t have to start from scratch with large GPU clusters. There’s no point in repeating the pretraining that has already been done. I’m not sure that compute is still the bottleneck right now, it’s fairly cheap to train LLMs. Many optimizations like DeepSpeed dramatically reduce computation requirements / increase throughout (I.e. 13b model on a single GPU). If we’ve learned anything from the massive LLMs like PaLM it’s that scaling autoregressive models to infinity has diminishing returns and is not feasible to implement in inference infra. Google themselves acknowledge this resource limitation in the Med-PaLM paper when they discuss fine tuning a 450b parameter model. We’re really in more of a dataset and training task era of AI/NLP gains. Scaling masked the issues of poor quality training data and conventional language modelling up to a certain point but we’re starting to see the problems with that (hallucination) in all of the big (Galactica is an example of the limitations in this). OpenAI’s main advantage is that they paid humans to build a large labelled dataset for their RL objective. They’re offering ChatGPT for free to get more training data.
- aaron695 4y ago[dead]
- deleted 4y ago[deleted]
- pontifier 4y agoI guess I may be behind the times on some of these AI efforts, but I'm guessing that the model involves a big black box blob of data. I'd be interested to know how models evolve with additional training, and wonder if there are additive identities that would allow two separately trained models to be combined.