8 ms·

I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mi

• londons_explore 4y ago
  ML typically needs decently high bandwidth and low latency links between computers too. Sometimes the algorithm or the data is commercially sensitive. Those are the two main reasons a 'rent out my GPU while I sleep' scheme wouldn't work. Neither are insurmountable though.
  • deleted 4y ago
    [deleted]
• smoldesu 4y ago
  Company Alpha will probably just call up AWS or Azure and sleep easy. Company Beta can't really compete because they have to pay both themselves and their contributors; AWS could sell ML compute at-cost and still turn a profit. What made this type of business feasible/attractive for cryptocurrency is that the miners were Company Beta. There wasn't two mouths to feed or two pockets to line, just a direct and transparent reward scheme for people donating compute. Projects like Folding@Home have leveraged world-scale networks before, but I'm not aware of anyone who's managed to monetize it.
  • throwaway14356 4y ago
    It would require a problem that reasonably fits the solution. You for example get a keyboards and a set of hands with each node, a webcam, a microphone. I have no real idea but it seems the hand of cards isn't hopeless.
  • humanizersequel 4y ago
    >AWS could sell ML compute at-cost and still turn a profit How? Definitionally, they wouldn't be turning a profit.
    • smoldesu 4y ago
      Traditionally, people don't just get one thing from AWS. If their ML compute doesn't make money, S3 Object Storage will. My bigger point is that AWS and Azure have much greater control over their margins than a business like the one OP was describing.
  • latchkey 4y ago
    Large enough miners don't have the right hardware. Mining requires a tiny cpu, very little ram and gpus with a max of 8gb, oh and 100mbit. There was never an incentive to buy hardware that could be reused for other purposes since mining itself was such a cash cow. Company Beta likely started off with hardware for AI/ML and mined on the side when they didn't have enough customers.
• WithinReason 4y ago
  This exists already, but I can't find a link. Edit: I was probably thinking of vast.ai
• qwertyforce 4y ago
  https://vast.ai/ https://vast.ai/
• IshKebab 4y ago
  Yeah someone posted something like that on HN maybe a year ago. It's not especially useful though because most companies (who are actually going to pay for this service) aren't going to want to send their training data to random people, and ML training needs high performance links between the cards. Plus you'd have to deal with the fact that you're running on 100 different GPU models.
  • brap 4y ago
    In addition to privacy, performance, and portability, also: * Servers in a datacenter are much more reliable than a network of PCs (power goes off, someone decides to play Crysis, etc) * People will find ways to scam you (pretend like they’re doing the calculation while not actually doing it) * Economies of scale means a datacenter will probably be cheaper than what you’d have to pay the PC owners (power consumption, network, wear, etc) * The PC owners will have to trust this arbitrary code that they’re running (can you assure that there’s never a jailbreak resulting in a huge botnet?) I’ve pitched this idea more than a decade ago and quickly realized it has many issues.
    • drakenot 4y ago
      What if you remove the financial incentive? I'd contribute my GPU time to a Folding@Home style project if it meant that we had powerful, open LLMs that were free to use. I'm positive many others would as well. As far as worrying about scammers, could you send the same compute task and training data to multiple clients and validate the results against each other? If they differed, you could throw the results and try again, or send it to a 3rd to break the tie.
      • brap 4y ago
        You could remove the financial incentive, but then you’re really limiting the pool of resources (you have to rely on people’s kindness, which only works, partially, in noble cases like finding a cure for cancer or searching for extraterrestrial life). By sending the work to 3 people each time, you’re effectively cutting your (already limited) resource pool by 66%.
• rubyist5eva 4y ago
  > Like mining bitcoin except you are doing something actually useful and getting paid actual money for it But then how would crypto-scammers run ponzi after ponzi after ponzi if they did that?
  • throwaway14356 4y ago
    oh? That seems to be the answer. They wont actually have to care what the compute costs if they can print money back stage.
    • rubyist5eva 4y ago
      Sounds like work, as long as they prove they can prove they did it somehow - perhaps all the other computers that are sharing compute could also verify they did it somehow?
• qorrect 4y ago
  There is this https://vast.ai/hosting https://vast.ai/hosting
  • alexb_ 4y ago
    Nice! Knew once I thought about it that someone had to have done it already.
  • jw1224 4y ago
    Also https://www.runpod.io/ https://www.runpod.io/ (which I’ve been a very happy user of)
  • latchkey 4y ago
    https://cloud-gpus.com/ https://cloud-gpus.com/
• cuuupid 4y ago
  Surprised no one has commented this but the latency requires the model to be trained in tiny fragments on each device which is currently a field of research that is being explored. As it stands now basically all of a model needs to be loaded into memory. There’s a whole field here and people exploring this problem, colloquially solving this would enable Federated Learning and whoever figures this out will far eclipse OpenAI (if it’s ever solved).
  • k2xl 4y ago
    Where can I read more about this field of research?
    • latchkey 4y ago
      https://arxiv.org/ https://arxiv.org/ has a ton of papers on it.
      • elicksaur 4y ago
        Do you have some good search terms to get started down the rabbit hole?
        • franl 4y ago
          I too would like to go down this rabbit hole. I am going to poke around using the terms “distributed learning” and “federated learning” (They’re different areas, but somewhat related as far as I understand).
        • franl 4y ago
          This one is a few years old, but seems interesting: https://arxiv.org/abs/1802.05799v2 https://arxiv.org/abs/1802.05799v2
        • vagabund 4y ago
          Probably the biggest recent result: https://arxiv.org/abs/2209.04836 https://arxiv.org/abs/2209.04836 (author thread: https://twitter.com/SamuelAinsworth/status/1569719494645526529 https://twitter.com/SamuelAinsworth/status/15697194946455265...) See also: https://github.com/learning-at-home/hivemind https://github.com/learning-at-home/hivemind and more to OP's incentive structure: https://docs.bittensor.com/ https://docs.bittensor.com/ Latter two intend to beat latency with Mixture-of-Expert models (MoEs). If the results of the former hold, it shows that with a simple algorithmic transformation you can merge two independently trained models in weight-space and have performance functionally equivalent to a model trained monolithically.
• throwawaysleep 4y ago
  https://www.dcp.dev/ https://www.dcp.dev/
• sramam 4y ago
  https://petals.ml/ https://petals.ml/ seems to be headed in that direction.
• welshwelsh 4y ago
  >Like mining Bitcoin except you are doing something actually useful and getting paid actual money for it A business paying USD is never going to be competitive with a decentralized crypto compute market. Better to build something just like mining BTC, except you do useful work, and are paid in a cryptocurrency you can exchange for USD. Then businesses can build on top of that to make it more user friendly. The Golem network already lets you do this with CPU. It's way cheaper than centralized cloud computing could ever be, less than $0.003 per core per hour. There are plans to add GPU support to Golem, but they have been on the backlog for a long time because there is a shortage of devs.
  • theamk 4y ago
    I don't think businesses would be able to ever trust decentralized compute market. Where is the guarantee that the data/code that is sent to the network would not be intercepted and analyzed? It's much safer to sign a contract with a real organization, especially the one which has a reputation to uphold. If someone like DigitalOcean steals data from servers, they can be sued, and there will be penalties. If some decentralized miner somewhere does that? Nothing you can do. This only leaves decentralized compute to the applications where neither the data nor code does not matter. While I am sure there are applications like those (probably related to open-source or cryptocurrencies) I doubt they'll bring much money. (And don't say "homomorphic encryption" -- the overhead there is so high it is much cheaper to just get a centralized server)
• jakespracher 4y ago
  Vast.ai does this
• deleted 4y ago
  [deleted]
• rysertio 4y ago
  There's a lot of problems. 1. How can I confirm that you've done the computation? 2. Privacy and security issues. Can I trust you too process my sensitive information? 3. Availability: is there a guarranty you won't just do half of it and then be on a hiatus for 2 months. But for everywhere these problems are solved we have decentralized cloud computing. For others you need to solve these problems.
  • bitxbitxbitcoin 4y ago
    Some blockchain projects attempted to tackle this. Check out Akash.
    • latchkey 4y ago
      Akash is more about k8s, paid for by tokens.
  • latchkey 4y ago
    There are a number of entities working on this problem. Plenty of papers in Arvix on it as well.
  • kordlessagain 4y ago
    The Lightning Network may be used for invoices, identity and payment of use of model. It can also likely be used to handle prompts. Regarding privacy, not all things need to be private and if they do they should be run on a single tenant infrastructure. OpenAI is a multi-tenant cloud model, so guarantee of privacy there may be protected only by license and use agreement.
  • alasdair_ 4y ago
    >1. How can I confirm that you've done the computation? The same problem applies to things like Mechanical Turk and other croudsourcing. The way I've dealt with the issue in the past is to start with zero trust and to have them do computations that I already know the answer to. After that, they do computations that are matched with a random other participant (the two results should match, if they don't, compare against a third random participant). Later, when trust has been developed, you can start assuming their work work is trustworthy, but still check it randomly with computations you know the answer to, or a second person doing the same computation. Yes, this adds overhead (roughly 10% in aggregate) but it works fairly well. It works even better if there are penalties that you can impose for ffraudulent results (like cancelling ALL owed payouts).
• moneywoes 4y ago
  Why not just use AWS?
  • latchkey 4y ago
    Price and dependency on a single entity.
  • alasdair_ 4y ago
    > Why not just use AWS? Physical goods vendors that sell through Amazon have found that once they sell something highly in demand, and profitable, Amazon uses all of their internal knowledge of the sales to create their own Amazon Basics version of the same good and then promotes that as a cheaper alternative, capturing most of the profit. There is absolutely no reason not to believe that AWS will do the same thing and use their knowledge of the workloads and exact hardware requested to compete with you. The fact that they haven't done so yet isn't much evidence against them doing this in the future, since other parts of the same company already follow this approach.
• eptcyka 4y ago
  Data locality is an issue with large models even when using supercomputers with very high bandwidth fabric.
• latchkey 4y ago
  The compute workloads for AI/ML are vastly different than mining, which means that the hardware requirements are totally different (and actually a huge capex investment). Individuals renting out their home compute for things like this suffer from the problem of supporting so many different types of hardware profiles. BOINC is a prime example of this, but it doesn't have a funding mechanism.
• shivekkhurana 4y ago
  There is a service like that on Ethereum. It's called Golem [https://www.golem.network/ https://www.golem.network/].
• Aaronstotle 4y ago
  There a few crypto projects that aim to do this, this one is focused on artists https://rendertoken.com https://rendertoken.com I'm not sure whether or not they will be successful.
• JL-Akrasia 4y ago
  Here's a whitepaper for a project called AI Compute Token: https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogDEAlDPV9icT_v2pfwYS8/edit?pli=1 https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogD...
• drexlspivey 4y ago
  When Playstation 3 came out there was an app that shipped by default where you could leave it running and it would use your CPU for protein folding calculations to help researchers figure out the structure of proteins.
• fxtentacle 4y ago
  You can rent out your GPU to vast.ai But the issue with large language models is that you need ms-latency access to TBs of data, or else you won't be able to saturate your GPUs with useful work.
• NonTrivialZer0 4y ago
  The ACT token is a decentralized, token-based solution for powering AI inference that can be used to purchase compute resources on GCP, AWS, Azure, and the decentralized network. Each token is equivalent to a certain number of inferences by the AI Compute Unit (ACU), which is a stable average of GPU compute instances available in the pool.
• Patrick_Devine 4y ago
  I tried building a startup around this back in ~2012 called "Netkine". The basic premise was that compute is fungible and that all compute has a price (regardless of whether it is from a datacenter, or from someone's desktop PC). We seeded different compute on different networks around the SF Bay Area (either in low cost datacenters, or different networks like Comcast/AT&T) and had a customized version of Linux which could securely boot on a Windows gaming machine. You could then carve out different sized virtual machines on any machine in the network, and it would seamlessly create a VPN tying each of the instances together. Our biggest problem was finding customers on the demand side. What is the killer application which can take use of that kind of compute with those kinds of networking properties? What are the use cases? No one has written software that takes advantage of that kind of compute, because it's essentially a solution looking for a problem.
• tehsauce 4y ago
  It does exist! Those most notable are vast.ai and runpod.io I've been able to run ML experiments that would have cost 10s of thousands of dollars for a tiny fraction of that using vast.
• asciimike 4y ago
  Lots of these services: - Fluidstack: https://fluidstack.io https://fluidstack.io - Vast: https://vast.ai https://vast.ai - QBlocks: https://qblocks.cloud https://qblocks.cloud - RunPod: https://runpod.io https://runpod.io - Sonm: https://sonm.com https://sonm.com (blockchain) - Golem: https://golem.network https://golem.network (blockchain) - Rentaflop: https://rentaflop.com https://rentaflop.com (rendering specific, blockchain) - RNDR: https://rendertoken.com https://rendertoken.com (rendering specific, blockchain) If you want HPC specific cloud providers: - Crusoe Cloud: https://crusoecloud.com https://crusoecloud.com - Coreweave: https://coreweave.com https://coreweave.com - Lambda Labs: https://lambdalabs.com https://lambdalabs.com - Paperspace: https://paperspace.com https://paperspace.com As others have pointed out, the decentralized clouds can't offer high performance interconnects (e.g. InfiniBand) that a lot of folks are using for LLM training. There are definitely initiatives underway to reduce dependence on these interconnects and build performant distributed training (again, some threads below mention this), but I think it's mostly academic at this point. Disclosure: I run product at Crusoe Cloud, which aims to provide ML training at half the cost of a hyperscaler, while also being carbon reducing (https://crusoecloud.com/climate-impact/ https://crusoecloud.com/climate-impact/).
• WhackyIdeas 4y ago
  Considering the power of these AI models, it is quite clear to me that companies like Microsoft and Google should ABSOLUTELY not have the monopoly on them because they simply cannot be trusted as is, let alone with powerful AI. If there was a feasible crowdfunded solution to this and putting it in the hands of the people - I would certainly be prepared to lay down up to £5K.
• FractalHQ 4y ago
  Sheepit[0] does this with peoples Blender files, it’s awesome. I use it often and donate gpu time when I’m not using it. [0] https://www.sheepit-renderfarm.com/home https://www.sheepit-renderfarm.com/home