8 ms·
Vast.ai – marketplace for renting out your GPU, or renting someone else's GPU
- Thaxll 5y agoWhat code do you actually runs on those instances? For example is it possible to somehow run hashcat on a rented maching?
- Gelob 5y agoYes. You get a full Linux shell. You can deploy images and some already with hashcat installed
- The_rationalist 5y agoHow much does it cost for training BERT large?
- bruhhh 5y agojust do it on colab, save checkpoints, and keep training
- The_rationalist 5y agothx bruh
- jeroenhd 5y agoAh yes, a new take on the botnet-as-a-service genre. Running untrusted code, even inside a container, is a terrible idea. Container escapes get discovered all the time and even without them you could get yourself in serious trouble with the police by just letting random people use your network. Sure, you'll be able to defend yourself in court, but nobody has time to go to court over this stuff. If you run this inside a VM with a PCIe paasthrough and all network traffic tunneled through some kind of VPN then maybe it's worth the effort, but I just wouldn't risk it.
- winrid 5y agogood luck getting your hardware after it was confiscated, too.
- cinntaile 5y agoWhy bother though, the number of machines you can hack is very limited.
- judge2020 5y agoThe threat is using your IP address to do illegal stuff (read: serve/proxy CSAM).
- jeroenhd 5y agoThese decides have their performance and internet connection uplink listed, which make for very interesting targets. Yes, a hundred shitty smart scales are a nice target, but they're underpowered for more involved attacks. With a fiber uplink, a decent CPU and plenty of disk space advertised, this service can be very attractive, especially since you only need to pay for a minute to escape the boundaries of the container runtime if there's an exploit. Also, it doesn't matter that there's only a few computers if you're using other people's computers to break the law.
- deleted 5y ago[deleted]
- aaron695 5y ago
- tehsauce 5y agoAbsolutely incredible how cheap the compute power is on this platform. Vast has saved me tens of thousands of dollars running ML experiments.
- loxias 5y agoI love the idea of this. As a potential user, I'm disliking how many clicks it takes to find even an approximate price. Prices should be easiest to find.
- joshvm 5y agoYou can see spot prices here: https://vast.ai/console/create/ https://vast.ai/console/create/
- chockchocschoir 5y agoFrom the homepage (https://vast.ai/ https://vast.ai/) it's literally one click ("Search Marketplace" button). From the FAQ page, it's literally two clicks (first homepage, then marketplace button). Or am I missing something obvious here? Maybe the page looked different 7 hours ago?
- dylan604 5y ago"Search Marketplace" is not the text for a button I'm expecting to pricing. After looking further, I resorted to clicking "Search Marketplace" but gave up after it took forever with zero indication that anything was happening so I clicked something else. Something about the first loading of that page takes a lot of time. Since it's faster after the first load, so I'm assuming some large initial download (probably a large JS library???).
- 5y ago
- DenseComet 5y agoThis seems like a no go for anything where the dataset used for training includes any private data at all, or even public images that have been privately labeled by the company. It makes sense for hobbyists, but I doubt this entire platform was created for only hobbyist use in mind. They say they intend to implement encrypted hosting environments in the future, but considering the number of security exploits that Intel SGX and its equivalents have had, I'm not sure I'd trust that either.
- easton 5y agoAn interesting sci-fi/spy thriller could be getting someone arrested by training NeuralHash 2.0 with CSAM on their rig using a service like this. After all that news last year I'd be super paranoid about that, even if it's extremely unlikely that a hobbyist would attempt such a thing without backing from law enforcement.
- rileyphone 5y agoIANAL, but isn’t this against GPU terms of service, as in why companies can’t just host 3080s etc in cloud?
- callmekit 5y agoThe customer use license specifically talks about datacenters, not "cloud": "The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted" https://www.nvidia.com/en-us/drivers/geforce-license/ https://www.nvidia.com/en-us/drivers/geforce-license/
- deleted 5y ago[deleted]
- mhitza 5y agoTIL that a GPU has a ToS?! What's next ToS'd RAM and storage?
- callmekit 5y agoTechnically it's not ToS to GPU, it's a license to the software: https://www.nvidia.com/en-us/drivers/geforce-license/ https://www.nvidia.com/en-us/drivers/geforce-license/ So using a completely open source or third party driver and no CUDA is fine.
- paxys 5y agoNeat idea, but it seems to be born out of current market conditions and might not be sustainable going forward once supply/demand for GPUs normalize. As someone running these workloads I'd only be desperate enough to pay $250/mo for a 3090 today because I cannot buy it for $1500 at the store straight up. And if you drop the price by a third now it isn't worth it for the host anymore.
- d110af5ccf 5y agoElectricity plus some small premium should be tenable under any market condition. At least it works for public clouds so far even as hardware prices have fluctuated over time. There will likely always be someone who'd rather briefly rent than buy. I do wonder if what's charged for electricity and premium can be competitive with large data centers in the long run. Amazon already has spot pricing. I expect it would be favorable for consumers with cheap electricity or who are already paying to heat their house. I also wonder what effect renewables increasing electricity spot price volatility might have on this market in the future.
- paxys 5y agoPublic clouds have economies of scale, reliability and capacity all working in their favor. There's a reason there has never been a market for renting out spare CPU cycles or hard disk space or any other resource piecemeal from your computer.
- lmeyerov 5y agoDO and friends wouldn't exist in the CPU side if that was true, so a spot economy in GPU side makes sense, and even more so bc they are higher-end time shares Even if we didn't have an open ticket with Azure since Christmas for more GPUs, the ability to burst non-sensitive tasks on say 20 cheaper GPUs for 5-30 min is attractive. That stuff adds up fast, and lowering cost both opens accessibility for low-end users and helps scale what power users can do. My bigger surprise is vast has been around awhile and only ~20 available servers, so I'm guessing < 100 total. Is this a friction issue?
- thebean11 5y ago
- vintermann 5y agoI tried out this service last time bitcoin was in a slump, in 2018. I was out of work, and toying with the idea of commercializing a Norwegian (bokmål-nynorsk) language model which I had trained. Prices for compute were competitive, but most instances had lousy bandwidth. You don't need bandwidth to run bitcoin miners, apparently. The people behind it were very friendly and helpful. However, there was an incident in 2020, long after I'd gotten a job and abandoned my ML project. They tried to charge my card a small amount ($1.70), and sent a message that my card was expired, which indeed it was. I asked them what that was all about, since I'd not rented any instances for years, and I never got any explanation other than "that's weird".
- Reubend 5y agoWow, I was definitely considering trying this out until I saw your comment. They charged you mistakenly, didn't detect it on their side, and then never gave you a reasonable explanation for why? For me, that's a definite no-go.
- robga 5y agoI’ve used vast.ai many times over several years. I use the “top up” approach. Never any unsanctioned payments.
- franga2000 5y agoThat's such a small thing tho. It was a tiny amount and presumably they would've refunded it had it gone through. It's also possible they would've discovered it themselves soon enough, since many companies only look at the money books at the end of the month. Also, I doubt you'd get a "reasonable explanation" from any other company - at best they'll say "sorry it was a (human|software) error, we've taken steps to ensure it won't happen again", as disclosing anything more is usually against company policy.
- bruhhh 5y agothe amount doesnt matter, it's about getting charged for no reason. You never know how much it would be for the next person they charge
- ghoomketu 5y agoSpeaking as someone who has very little knowledge of ML, let's say I wanted to create a site like www.remove.bg and wanted to use this service to training my own models (from scratch). Is it feasible to do? How many hours ballpark would it take to train for something like this. How does it upload my training images to the target machine? I see the interruptible pricing for RTX3080 is $0.130/hr. Anybody with some ML experience calculate approx cost using this? P.S. I have no intention of creating a remove.bg clone, just curious if it can be done and if so how much approx it will costs.
- nicd 5y agoI would guess a few days of training, but this will vary widely depending on the type of model and how much data you train on.
- avipars 5y agoI'm wondering why researchers would rather use this than pay AWS? Is it significantly cheaper?
- Blahah 5y agoIt's practically impossible now to get new GPU capacity on AWS, so AWS is not really a competitive option for labs that didn't have it before the recent restrictions. Vast.ai is much more usable.
- avipars 5y agoSurprising
- robga 5y agoYes, it’s been a while since I used it but about half the price of an AWS spot instance.
- avipars 5y agoHalf price is a good deal
- quanto 5y agoI am interested in the technical aspect. What is the tech stack to enable such a vastly distributed host network? Presumably, people have to open up their home networks, but the company has to ensure that the host machines are secure.
- andybak 5y agoWell that answers my question "Would it be worth renting out my unused 1080?"... No...
- bruhhh 5y agoI wonder how much data mining is going on on these machines. They can easily benefit from the huge free, potentially well-labeled, data they can re-sell to other companies
- boredumb 5y agoI've always wanted to create something like this and make people use a specific cryptocurrency to rent or receive payments that they could also mine with a GPU and call it 'Ouroboros'.
- hamsamrma 5y agoplease do
- ada1981 5y agoCan this be profitable for folks to buy new gear for? They claim 2-4x crypto revenues, and crypto revenues are basically break even now right?
- gmemstr 5y agoWhile I love the idea, the requirement for Ubuntu means I can't contribute my compute though (and neither can any of the PC gamers with Windows machines, or anyone else on other distros it seems). As much as I would love to do this over mining a cryptocurrency (which I don't do in the first place), I can't.
- andrewstuart 5y agoI found this service interesting. This is how it pitches itself: "Rent out your GPU to make your hobby pay for itself. Transform your mining farm into a GPU training center and earn ~2x to ~4x more per gpu-hour than mining cryptocurrency. We connect you with customers and provide simple tools to streamline hosting. You set your own prices and schedules. Get started today." The biggest obstacle of course is security. The founders described it as "kinda like airbnb for compute" here: https://www.reddit.com/r/gpumining/comments/8xu04h/vastai_beta_is_live_machines_can_be_listed_today/ https://www.reddit.com/r/gpumining/comments/8xu04h/vastai_be...
- a_bonobo 5y agoFor me, the second-biggest obstacle would be data size and correlated, data transfer speed. I regularly analyse GBs to TBs of data on supercomputers which have PBs of free space, regular consumer PCs don't have the harddrive space for that kind of thing. And even if they have, they may not have the transfer speeds to make copying my data there feasible - by the time copying has finished, I may have already run the analysis on a slower CPU
- HWR_14 5y agoAre the PBs of free space necessary? I thought training a ML model on a terabyte of data was computation expensive, but not storage intensive.
- a_bonobo 5y agoSometimes! Not for the GPU part, but to make the summary data for the ML data. For example, genome sequencing data and intermediate results are easily in the TB area of space, but the resulting table of genomic variants (k-mers like n-grams in NLP) is only a few hundred GB.
- HWR_14 5y agoI'm confused from your post where the petabytes of storage come in.
- realPubkey 5y agoUsed this several times, works great. But for smaller training tasks (max 1 GPU), colab pro plus is much cheaper
- deleted 5y ago[deleted]
- tullie 5y agoLove vast.ai. Only wish is for it to be easier to work with persistent disks over time. I only use it for one off expensive jobs now.