11 ms·
Rent out GPUs to AI researchers and make ~2x more than mining cryptocurrencies
- inetknght 9y agoGPUs are already too expensive for gamers and now you want to do this?!
- zawerf 9y agoIsn't this good for gamers? They get to sell their idle gpu time and basically get a gpu for free. If he's really paying 2x more than cryptomining, this translates to paying off a gtx 1080 in a few months even taking into account electricity costs.
- 0xfaded 9y agoHow do GPU's "wear", or in other words degrade with use. I often see people stating they would not buy an ex-miner GPU because of it's hard life.
- KVFinn 9y ago>How do GPU's "wear", or in other words degrade with use. I often see people stating they would not buy an ex-miner GPU because of it's hard life. It's kind of overblown. If you run well within the temperature and power limits they are still usually good for years at 24/7. Usually they run a bit hotter or slower because something has evaporated or slightly worn out, or maybe a fan died that needs to be replaced, but otherwise are fine. If you are lending out your own GPUS presumably you can set a reasonable power and temperature limit.
- snuxoll 9y agoI'd be lest worried about the GPU itself dying and more the fans running 24/7 at higher speeds. Still, a decent fan should last for years even running at 100% PWM - and most GPU's come with fairly length (often lifetime) warranties. I would love to let my 1080 Ti be put to use when I'm not playing games on it, cryptocurrenty mining was fairly profitable until a month or so ago and now the power usage (even at $0.10/KWh) isn't covered by the returns after paying the tax bill (I could always file the revenue under my LLC and deduct the electricity expenses and even deprecate the hardware cost, but without a dedicated rig and power monitoring it's a tax audit nightmare waiting to happen).
- largote 9y agoThe silicon itself gets effectively no wear and the board is actually better because it doesn't get heat-cycled as much. The fans are in a bit worse shape, but those tend to be easily replaceable if they fail.
- proofofmoon 9y agoAI research demand is a lot easier for AMD/Nvidia to project, which allows them to ramp up production and keep prices stable. Side note: AI research is more beneficial to society than running a few instances of Crysis 9 at 4K 120HZ.
- Samin100 9y agoAlso much better than calculating the inverse of hash functions!
- lovelearning 9y ago...potentially more beneficial. Also potentially very harmful depending on what it gets used for.
- blattimwind 9y agoLet's just say that AI research can be useful, and thus this use of GPU time and electricity can be useful; in contrast to the alternative alternative-GPU-use, which is not useful in the first place.
- ClassyJacket 9y agoIf you were paying enough that it was worth people buying GPUs just for this, you'd be better off just buying them yourself.
- mathgeek 9y agoMaybe. Since demand already outstrips supply for GPU's, it might be better to be the scale owner than to buy pickaxes.
- corysama 9y agohttps://medium.com/@rendertoken https://medium.com/@rendertoken aims to do the same thing, but for 3D render farm customers.
- dpwm 9y agoThere's a lot of comments on that reddit thread about how awesome this would be as a service. But there's a big problem of trust with this for ML. How do I know you actually ran what I paid you for and not just generated random data that looks right in the shape I wanted it? You could farm it out to two people and if the results disagree, then payment is decided by a third. But then you've just doubled the (paid) workload, and you've not really solved collusion by a significant portion of the workers.
- TrainedMonkey 9y agoThis is identical to the problem blockchain solves - hard to solve, but easy to verify. For example, if the workload is training a neural network on a set of samples it is relatively cheap verify performance on a training sample. Alternatively you could give X% of tasks to multiple workers for cross checking. In your example X% is 100%, but it does not have to be that high.
- mattkrause 9y agoSay you do a single feedforward pass. How do you verify that its output is the result of a trained network-and particularly a completely trained one?
- TrainedMonkey 9y agoAside from doing the same work X% of the time I can think of only one other option. The network could be overloaded with additional nodes such that it would also compute some function on the input data with the known output. For example a hashing function. Having said that, single pass does not strike me as a particularly useful unit of work. You would want to batch the samples, and that leads back to a strategy of verifying a portion of them based on the host trust metric.
- gormanc 9y agoAccording to the website (https://vectordash.com/hosting/ https://vectordash.com/hosting/) they use a highly isolated Ubuntu image, so the person hosting the service shouldn't have access to the VM with your model or data on it. It would be nice if there was some third party audit of the software though, the models, the code, and even the training data can be pretty sensitive for researchers.
- eridius 9y agoThis uses LXC Containers. I've always heard that containers are not a replacement for a good security model. What's the risk that someone could use Vectordash to attack the host computer?
- Samin100 9y agoLXC unprivileged containers are actually pretty secure by design. The Canonical LXC page does a pretty great job of explaining why: https://linuxcontainers.org/lxc/security/ https://linuxcontainers.org/lxc/security/
- dgacmu 9y agoPlease (OP, and others allowing access to your machines), be cautious on this front. While LXC unprivileged containers are a start at isolation, LXC containers + GPU passthrough has a much, much larger attack surface (the nvidia binary blob drivers are complex and out of your control). The most common ways of giving GPU access to a container involve installing the CUDA drivers in the host and simply allowing the container to access them. Depending on your threat model, this may be borderline OK, or may be insanely high risk. We've seen driver exploits in the recent past: http://nvidia.custhelp.com/app/answers/detail/a_id/3140/~/security-bulletin%3A-nvidia-unix-graphics-driver-exploit-advisory http://nvidia.custhelp.com/app/answers/detail/a_id/3140/~/se.... that can vector through the GPU to gain access to arbitrary host memory (aka, you're in deep trouble). The only "right" way to give access to your GPUs requires IOMMU support and running in a true guest VM, which Nvidia's consumer cards and drivers are explicitly prevented from doing (because they'd like you to buy the Tesla versions, thanks). This may be a totally acceptable risk on an isolated mining rig, but people should be aware of the heightened risks they're exposing themselves to. Breaking your machine is well within the capabilities of state-sponsored actors on this one, and from time to time (just after vulnerabilities get announced), could be decently within advanced-script-kiddie zone.
- Samin100 9y agoThis is a very, very valid point. I'm going to mention this on the website & even advise people to run the client on an isolated machine (instead of their daily driver gaming rig). Thank you for pointing this out. Also, one of my friends working on this project is a sophomore in the CS dept at CMU, and given your interest in distributed systems & DL, would it be possible to meet up for a couple of minutes and discuss security a bit more in depth? (if yes, I'll shoot you an email)
- deleted 9y ago[deleted]
- Samin100 9y agoCreator of Vectordash here! If you have any questions about the platform, feel free to ask away! P.S. I'm @samin100 on Twitter if you enjoy tweets about GPUs!
- chronial 9y agoYou mention "free unlimited bandwidth" on the homepage in a column named "1 TB transfer". A lot of people have upstreams in the order of 1MBit/s – with such speeds, uploading a TB would take 92 days. That seems misleading ^^.
- Samin100 9y agoAgreed! A lot of hosts have requested for the ability to set a bandwidth cap. I'll change that part of the site & be a bit more clear!
- Flammy 9y agoI saw people on the Reddit thread saying the Ubuntu requirement was a showstopper for a lot of them. You might look into the Win 10 linux support, Ubuntu is one o the supported distros. Not sure if it would have full access to resources, have just used it a bit at work and setup was super simple.
- Samin100 9y agoI remember a few weeks ago a when some Microsoft software engineers came to the career fair at Univ. of Maryland. One of them was working on Bash on Ubuntu on Windows, so I ended up bugging him for about an hour on when they'd support GPU passthrough for Ubuntu on Windows. He said it was on their list of most requested features, but refused give me a data on when it would be ready :/
- flowless 9y agoKind-of +1 about this, I don't really want to run dedicated Ubuntu for something like this but I have no problem running virtualized Ubuntu with GPU via VTd passthru. Recently we've built such host - server board with 2 GPUs that now runs two powerful virtualized desktops. Host uses NixOS and libvirt.
- senatorobama 9y agoHow do I know if I will make a profit given electricity costs?
- Samin100 9y agoThis website has a pretty good calculator: https://whattomine.com https://whattomine.com Just replace your miner earnings with the earnings listed on http://vectordash.com/hosting http://vectordash.com/hosting
- Danieru 9y agoThis sounds pretty awesome. I wish I had perminent internet to the house so I could offer my rigs. I hope this service takes off, it should be a much better use of electricity and gpus than mining. Please do keep developing this, with the down turn in mining there should be a large pool of potential gpus. If this service can connect those gpus to researchers then that should be a lucrative business Ling term.
- boulos 9y agoDisclosure: I work on Google Cloud (and launched Preemptible VMs). Cool! Like you, I wish people would make productive use of spare cycles. Can I suggest you add a note comparing this to BOINC/gridcoin as well? I think your marketplace is a better idea, but because of the security implications dgacmu pointed out downthread, it shouldn’t be treated lightly. Also, I really like your white paper (https://vectordash.com/primer https://vectordash.com/primer) as it’s nicely formatted and has pretty icons. I think you should update the comparisons to the cloud providers though. Comparing a Pascal-series (1080 Ti) part to a Kepler part (K80) isn’t particularly fair, as NVIDIA charges about the same for that part as the later P100s and V100s. That is, people should be comparing what the best price/performance ratio is, and all three providers have better options. There’s a secondary question about whether the right comparison is to AWS Spot / Preemptible VMs / Low Priority VMs, but I’m not sure from a quick skim what the likely preemption rate will be with your setup. Finally, let me echo dgacmu in saying it’s too bad that NVIDIA forces all providers to only provide the expensive parts. The best price/performance is certainly the consumer parts (that 1080 Ti compares favorably to a P100 at a fraction of the cost), but both the EULA and otherwise prevent it. You’re not running a direct service / providing things in a data center which is a cool workaround! But you should definitely be more clear about the risks of letting untrusted third-party code to execute against your GPU. Again, great project!
- navinsylvester 9y agoWe at Gridmarkets have cracked this one(spare cycles) and are now further enhancing the model.
- tudorconstantin 9y agoGreat idea and I am glad somebody actually tries to make an implementation. But unfortunately this won't replace mining: the large scale mining farms have high end GPUs in their rigs, but the rest of the HW is very low end, because that works perfectly for mining and they want their ROI as low as possible. I have a 6 GTX 1070 GPU rig, which would be decent GPUs for AI/ML, but the rest of the rig has a Celeron CPU, 4gb RAM,and a 5400 RPM HDD. Oh, and to be able to see all the GPUs, I had to downgrade all the PCIEx slots to 1x. I am curious what kind of ML tasks would be able to fully utilize these GPUs, on such a low end hardware.
- QasimK 9y agoThe ones that train on a GPU? I'm not sure the rest of the system will have that much of an impact.
- doall 9y agoConsidering for hosting. One concern is that I have to expose my IP address to unknown users. I don't know if it's possible, but it will be great that if I can hide my location and purely lend my GPU with less security concerns.
- Samin100 9y agoYou don't need to expose your IP address at all! The AI/ML researchers can only connect to one of our proxy servers instead of directly connecting to your machine. This means hosts do not have to mess with port forwarding or networking (even if your computer is behind a university firewall, you can still be a host!)
- deleted 9y ago[deleted]
- doall 9y agoWow that's great! Another question. It is always possible that I have some downtime (unstable network connection, accidentally shutdown my pc, power outage, etc) and it may become unfair that users having significantly less downtime (better network, UPS, etc) having same earnings with users with much more downtime. Is there any plan to deal with such kind of concern? By the way, Vectordash is the most interesting idea & service I have seen in a while :)
- ISL 9y agoWe're coming into the warm seasons in the northern hemisphere, but if this effort survives into autumn, it'll be quite tempting for home-heating. Stoked to see distributed-compute as a paying service making another try. One of these days, it is going to fly.
- evanb 9y agoPeople are trying this for mining: https://techcrunch.com/2018/03/08/qarnot-unveils-a-cryptocurrency-heater-for-your-home/ https://techcrunch.com/2018/03/08/qarnot-unveils-a-cryptocur...
- ezoe 9y agoAs others have already pointed out, there are trust issues. The seller can fake the computation to return bogus result is one thing. But even if there is no malicious intent, the resulting computation is still ended up bogus result, the malfunction of the hardware. Commodity hardware isn't that reliable and there are so many commodity GPUs in the wild that looks like working but return incorrect result.
- alkonaut 9y agoall of these issues can be worked around by random validation, sumbitting all/some blocks to multiple people (redundant calculation). This is no different from existing efforts such as SETI@Homr and Folding@Home. The difference being the compensation which increases the incentive for fraud. The key to the trust issue is not trusting the remote computer, because you can’t. You have to validate somehow.
- ezoe 9y agoThat solution requires that all computation is deterministic and the result of computation can be easily comparable. I think not all the computation is that simple.
- alkonaut 9y agoIt only assumes that including incorrect results isn’t catastrophic for the end result, and that using mostly correct results will give a mostly correct outcome. If that isn’t the case I’d compute every block 2..n times instead.
- mathgeek 9y agoThis is an excellent point. You're also dealing with a market where demand is so high that GPU's running under constant high load are then resold for more than the MSRP of new ones.
- rb2k_ 9y agoI wonder when people will start just renting a 99 Euro server with a GTX 1080 from https://www.hetzner.de/dedicated-rootserver/ex51-ssd-gpu https://www.hetzner.de/dedicated-rootserver/ex51-ssd-gpu and sell it for $367.2 / month on Vectordash
- PascLeRasc 9y ago"Airbnb but for GPUs" is a buzzphrase I never could have imagined being real.