16 ms·
Launch HN: Snark AI (YC S18) – Distributed Low-Cost GPUs for Deep Learning
Hi HN,
We are Sergiy, Davit and Jason, founders of Snark AI (https://snark.ai https://snark.ai). We provide low-cost GPUs for Deep Learning training and deployment on semi-decentralized servers.
We started Snark AI during our PhD programs at Princeton University. As deep learning researchers we always experienced lack of GPU resources. Renting out GPUs on the cloud didn't fit in our budget, and purchasing GPU cards was difficult -- at that time, so many GPUs were being taken away by the crypto-miners. Then we found out that GPU mining profits lag far behind public cloud GPU prices.
On top of that, we figured out that there's a way to run Neural Network inference and crypto-mining simultaneously without hurting mining hash rate. This observation is a little counterintuitive, but it turns out that anti-asic hashing algorithms are designed to be extremely memory intensive, which leaves a good chunk of the CUDA cores idle. We can utilize the leftover compute power to run Neural Network inference extremely cost efficiently, which could be a life savior for large-scale inference tasks. http://snark.ai/blog http://snark.ai/blog
At the same time, we provide low cost raw hardware access for Neural Network training. We aim to be up to 10 times cheaper than on-demand instances on public cloud, undercutting preempteble/spot instance by up to 2x. When the GPU is idle our algorithms efficiently switch to mining to reduce costs. Try it out at https://lab.snark.ai https://lab.snark.ai, with 10 hours of free GPU time. We made it very simple to access the hardware through a single command line after `pip3 install snark`. More information on usage here https://github.com/snarkai/snark-doc https://github.com/snarkai/snark-doc. We are also working on creating a hub for NNs, similar to docker hub. It is still work in progress but you can take a look at couple examples at https://hub.snark.ai/explore https://hub.snark.ai/explore.
We would love to get your feedback, to understand how was the experience for training Deep Networks through our platform and then deploying.
- minimaxir 8y agoSo what is the pricing? The website has a login-wall to access the pricing.
- Fede_V 8y agoI'm sure there is a practical reason why companies do this, but it drives me crazy and makes me want to ignore what could potentially be a really cool idea. Please make the pricing transparent and viewable without a registration.
- davidbuniat 8y agoyeah you are right @Fede_V, sorry about this, just made it public https://lab.snark.ai/pricing https://lab.snark.ai/pricing
- deleted 8y ago[deleted]
- paulschraven 8y agoNVIDIA GTX1080 - 0.25 / hour NVIDIA GTX1070 - 0.2 / hour NVIDIA P106-100 - 0.095 / hour (All values in $ I assume)
- 1996 8y agoI hope not, or it is an epic fail 0.25x24x30.5=183 For 100 Eur, or about $120, you can get a 1080 inside A DEDICATED SERVER (!!) at Hetzner: https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?country=us https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu?co... I guess I have a business idea then: charge 0.5/h to rent the 1080, pocket the $63 difference per month and call it profit, undercutting Snark by 33% without even doing any crypto mining or anything on the side. Or, just call it step 1 for mega profits! Step 2: resell the CPU computing power, step 3: resell the SSD storage space, step 4: resell the bandwidth, etc. (not sure you can resell the unused RAM, but that's another "innovative business" waiting to happen!!) Compared to renting servers and properly configuring them (anycast, geoip, etc) I often fail to see the value that "distributed" or "cloud" offer provide besides fast scalability. It is nice to be able to put 4x more GPUs online in a few hours instead of a day, but I am not sure it commands a 33% premium except in very specific marginal cases.
- kowdermeister 8y agowow, thanks, last time I checked (couple of years ago) they had close to none GPU offerings
- mmt 8y agoWasn't that true, in general, for cloud and VPS providers? 2 years is a Moore's Law doubling, which I've found tends to mean hardware offerings will be different. They're not necessarily dramatically different, if there's no new/unmet market demand, but this was a noteworthy enough one that I was (and still am) touting it as an advantage of own-hardware over cloud infrastructure.
- davidbuniat 8y agoYeah, We present prices on hourly basis and you can spin up pods in matter of seconds. Actually we will try to reach them and see if partnership can make our services even cheaper, thanks for pointing this out.
- davidbuniat 8y agoThanks for raising this, we will put the prices public soon.
- davidbuniat 8y ago[UPDATE] prices are public https://lab.snark.ai/pricing https://lab.snark.ai/pricing
- technologia 8y agoI don't know if I overlooked it, but it doesn't seem possible to delete my account if I so choose.
- davidbuniat 8y agoJust shoot us an email at support@snark.ai, we will handle it :)
- technologia 8y agoI just realized how my commment sounded, sorry about that; I only wanted to make a note of that while I evaluated your platform for broader use.
- davidbuniat 8y agoah no worries, thanks for helping us out! sure will add automatic deletion support.
- avarun 8y agoI'm not personally super invested in this, but you might want to check to see if that violates GDPR.
- paulschraven 8y agoAre you planning to add cards manufactured for ML, such as the NVidia Titan V (110 teraflops)?
- harias 8y agoVectordash has some : https://vectordash.com/pricing/ https://vectordash.com/pricing/
- davidbuniat 8y agoyes, we are currently focusing on large-scale compute, but we are planning to add more variety for hardware.
- scottlegrand2 8y agoHope you're not planning on putting any of those fancy Titan Vs in a Datacenter! Rumor has it that ever since Baidu installed 100,000 GeForce GPUs in a Datacenter, they've made that against their EULA (unless you're mining cryptocurrencies which is apparently A-OK). http://www.datacenterdynamics.com/content-tracks/servers-storage/nvidia-updates-geforce-eula-to-prohibit-data-center-use/99525.fullarticle http://www.datacenterdynamics.com/content-tracks/servers-sto...
- davidbuniat 8y agoGood catch! We don't own hardware ourselves, but we are considering EULA with our partners.
- deleted 8y ago[deleted]
- harias 8y agoAre you the same guy : https://www.reddit.com/r/gpumining/comments/86ofw2/rent_out_your_gpu_compute_to_ai_researchers_and/ https://www.reddit.com/r/gpumining/comments/86ofw2/rent_out_...
- 1996 8y agoThen it is a copycat of nicehash. I wonder how they plan to undercut nicehash?
- lozaning 8y agoI understand them to be totally different services. How would someone with ML training data on hand use nicehash to get any work done? Is snark setup to do mining? As I see it they serve totally distinct and separate markets.
- merkleforest 8y agoIt's interesting that you're drawing the comparison with NiceHash. Snark AI will be a marketplace like NiceHash but targets mostly deep learning training and inference on the user end.
- davidbuniat 8y agoNo, we only work with large scale hardware providers that guarantees security and reliability. We specialize in Deep Learning services including training and deployment.
- whoisjuan 8y ago"3-5 times cheaper than public cloud"... That's very vague. What's the actual minimum price per hour?
- davidbuniat 8y agoThanks for asking, it is cheaper than preemptible instance of K80 0.135$/h on Google with an equivalent GPU of P106 0.095$/h on Snark.
- ipsum2 8y agoThat's definitely not 3-5 times cheaper.
- sgrove 8y agoI think snark isn't pre-emptible, they're just giving the best possible price on GCE (steelman argument, which is impressive), but it is a bit confusing. From https://cloud.google.com/compute/pricing https://cloud.google.com/compute/pricing it looks like the non-pre-emptible K80 pricing is $0.45 USD per GPU. Can someone from Snark correct me if I'm wrong?
- davidbuniat 8y agoYep, totally agree!
- TekMol 8y agoFunny, I wanted to take a look at snark.ai to see if it can be used to run WebGL inside a browser. For some reason, the snark.ai homepage brings my laptop to its knees though. Do I need a GPU cluster to see it? Anyhow: Can I use snark.ai to run WebGL in a browser?
- davidbuniat 8y agoHaha, jokingly we use GPUs on our landing page to rent out to others. :D On a side note, animations are slightly tough, need to optimize them. Regarding WebGL, actually that is an interesting point, would like to know more about the use case.
- TekMol 8y agoWebGL: I mean can I install a desktop environment on your machines, start a browser in it and then WebGL will be fast because it has access to a fast GPU?
- davidbuniat 8y ago@TekMol, we can give a try together to see if the streaming speed will be enough for running your WebGL application smoothly.
- wsu718 8y agoReally interesting. Is storage persistent? Will files get deleted when I stop a machine?
- davidbuniat 8y agoThanks for asking, thats a good point! Compared to spot/preemptible, your files will be persistent. You can stop your running pod, add more GPUs and then continue training your model.
- mbajkowski 8y agoReading through your blog entry: "When the data to be processed is sensitive, Snark Infer will dispatch the task only to Privacy Shield Verified GPU providers." - can you shed some light as to a) what a privacy shield verified GPU provider is? b) how data sensitivity is specified or determined? (I did not see either mentioned in the docs yet)
- davidbuniat 8y agoGreat you asked this. We try to say that for privacy sensitive data processing (e.g. face recognition), we are eager to work closely with clients and hardware providers to ensure security and compliance to regulations such as Privacy Shield, GDPR, etc. Do you have a such use case, would be great to get in touch with.
- karimf 8y agoIf you guys are searching for potential early customer, maybe you can hang around on Fast.ai course forum[0]. There are a lot of people who are just trying DL for the first time, like me, and are looking for the cheapest way possible to get started experimenting. One of your selling point you said is the cheaper price, so maybe it would work out well. http://forums.fast.ai/ http://forums.fast.ai/
- davidbuniat 8y agoGood suggestion! Our cheapest instances are actually perfect for learning ML/DL. We are also working on adding Jupyter support so it will be even easier. What do you think?
- karimf 8y agoYup. Out of the box support for Jupyter would be a great addition, given the environment is the same as what the user requires (in case of fast.ai course the environment should be similar to the course's required environment). You could also try to reach out to Jeremy or Rachel, see if they can help with anything. They sometimes hang around here on HN too.
- corporateslaver 8y agoPlease stop marketing fast.ai on hacker news
- cookingrobot 8y agoWhy, it’s really good? I was going to make the same suggestion, but to compete with Paperspace they should offer an image that comes with all the course dependencies preinstalled.
- merkleforest 8y agoThanks for the suggestion! Snark AI will offer a pod type with all fast.ai course dependencies installed and easy jupyter notebook access.
- hartator 8y agoWe might be an early customer at SerpApi. [1] In your pricing, you say you are selling, for example, P106 at 0.095$/h, but in your explanation, you are saying you are using idle cycle to mine cryptos (or the reverse, idle cycle to process ML tasks). When I rent a P106, do I have full access to the cores or just partial? [1] https://serpapi.com https://serpapi.com
- davidbuniat 8y agoGreat! Once you rent a P106 you have full access to the GPU, and we don't run mining or ML tasks. If you want to deploy large-scale computation and significantly reduce your costs, we can help by running mining at the same time under your consent. This only applies to Deep Learning inference.
- wei2012 8y agoThe Graphic Card's price will reduce very soon due to bitcoin pricing drop.
- davidbuniat 8y agoThat's what we count on as well.
- hwoolery 8y agoSerious question (as opposed to a snarky one): what happens when/if the crypto currency market takes a large downturn? If your pricing is based off using their cycles, Snark.ai would have to raise prices, right?
- davidbuniat 8y agoIn case of crypto downturn renting GPUs will be even cheaper. Actually, it will be a bigger win for our users and also for us.
- hwoolery 8y agoAssuming the miners still want to rent out the hardware and not sell it :) Even then, you'd have an instance of two parties both taking slices of the profit, which is hard to compete in a practical manner against the Amazon/Google
- merkleforest 8y agoYeah exactly. Hopefully AI computation market will grow big in time so those crypto-miners can pivot into more profitable business of GPU cloud for AI instead of selling the hardware in crypto downfall. In the long term, Snark AI wants to create the marketplace where all the qualified GPU providers can bid for the best market price which will benefit everyone.
- thisisit 8y ago> we figured out that there's a way to run Neural Network inference and crypto-mining simultaneously without hurting mining hash rate I understand that you figured out the technical side of things but curious about the human nature side of things. Let's say currently you can have both NN and mining running. From a cryptocurrency price point of view two things can happen: a. Prices go up - Wont the incentive change towards mining? How do you guys plan to handle such situations? b. Prices go down - In this case, the intuitive thing to happen is that GPU pricing to get cheaper. But then, will you guys passing on such benefits to your customers? Because the whole point is to be able to make a steady income from the GPUs, mining or not.
- davidbuniat 8y agoLets agree that if the mining price goes up, it will be always cheaper than public cloud. Otherwise, you would have run mining on AWS/GCP and be profitable. In this scenario, we are still able to offer cheaper price to the customers and higher price to the miners even by just binary switching. Assuming the gap is not negligible. If the price goes down, customers will be able to set lower price. As long as GPU holders profit margin is high enough given electricity and maintenance costs, they will do the compute. If the marketplace matures, pricing of mining, deep learning, rendering and other tasks will be driven by the market. At Snark AI we are working on towards creating this marketplace that will provide optimal benefits to all parties.
- thisisit 8y ago> Lets agree that if the mining price goes up, it will be always cheaper than public cloud I am confused. Let's say your price is $x while public cloud is $x+y. When cryptocurrency price rise the incentive is to increase your price to $x+y, if that is the profit point. In which case you will no longer be cheaper than public cloud. Any differences will be negligible. So, I am not sure you can claim to be always cheaper than public cloud.
- davidbuniat 8y agoFollowing your notation, consider our new price will be $x+y', where y' is the mining price difference. You are right my claim that y'<y is slightly weak (was based on "gap is not negligible" assumption, see below). "gap is not negligible" - means if y' gets near to y, then y will get even higher and there will be always a market gap, which I think you disagree with. Based on your suggestion, on extreme scenario I would soften my claim to y'=<y, without us making profit. :)