4 ms·

GPU pricing of alternative clouds (lowest price to highest): [A100 PCI] Lambda Labs: $1.10/hr TensorDock: $2.06/hr Coreweave: $2.46/hr Paperspace: $3.09/hr

GPU pricing of alternative clouds (lowest price to highest): [A100 PCI] Lambda Labs: $1.10/hr TensorDock: $2.06/hr Coreweave: $2.46/hr Paperspace: $3.09/hr [A100 SXM] Lambda Labs: $1.25/hr TensorDock: $2.06/hr Coreweave: N/A (I think PCI only) Paperspace: N/A (I think PCI only) [A40] TensorDock: $1.28/hr Coreweave: $1.68/hr Paperspace: N/A Lambda Labs: N/A [A6000] Lambda Labs: $0.80/hr Tensordock: $1.28/hr Paperspace: $1.89/hr Coreweave: $1.68/hr [V100 SXM4] Lambda Labs: $0.55/hr TensorDock: $0.80/hr Coreweave: $1.00/hr Paperspace: $2.30/hr [A5000] TensorDock: $0.77/hr Coreweave: $1.01/hr Paperspace: $1.38/hr Lambda Labs: N/A Jonathan thanks for the post. A question: it sounds like TensorDock partners with 3rd-parties who bought these servers and TensorDock doesn't actually own any of the servers you rent out. If that's the case, how do you ensure security? If not, please ignore. [References] https://www.paperspace.com/pricing https://www.paperspace.com/pricing https://lambdalabs.com/service/gpu-cloud#pricing https://lambdalabs.com/service/gpu-cloud#pricing https://coreweave.com/pricing https://coreweave.com/pricing https://www.tensordock.com/product-core https://www.tensordock.com/product-core

jonathanlei 4y ago
Thanks for the comment! Quick answer: it's complicated. Long answer: We own a substantial amount of compute ourselves, as far as Singapore where we have fully-owned hardware at Equinix SG1. We started with our own crypto mining operation just outside of Boston, but as wholesale consumers approached us in 2020 due to pandemic surges, we added business internet and power backups. Suddenly we were operating a rudimentary "office data center." Two large reseller sites sell on our fully-owned hardware. Boston's electric costs are very high ($0.23/kWh), so we're gradually moving more hardware to tier 3/4 data centers that are cheaper on a per-unit basis. But, we partner with 3rd parties too (4 large scale 1000+ GPU operators each to be exact) to resell their compute. This is also how we'll enter Europe... we're working closely with an existing supplier that colocates servers at Hydro66 in Sweeden and another at Scaleway Paris. We provide the software, they provide the hardware, and we pay them a special rate based on the volume we're doing. Partnering with others is the only way we can handle large scale without insanely high capex costs (that being said, we do get preferential pricing as an NVIDIA Inception Program member, which we take advantage of for our own fully-owned hardware). We have a doc in it here: https://docs.tensordock.com/infrastructure/reservable-instances#prospective-suppliers https://docs.tensordock.com/infrastructure/reservable-instan... We're also working on a marketplace (client site: https://www.tensordock.com/product-marketplace https://www.tensordock.com/product-marketplace, host site: https://www.tensordock.com/host https://www.tensordock.com/host). We expect a beta version to be up and running in the next ~2 weeks. With this, we'll have a script that hosts will use to install a small version of OpenStack. Then, they set prices, and customers can deploy directly on that hardware. By aggregating all these hosts together on the same marketplace, we hope we can slash the price of compute. So far, owning our own hardware has allowed us to negotiate better rates and enter markets where previous services don't exist (namely, Singapore, where we sell subscription servers with 1070 GeForce cards for $150/month — unheard of pricing for an APAC city). Eventually, we hope there'll be suppliers in every city selling on our marketplace or core cloud product so that we can really become the #1 place for ML startups to provision compute. In a way, we want to be the Amazon of cloud computing. Amazon, in a way, created a global marketplace. Yes, they sell their own products, but they also sell others' products. By doing so, you know that you're getting a good deal on whatever you buy. We want to end up being the same thing for compute, but that's still a few years off :) TL;DR - we own a lot of hardware, and we resell a lot of hardware. But in the future, we want to focus on the reselling aspect to truly be able to nail the user experience and handle demand surges while maintaining low costs.
- porker 4y ago
  And data security? Can you answer that part of the question? Most of my ML training is done using personal data or sensitive documents, and I have not found a cheap provider yet that I can use.
  - wintermute9124 4y ago
    Yes. Curious on this too. Lambda/Paperspace/Coreweave do own their own servers (Lambda being the cheapest). That alleviates some security concerns. It all depends on your security requirements though.
    - jonathanlei 4y ago
      Whoops, apologies for missing this! For our core cloud product, we only partner with established providers. Large-scale compute wholesalers with $5m+ of compute each in secure data centers. These companies' entire businesses are built on selling secure compute to customers like us and other medium/large businesses. Basically, this isn't some random dedicated server host off of LowEndTalk :) We have data protection agreements with all of them, and we can also do bare metal machines on request so that you have full control over your physical machine.
freediver 4y ago
A100 is available for $1.03 from GCP and V100 is available at $.27 per hour from Alibaba according to CloudOptimizer [1] [1] https://cloudoptimizer.io https://cloudoptimizer.io
- wintermute9124 4y ago
  I have never seen this tool. Thanks for sharing! The Alibaba price you cited is for interruptible/spot. For V100 on-demand (uninterruptible) Oracle is least expensive from that list at $1.275/hr.
  - freediver 4y ago
    Thanks, I created this tool to exactly find cheapest GPUs then expanded to everything else. Interruptible is sufficient for model training which is what these high end GPUs are typically used for (and you want cheapest possible)
    - wintermute9124 4y ago
      Didn't realize you made this tool. It's super useful. Some unsolicited feedback, if you're still actively developing: - You should consider some of the lesser known cloud providers (e.g. Coreweave/Lambda Labs/TensorDock). - Add information about whether the servers support NVLink/NVswitch. For example, A100s come in 3 flavors: PCIe without NVLink bridges, PCIe with NVLink bridges, and SXM with NVlink/NVSwitch fabric.
      - freediver 4y ago
        There are hundreds of smaller providers, each having a different API , if having it at all, so this is not possible for a one man operation. CloudOptimizer is already the largest cloud comparison tool on the web (12 cloud providers listed)
    - jonathanlei 4y ago
      Wow, cool! Yes - interruptible can be very cheap... I'll add it to our backlog so we do that instead of idly mining I was wondering, do you happen to have an API for listing servers? We're launching a marketplace later in August (https://www.tensordock.com/product-marketplace https://www.tensordock.com/product-marketplace), and we expect pricing to be really really cheap. Like #1 in industry cheap while retaining. Interruptible, if we add that, would probably be even less than those prices listed. It'd be really cool if we could auto-update availabilities of GPU servers through an API so that we can list our servers on your tool as well :)
Havoc 4y ago
>TensorDock doesn't actually own any of the server Vaguely recall reading they're using a mixed model, but OP will hopefully confirm
antupis 4y ago
Somebody should build "cheap" service to EU. Quick googling and Paperspace is only one with datacenter at EU. Those GDPR requirements are now rather nasty so you usually cannot move customer/third-party data outside EU.
- jonathanlei 4y ago
  We will consider! We are working with two potential suppliers in the area... expect more news by the end of the year if those deals go through! :)
fxtentacle 4y ago
A100 at €0.44 = about $0.5 https://puzl.ee/cloud-kubernetes/configurator https://puzl.ee/cloud-kubernetes/configurator I'm quite happy with them, but storage is limited to 1GBit/s and you should know basic kubectl to use it fully.
- mdda 4y ago
  Isn't that for only one quarter of the A100?
  - artemisart 4y ago
    Yes... a full A100 (+ reasonable CPU etc) looks closer to 2€.

jonathanlei 4y ago

Thanks for the comment! Quick answer: it's complicated. Long answer: We own a substantial amount of compute ourselves, as far as Singapore where we have fully-owned hardware at Equinix SG1. We started with our own crypto mining operation just outside of Boston, but as wholesale consumers approached us in 2020 due to pandemic surges, we added business internet and power backups. Suddenly we were operating a rudimentary "office data center." Two large reseller sites sell on our fully-owned hardware. Boston's electric costs are very high ($0.23/kWh), so we're gradually moving more hardware to tier 3/4 data centers that are cheaper on a per-unit basis. But, we partner with 3rd parties too (4 large scale 1000+ GPU operators each to be exact) to resell their compute. This is also how we'll enter Europe... we're working closely with an existing supplier that colocates servers at Hydro66 in Sweeden and another at Scaleway Paris. We provide the software, they provide the hardware, and we pay them a special rate based on the volume we're doing. Partnering with others is the only way we can handle large scale without insanely high capex costs (that being said, we do get preferential pricing as an NVIDIA Inception Program member, which we take advantage of for our own fully-owned hardware). We have a doc in it here: https://docs.tensordock.com/infrastructure/reservable-instances#prospective-suppliers https://docs.tensordock.com/infrastructure/reservable-instan... We're also working on a marketplace (client site: https://www.tensordock.com/product-marketplace https://www.tensordock.com/product-marketplace, host site: https://www.tensordock.com/host https://www.tensordock.com/host). We expect a beta version to be up and running in the next ~2 weeks. With this, we'll have a script that hosts will use to install a small version of OpenStack. Then, they set prices, and customers can deploy directly on that hardware. By aggregating all these hosts together on the same marketplace, we hope we can slash the price of compute. So far, owning our own hardware has allowed us to negotiate better rates and enter markets where previous services don't exist (namely, Singapore, where we sell subscription servers with 1070 GeForce cards for $150/month — unheard of pricing for an APAC city). Eventually, we hope there'll be suppliers in every city selling on our marketplace or core cloud product so that we can really become the #1 place for ML startups to provision compute. In a way, we want to be the Amazon of cloud computing. Amazon, in a way, created a global marketplace. Yes, they sell their own products, but they also sell others' products. By doing so, you know that you're getting a good deal on whatever you buy. We want to end up being the same thing for compute, but that's still a few years off :) TL;DR - we own a lot of hardware, and we resell a lot of hardware. But in the future, we want to focus on the reselling aspect to truly be able to nail the user experience and handle demand surges while maintaining low costs.

porker 4y ago
And data security? Can you answer that part of the question? Most of my ML training is done using personal data or sensitive documents, and I have not found a cheap provider yet that I can use.
- wintermute9124 4y ago
  Yes. Curious on this too. Lambda/Paperspace/Coreweave do own their own servers (Lambda being the cheapest). That alleviates some security concerns. It all depends on your security requirements though.
  - jonathanlei 4y ago
    Whoops, apologies for missing this! For our core cloud product, we only partner with established providers. Large-scale compute wholesalers with $5m+ of compute each in secure data centers. These companies' entire businesses are built on selling secure compute to customers like us and other medium/large businesses. Basically, this isn't some random dedicated server host off of LowEndTalk :) We have data protection agreements with all of them, and we can also do bare metal machines on request so that you have full control over your physical machine.

freediver 4y ago

A100 is available for $1.03 from GCP and V100 is available at $.27 per hour from Alibaba according to CloudOptimizer [1] [1] https://cloudoptimizer.io https://cloudoptimizer.io

wintermute9124 4y ago
I have never seen this tool. Thanks for sharing! The Alibaba price you cited is for interruptible/spot. For V100 on-demand (uninterruptible) Oracle is least expensive from that list at $1.275/hr.
- freediver 4y ago
  Thanks, I created this tool to exactly find cheapest GPUs then expanded to everything else. Interruptible is sufficient for model training which is what these high end GPUs are typically used for (and you want cheapest possible)
  - wintermute9124 4y ago
    Didn't realize you made this tool. It's super useful. Some unsolicited feedback, if you're still actively developing: - You should consider some of the lesser known cloud providers (e.g. Coreweave/Lambda Labs/TensorDock). - Add information about whether the servers support NVLink/NVswitch. For example, A100s come in 3 flavors: PCIe without NVLink bridges, PCIe with NVLink bridges, and SXM with NVlink/NVSwitch fabric.
    - freediver 4y ago
      There are hundreds of smaller providers, each having a different API , if having it at all, so this is not possible for a one man operation. CloudOptimizer is already the largest cloud comparison tool on the web (12 cloud providers listed)
  - jonathanlei 4y ago
    Wow, cool! Yes - interruptible can be very cheap... I'll add it to our backlog so we do that instead of idly mining I was wondering, do you happen to have an API for listing servers? We're launching a marketplace later in August (https://www.tensordock.com/product-marketplace https://www.tensordock.com/product-marketplace), and we expect pricing to be really really cheap. Like #1 in industry cheap while retaining. Interruptible, if we add that, would probably be even less than those prices listed. It'd be really cool if we could auto-update availabilities of GPU servers through an API so that we can list our servers on your tool as well :)

Havoc 4y ago

>TensorDock doesn't actually own any of the server Vaguely recall reading they're using a mixed model, but OP will hopefully confirm

antupis 4y ago

Somebody should build "cheap" service to EU. Quick googling and Paperspace is only one with datacenter at EU. Those GDPR requirements are now rather nasty so you usually cannot move customer/third-party data outside EU.

jonathanlei 4y ago
We will consider! We are working with two potential suppliers in the area... expect more news by the end of the year if those deals go through! :)

fxtentacle 4y ago

A100 at €0.44 = about $0.5 https://puzl.ee/cloud-kubernetes/configurator https://puzl.ee/cloud-kubernetes/configurator I'm quite happy with them, but storage is limited to 1GBit/s and you should know basic kubectl to use it fully.

mdda 4y ago
Isn't that for only one quarter of the A100?
- artemisart 4y ago
  Yes... a full A100 (+ reasonable CPU etc) looks closer to 2€.