3 ms·

Since we're on the topic of your business: I am training a decent amount of neural nets nowadays (mostly, around the new-gen robotics policies) and use vast.ai

by krasin 2y ago

Since we're on the topic of your business: I am training a decent amount of neural nets nowadays (mostly, around the new-gen robotics policies) and use vast.ai instances with 8x RTX 4090 cards. I've been interested to give 8x MI300x a try, as they are supposed to be cheaper per FLOPs, but it looks like your service does not provide on-demand pay-per-second instances. Any plans to change that?

latchkey 2y ago
I would love nothing more than to be able to enable on-demand GPUs, but unfortunately this is a limitation from AMD right now. We can't do PCIe pass through to a virtual machine, it just doesn't work. This is why our minimum is 8 right now. If you look at all of our competitors, they have the same issue. Even Azure "VM", is 8 at a time, but they are all sold out due to high demand. It kind of makes sense since their history is only supporting the high end GPUs in their HPC solutions, where they don't use VM's. They've committed to us directly that they will fix this issue. I updated our pricing page to note this.
- trogdor 2y ago
  >I updated our pricing page to note this Nice job on your website! Sincerely, Someone who recently gave you a hard time for not having info on your website.
  - latchkey 2y ago
    Thank you! That is awesome feedback. Your hard time helped motivate me to work on it as more of a priority.
- krasin 2y ago
  > I would love nothing more than to be able to enable on-demand GPUs, but unfortunately this is a limitation from AMD right now. We can't do PCIe pass through to a virtual machine, it just doesn't work. This is why our minimum is 8 right now. If you look at all of our competitors, they have the same issue. Even Azure "VM", is 8 at a time, but they are all sold out due to high demand. Thank you for the response! Renting 8 GPUs at once is fine and desired; it's not the issue. The issue is that right now one has to commit to at least 1 week of use; my use patterns are bursty and it does not map well to the current proposition.
  - latchkey 2y ago
    This is very good feedback. We are just getting off the ground and on/off-boarding is still a bit of work for us. Right now, we are trying to attract people who are a mix of wanting to kick the tires on a new product, as well as take compute for longer term. I did mention in the pricing section that we can store your data locally, as part of the advertised pricing. This is our effort to recognize your use case. I also understand that you want to optimize and don't want to pay for something that you're not using. We will eventually get to that point, but honestly just not there yet. Think also from our end, we have these GPUs and if you're not using them... then who is? We've put out the capex/opex to make these available to you at any time, so the only way to be efficient on our side is to do a week long block right now. Regardless, if you want to reach out to me directly, please do so. Maybe there is a middle ground we can both work from. Happy to consider all options and getting in early with us will always have first mover advantages.
    - krasin 2y ago
      Thank you for being open; if my spend on vast.ai grows another 10X, I will consider reaching out. Right now, I am still a fairly small fish for your supercomputer.

by krasin 2y ago

latchkey 2y ago

I would love nothing more than to be able to enable on-demand GPUs, but unfortunately this is a limitation from AMD right now. We can't do PCIe pass through to a virtual machine, it just doesn't work. This is why our minimum is 8 right now. If you look at all of our competitors, they have the same issue. Even Azure "VM", is 8 at a time, but they are all sold out due to high demand. It kind of makes sense since their history is only supporting the high end GPUs in their HPC solutions, where they don't use VM's. They've committed to us directly that they will fix this issue. I updated our pricing page to note this.

trogdor 2y ago
>I updated our pricing page to note this Nice job on your website! Sincerely, Someone who recently gave you a hard time for not having info on your website.
- latchkey 2y ago
  Thank you! That is awesome feedback. Your hard time helped motivate me to work on it as more of a priority.
krasin 2y ago
> I would love nothing more than to be able to enable on-demand GPUs, but unfortunately this is a limitation from AMD right now. We can't do PCIe pass through to a virtual machine, it just doesn't work. This is why our minimum is 8 right now. If you look at all of our competitors, they have the same issue. Even Azure "VM", is 8 at a time, but they are all sold out due to high demand. Thank you for the response! Renting 8 GPUs at once is fine and desired; it's not the issue. The issue is that right now one has to commit to at least 1 week of use; my use patterns are bursty and it does not map well to the current proposition.
- latchkey 2y ago
  This is very good feedback. We are just getting off the ground and on/off-boarding is still a bit of work for us. Right now, we are trying to attract people who are a mix of wanting to kick the tires on a new product, as well as take compute for longer term. I did mention in the pricing section that we can store your data locally, as part of the advertised pricing. This is our effort to recognize your use case. I also understand that you want to optimize and don't want to pay for something that you're not using. We will eventually get to that point, but honestly just not there yet. Think also from our end, we have these GPUs and if you're not using them... then who is? We've put out the capex/opex to make these available to you at any time, so the only way to be efficient on our side is to do a week long block right now. Regardless, if you want to reach out to me directly, please do so. Maybe there is a middle ground we can both work from. Happy to consider all options and getting in early with us will always have first mover advantages.
  - krasin 2y ago
    Thank you for being open; if my spend on vast.ai grows another 10X, I will consider reaching out. Right now, I am still a fairly small fish for your supercomputer.