4 ms·

> If you have to sell the hardware and support people buying it, that is a world of trouble. What is the difference between this and having to sell the cloud a

> If you have to sell the hardware and support people buying it, that is a world of trouble. What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription?

themoonisachees 3y ago
A lot. It's the same reason amazon doesn't sell servers and instead gives you access to a single instance that everyone pretends is the same but in reality is massively transient.
- latchkey 3y ago
  We give full bare metal access. If you just want one GPU, we give you a VM with PCIe pass through. If you take a whole box, we can give BMC and can give you access to the PDU itself, to hard reboot things remotely. It is as if you own the whole machine yourself.
latchkey 3y ago
Good question. Not much. If anything, what I'm doing is even harder because I will have multiple sources for the hardware. I have to deal with all of the hardware and data center issues, as well as the customers who rely on us to provide them access. Good thing that I'm a glutton for punishment.
alephnerd 3y ago
> What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription Margins. Pricing for cloud compute is much higher and servicing and management for the provider is much cheaper. If I sold hardware directly, then I'm often on the hook for support contracts which can get pricy with hardware and distract from shipping future facing product features, as customers who purchase directly have longer upgrade windows due to logistical overhead.
- latchkey 3y ago
  The pricing isn't necessarily much higher. The pricing is set up to amortize the cost of running things over a period of time. If you're only going to use it for a short time, you pay more than someone who commits to a 3 year contract. It also isn't just the hardware capex, it is everything involved under the covers. Market pricing also factors into that as well. This is something I've struggled with myself quite a bit. I know all of my costs and what I'd like to charge, but because my offering is so brand new, until my competitors announced their pricing, I wasn't sure what the market would tolerate. Selling hardware directly is hard for exactly what you state though. Service contracts are a pain in the butt. All of this latest AI hardware has high rates of failures too. Up until recently with AMD coming to market with a great offering, the only thing people want any sort of quantity on are nvidia products. Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable.
  - alephnerd 3y ago
    > it is everything involved under the covers Yep! You explained it better than me! > Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable Yep! And if would have bogged them down by slowing down R&D cycles and even fulfilling orders as they obviously are not placing orders the same size as Nvidia or AMD. I wonder what this portends for SambaNova and other similar vendors as well.
    - latchkey 3y ago
      > I wonder what this portends for SambaNova and other similar vendors as well. Time will tell. This is definitely an interesting development.
    - htrp 3y ago
      >I wonder what this portends for SambaNova and other similar vendors as well. They're focused on services based full stack deployments afaik. They come in with a rack and sell you on models as well.
ethbr1 3y ago
> What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription Knowledge/training. If you're shipping a brand new hardware arch, exposed as raw hardware, then you're on the hook for training everyone in the world and fixing all their weird edge case uses. I.e. are you willing to invest in Intel/AMD/Nvidia-scale QA and support? If you're exposing a PaaS (or even IaaS), then you have some levers you can tweak / mask behind the scenes, so only your team need be experts at low-level operations. For a fast-paced company, the latter model makes a lot more sense, at least until hardware+software stabilizes.
- latchkey 3y ago
  > I.e. are you willing to invest in Intel/AMD/Nvidia-scale QA and support? At least in our experience, the first line of support is from the chassis vendors. You don't go to a store and buy MI300x. You buy them from someone like SMCI/Dell, who provides the support. Of course, behind the scenes, they might be talking to AMD. Even those chassis companies often have other providers of their gear (like Exxact) as another line of defense as well. In the case of Groq, it would have been death by 1000 cuts to have to support end users directly, especially if they are selling small quantities. It is much easier to just build data centers full of gear, maintain it yourself and then just rent the time on the hardware.

themoonisachees 3y ago

A lot. It's the same reason amazon doesn't sell servers and instead gives you access to a single instance that everyone pretends is the same but in reality is massively transient.

latchkey 3y ago
We give full bare metal access. If you just want one GPU, we give you a VM with PCIe pass through. If you take a whole box, we can give BMC and can give you access to the PDU itself, to hard reboot things remotely. It is as if you own the whole machine yourself.

latchkey 3y ago

Good question. Not much. If anything, what I'm doing is even harder because I will have multiple sources for the hardware. I have to deal with all of the hardware and data center issues, as well as the customers who rely on us to provide them access. Good thing that I'm a glutton for punishment.

alephnerd 3y ago

> What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription Margins. Pricing for cloud compute is much higher and servicing and management for the provider is much cheaper. If I sold hardware directly, then I'm often on the hook for support contracts which can get pricy with hardware and distract from shipping future facing product features, as customers who purchase directly have longer upgrade windows due to logistical overhead.

latchkey 3y ago
The pricing isn't necessarily much higher. The pricing is set up to amortize the cost of running things over a period of time. If you're only going to use it for a short time, you pay more than someone who commits to a 3 year contract. It also isn't just the hardware capex, it is everything involved under the covers. Market pricing also factors into that as well. This is something I've struggled with myself quite a bit. I know all of my costs and what I'd like to charge, but because my offering is so brand new, until my competitors announced their pricing, I wasn't sure what the market would tolerate. Selling hardware directly is hard for exactly what you state though. Service contracts are a pain in the butt. All of this latest AI hardware has high rates of failures too. Up until recently with AMD coming to market with a great offering, the only thing people want any sort of quantity on are nvidia products. Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable.
- alephnerd 3y ago
  > it is everything involved under the covers Yep! You explained it better than me! > Groq probably realized that people buying 1-2 cards at a time, wasn't going to be profitable Yep! And if would have bogged them down by slowing down R&D cycles and even fulfilling orders as they obviously are not placing orders the same size as Nvidia or AMD. I wonder what this portends for SambaNova and other similar vendors as well.
  - latchkey 3y ago
    > I wonder what this portends for SambaNova and other similar vendors as well. Time will tell. This is definitely an interesting development.
  - htrp 3y ago
    >I wonder what this portends for SambaNova and other similar vendors as well. They're focused on services based full stack deployments afaik. They come in with a rack and sell you on models as well.

ethbr1 3y ago

> What is the difference between this and having to sell the cloud access and supporting the people who buy a subscription Knowledge/training. If you're shipping a brand new hardware arch, exposed as raw hardware, then you're on the hook for training everyone in the world and fixing all their weird edge case uses. I.e. are you willing to invest in Intel/AMD/Nvidia-scale QA and support? If you're exposing a PaaS (or even IaaS), then you have some levers you can tweak / mask behind the scenes, so only your team need be experts at low-level operations. For a fast-paced company, the latter model makes a lot more sense, at least until hardware+software stabilizes.

latchkey 3y ago
> I.e. are you willing to invest in Intel/AMD/Nvidia-scale QA and support? At least in our experience, the first line of support is from the chassis vendors. You don't go to a store and buy MI300x. You buy them from someone like SMCI/Dell, who provides the support. Of course, behind the scenes, they might be talking to AMD. Even those chassis companies often have other providers of their gear (like Exxact) as another line of defense as well. In the case of Groq, it would have been death by 1000 cuts to have to support end users directly, especially if they are selling small quantities. It is much easier to just build data centers full of gear, maintain it yourself and then just rent the time on the hardware.