4 ms·

I'm still not sure why anyone would buy a dell server. Supposedly xAI buys from them, which probably accounts for this, but it is generally a much more sensible

I'm still not sure why anyone would buy a dell server. Supposedly xAI buys from them, which probably accounts for this, but it is generally a much more sensible choice to buy from companies that are lower-cost and less focused on selling you "enterprise support" (supermicro et al).

throwaway48476 1y ago
Probably iDrac. AI servers are mostly just white boxed nvidia reference designs. Not much markup.
- bluedino 1y ago
  HPE is finally moving away from their 'Cray' systems which are just whitebox AMI systems. You get none of their iLO etc even though they are 'HPE'
alephnerd 1y ago
> buy from companies that are lower-cost and less focused on selling you "enterprise support" (supermicro et al). From past experience, vendors like SuperMicro aren't much cheaper at scale because Dell can give 80% discounts on volume, and most Dell server sales are done via Channel with an MSP who will manage and administer the system. Ofc, this continues to reinforce my belief that the "AI boom" in it's current form (and once you remove the scooby doo monster mask) is basically a "Datacenter/Telecom Bubble" 2.0 like back in 2000-01. The multiples, messaging, vendors, and margins are almost the exact same. During the Dot-com Boom, Telecom and hardware companies were also expanding DC and telecom capacity massively (to surf and host a website on the Internet, you kinda need Internet and web hosting), and when the Dotbomb happened, the Telco bubble collapsed subsequently as well (remember WorldCom/MCI?) That's why the 2000s were horrible for anyone with a CS/CE/CSE/EE degree, because both software AND hardware industries collapsed. Imagine a world in the late 2020s where you cannot land a job as a Fullstack Engineer OR an ML Infra Engineer - that was the 2000s except with older stacks. That's also why I'd be optimistic if a bust happens - the overcapacity in compute that arose from the Telecom and Dot-Com Busts both helped usher the Cloud, SaaS, E-Commerce, and Social Media boom because the infra has been laid and became cost effective. It is also in this context that Paul G's "cockroach" and "ramen profitability" essay came to the fore.
- oblio 1y ago
  The thing is, the bust froze everything for 1 or 2 years, which sucked for regular IT folks plus this time there also a huge oversupply of IT graduates. I wouldn't be surprised if there are 10x more people working in IT then back then.
  - alephnerd 1y ago
    Mind you, I was just a kid when it happened so I'm basing it off of 2nd hand stories, but my understanding was anyone with a pulse (ie. A hs grad or a college dropout) could get hired into the industry at the time (eg. The KOTH joke that if Peggy was laid off from being a substitute teacher she'd learn to design software). At least ime in the 2010s and 2020s, while you had some aspect of that with bootcamp grads, it wasn't to the same degree. Or was it? Like I said, I was a kid when the busts happened so I have no frame of reference.
    - shermantanktop 1y ago
      I had a pulse and got into this industry in the 90s through the back door. It’s been a long run and I’ve seen busts come and go. It’s worked out really well for me, even while I’ve watched a surprising number of highly-credentialed new hires flop out over the years. I continue to appreciate people with oddball resumes.
      - tracker1 1y ago
        Similar here, no formal higher education starting in the early-mid 90's, just a lot of reading/learning and experimenting along the way... Even while working, I've spent 15-20 hours a week most of my career on learning activities. Somewhat hardware oriented and worked a few tech support/helpdesk and internal IT jobs in the early-mid 90's when I got more into programming as a side-step from design/art.
- firesteelrain 1y ago
  > That's why the 2000s were horrible for anyone with a CS/CE/CSE/EE degree, because both software AND hardware industries collapsed Odd, anecdotally I remember people saying that, but had no issue getting a job in 2004 as a fresh CS grad.
axus 1y ago
For the international hardware warranty transfer.
bluedino 1y ago
You want enterprise support. GPU quality is atrocious, you will have your Dell tech in there replacing GPUs and fans all the time.
- c0balt 1y ago
  Can second this, the amount of GPU failures we have with Lenovo systems on just <50 nodes is significantly higher than we expected. Having a Lenovo support person at least twice a month on premise at the middle of the bathtub curve is probably also costing them (and implicitly us) a good chunk of money.
- grubbs 1y ago
  Interesting. I work in higher ed and we have thousands of GPUs under my team. Rarely ever seen a failure. Mostly when we put consumer grade GPUs in servers (Nvidia doesn't like this). True server-grade GPUs never have any problems.
  - ecshafer 1y ago
    IS this for some kind of HPC cluster? What kind of utilization are you at? For an AI company these GPUs are going to be at near 100% utilization 24/7. These kinds of loads destroy hardware quick.
  - bluedino 1y ago
    Every site I've worked at has plenty of GPU failures. Not consumer grade either, H100/A100
jandrewrogers 1y ago
At scale, the savings of slightly cheaper hardware is often dwarfed by supply chain management costs, both direct and indirect. How much is it worth to your business to get the hardware deployed six weeks sooner and/or with a high probability of timely delivery? I imagine xAI wanted to move with maximum speed. This is a big part of the decision process. North American supply chains are slightly more expensive but also fast and responsive. The advantage of someone like Dell is also that they can do almost everything in-house, which avoids the overhead (to the business) of coordinating integration of components manufactured in Taiwan, the US, etc into the final build. I’ve done it both ways. There are real tradeoffs so context matters.
- pclmulqdq 1y ago
  I am pretty sure that at scale, the savings is so great that this is when you will tolerate a worse experience for cheaper gear. Hence Google and Facebook pinching pennies. The advantage of buying Dell (or IBM) is that if you aren't buying enough computers to have your own dedicated people, you get someone to take care of you. Dell may be able to get you stuff faster, but the Taiwanese shops are also very good at having an agile supply chain.
electroly 1y ago
Nobody ever got fired for buying Dell with ProSupport. I buy Dell servers at work because it's not my money but it is my ass on the line if something goes wrong. The quotes you get from your rep look nothing at all like the retail pricing on Dell's website.
- jjice 1y ago
  Is this the support I've heard about from Dell? I've always heard about it from a consumer perspective, but I've been told that they'll come out to you same or next day for hardware support? If that's the case, that's pretty damn impressive.
  - electroly 1y ago
    Yes, they have both next-day and same-day on-site support plans. For hardware failures they will bring the part to your office/data center and perform the replacement. We're in a third party colocation facility and Dell is happy to dispatch techs there; we don't have to be present.
    - tracker1 1y ago
      Had this for a work laptop a few years ago... That said, I think it was a design flaw in that model, had to have the MB replaced twice under warranty, the third time it was out of warranty and I just got assigned a new laptop.
  - criddell 1y ago
    My daughter took her Alienware laptop to school with her. The keyboard broke and we contacted Dell support hoping to just buy a new keyboard. Instead, they sent a technician to her dorm next day who fixed it on-site. What really impressed me was that we bought the laptop in the US where we live and she was going to school in Canada.
    - jlund-molfese 1y ago
      That’s really cool! Was this just with the standard warranty, or did you have to get some sort of extended support plan?
dfox 1y ago
My experience with Dell is that they are not that focused on selling enterprise support (at least compared to HPE), at most they will push for bundling hardware (cables, cable trays, front covers, PERC...) that you do not really need in order to get better volume discount. Price-wise I don't see a meaningful difference between Dell and SuperMicro (or even "non-traditional" server vendors like Asus and Gigabyte).
moondev 1y ago
To me the OOB/BMC interface of the server is the main tangible difference not the support contract implementation. IDRAC is miles ahead of SMC/MEGARAC. It's not even close.
psds2 1y ago
In todays market I think there is a lot more consideration to speed of delivery over price. Dell has historically been NVIDIAs #1 OEM partner so it would not surprise me if Dell has supply chain advantages over other server vendors.
cge 1y ago
For academia, I've found that Dell seems to seek and get sole-supplier contracts with universities, so research groups are forced to use their grant money to buy from Dell, often at inflated costs.
jaas 1y ago
We buy them because our experience is that they are extremely reliable and their iDrac management system is better than the alternatives, which saves us time (thus money). Maybe they aren’t the cheapest at initial purchase, but less maintenance and the ease of administration makes up for it.

throwaway48476 1y ago

Probably iDrac. AI servers are mostly just white boxed nvidia reference designs. Not much markup.

bluedino 1y ago
HPE is finally moving away from their 'Cray' systems which are just whitebox AMI systems. You get none of their iLO etc even though they are 'HPE'

alephnerd 1y ago

> buy from companies that are lower-cost and less focused on selling you "enterprise support" (supermicro et al). From past experience, vendors like SuperMicro aren't much cheaper at scale because Dell can give 80% discounts on volume, and most Dell server sales are done via Channel with an MSP who will manage and administer the system. Ofc, this continues to reinforce my belief that the "AI boom" in it's current form (and once you remove the scooby doo monster mask) is basically a "Datacenter/Telecom Bubble" 2.0 like back in 2000-01. The multiples, messaging, vendors, and margins are almost the exact same. During the Dot-com Boom, Telecom and hardware companies were also expanding DC and telecom capacity massively (to surf and host a website on the Internet, you kinda need Internet and web hosting), and when the Dotbomb happened, the Telco bubble collapsed subsequently as well (remember WorldCom/MCI?) That's why the 2000s were horrible for anyone with a CS/CE/CSE/EE degree, because both software AND hardware industries collapsed. Imagine a world in the late 2020s where you cannot land a job as a Fullstack Engineer OR an ML Infra Engineer - that was the 2000s except with older stacks. That's also why I'd be optimistic if a bust happens - the overcapacity in compute that arose from the Telecom and Dot-Com Busts both helped usher the Cloud, SaaS, E-Commerce, and Social Media boom because the infra has been laid and became cost effective. It is also in this context that Paul G's "cockroach" and "ramen profitability" essay came to the fore.

oblio 1y ago
The thing is, the bust froze everything for 1 or 2 years, which sucked for regular IT folks plus this time there also a huge oversupply of IT graduates. I wouldn't be surprised if there are 10x more people working in IT then back then.
- alephnerd 1y ago
  Mind you, I was just a kid when it happened so I'm basing it off of 2nd hand stories, but my understanding was anyone with a pulse (ie. A hs grad or a college dropout) could get hired into the industry at the time (eg. The KOTH joke that if Peggy was laid off from being a substitute teacher she'd learn to design software). At least ime in the 2010s and 2020s, while you had some aspect of that with bootcamp grads, it wasn't to the same degree. Or was it? Like I said, I was a kid when the busts happened so I have no frame of reference.
  - shermantanktop 1y ago
    I had a pulse and got into this industry in the 90s through the back door. It’s been a long run and I’ve seen busts come and go. It’s worked out really well for me, even while I’ve watched a surprising number of highly-credentialed new hires flop out over the years. I continue to appreciate people with oddball resumes.
    - tracker1 1y ago
      Similar here, no formal higher education starting in the early-mid 90's, just a lot of reading/learning and experimenting along the way... Even while working, I've spent 15-20 hours a week most of my career on learning activities. Somewhat hardware oriented and worked a few tech support/helpdesk and internal IT jobs in the early-mid 90's when I got more into programming as a side-step from design/art.
firesteelrain 1y ago
> That's why the 2000s were horrible for anyone with a CS/CE/CSE/EE degree, because both software AND hardware industries collapsed Odd, anecdotally I remember people saying that, but had no issue getting a job in 2004 as a fresh CS grad.

axus 1y ago

For the international hardware warranty transfer.

bluedino 1y ago

You want enterprise support. GPU quality is atrocious, you will have your Dell tech in there replacing GPUs and fans all the time.

c0balt 1y ago
Can second this, the amount of GPU failures we have with Lenovo systems on just <50 nodes is significantly higher than we expected. Having a Lenovo support person at least twice a month on premise at the middle of the bathtub curve is probably also costing them (and implicitly us) a good chunk of money.
grubbs 1y ago
Interesting. I work in higher ed and we have thousands of GPUs under my team. Rarely ever seen a failure. Mostly when we put consumer grade GPUs in servers (Nvidia doesn't like this). True server-grade GPUs never have any problems.
- ecshafer 1y ago
  IS this for some kind of HPC cluster? What kind of utilization are you at? For an AI company these GPUs are going to be at near 100% utilization 24/7. These kinds of loads destroy hardware quick.
- bluedino 1y ago
  Every site I've worked at has plenty of GPU failures. Not consumer grade either, H100/A100

jandrewrogers 1y ago

At scale, the savings of slightly cheaper hardware is often dwarfed by supply chain management costs, both direct and indirect. How much is it worth to your business to get the hardware deployed six weeks sooner and/or with a high probability of timely delivery? I imagine xAI wanted to move with maximum speed. This is a big part of the decision process. North American supply chains are slightly more expensive but also fast and responsive. The advantage of someone like Dell is also that they can do almost everything in-house, which avoids the overhead (to the business) of coordinating integration of components manufactured in Taiwan, the US, etc into the final build. I’ve done it both ways. There are real tradeoffs so context matters.

pclmulqdq 1y ago
I am pretty sure that at scale, the savings is so great that this is when you will tolerate a worse experience for cheaper gear. Hence Google and Facebook pinching pennies. The advantage of buying Dell (or IBM) is that if you aren't buying enough computers to have your own dedicated people, you get someone to take care of you. Dell may be able to get you stuff faster, but the Taiwanese shops are also very good at having an agile supply chain.

electroly 1y ago

Nobody ever got fired for buying Dell with ProSupport. I buy Dell servers at work because it's not my money but it is my ass on the line if something goes wrong. The quotes you get from your rep look nothing at all like the retail pricing on Dell's website.

jjice 1y ago
Is this the support I've heard about from Dell? I've always heard about it from a consumer perspective, but I've been told that they'll come out to you same or next day for hardware support? If that's the case, that's pretty damn impressive.
- electroly 1y ago
  Yes, they have both next-day and same-day on-site support plans. For hardware failures they will bring the part to your office/data center and perform the replacement. We're in a third party colocation facility and Dell is happy to dispatch techs there; we don't have to be present.
  - tracker1 1y ago
    Had this for a work laptop a few years ago... That said, I think it was a design flaw in that model, had to have the MB replaced twice under warranty, the third time it was out of warranty and I just got assigned a new laptop.
- criddell 1y ago
  My daughter took her Alienware laptop to school with her. The keyboard broke and we contacted Dell support hoping to just buy a new keyboard. Instead, they sent a technician to her dorm next day who fixed it on-site. What really impressed me was that we bought the laptop in the US where we live and she was going to school in Canada.
  - jlund-molfese 1y ago
    That’s really cool! Was this just with the standard warranty, or did you have to get some sort of extended support plan?

dfox 1y ago

My experience with Dell is that they are not that focused on selling enterprise support (at least compared to HPE), at most they will push for bundling hardware (cables, cable trays, front covers, PERC...) that you do not really need in order to get better volume discount. Price-wise I don't see a meaningful difference between Dell and SuperMicro (or even "non-traditional" server vendors like Asus and Gigabyte).

moondev 1y ago

To me the OOB/BMC interface of the server is the main tangible difference not the support contract implementation. IDRAC is miles ahead of SMC/MEGARAC. It's not even close.

psds2 1y ago

In todays market I think there is a lot more consideration to speed of delivery over price. Dell has historically been NVIDIAs #1 OEM partner so it would not surprise me if Dell has supply chain advantages over other server vendors.

cge 1y ago

For academia, I've found that Dell seems to seek and get sole-supplier contracts with universities, so research groups are forced to use their grant money to buy from Dell, often at inflated costs.

jaas 1y ago

We buy them because our experience is that they are extremely reliable and their iDrac management system is better than the alternatives, which saves us time (thus money). Maybe they aren’t the cheapest at initial purchase, but less maintenance and the ease of administration makes up for it.